Palo Alto Network’s Naveen Zutshi on how to make the CIO a data-driven hero

Let’s face it: for some time now, the IT department has not practiced what it’s preached for the rest of the business: Invest in gathering, synthesizing, analyzing, and operationalizing data to improve the business.

For the same reason that it’s been difficult for the business to follow its advice, it’s also been hard for IT. The challenge for both sides of the fence is that the number of data sources is growing rapidly and the ways of analyzing that data have grown just as fast. In addition, many of the capabilities are provided by developer or data science level tools, which turns the exploitation of data into a development project, leading to yet more bottlenecks.

What is increasingly saving the day is productized analytics that go beyond the horizontal platforms introduced for data preparation, integration, and discovery, which provide a new level of ease of use for data that was formerly trapped inside systems of record and the data warehouse. Now, a variety of solutions are more vertically oriented and deliver a much more detailed model of business activity and associated analytics, so that companies can start with a huge amount of work already done for them and build from there.

The challenge for CIOs and other executives is that it’s important to understand what a high resolution IT-driven data model would look like and how you’d use it to better manage the business from both a management and operational perspective. This would allow you to achieve the efficiency that the world of modern data promises.

The best way to learn these things is to study people who have figured it out and copy them. Recently, I had a fascinating talk with Naveen Zutshi, CIO of Palo Alto Networks, who has been a pioneer in the advanced use of data to manage IT and achieve better outcomes. This article encapsulates, at a detailed level, his recommendations about how you can use data to better manage IT.

Zutshi acknowledges that IT has not done a great job of looking at itself as a business and understanding and tracking its activities at a detailed level, along with measuring the value of the investment in IT. His remedy is to focus on six areas that build on the increasingly large amount of data captured by IT and then provide detailed answers and visibility to allow you to constantly improve performance.

“We need to think about IT just as we would any other part of the business,” Zutshi told me.

Focus Area 1: Are you more stable or less?

In assessing how well your IT is performing, you need to be able to judge the stability of your applications. To do this, Zutshi highly recommends using availability metrics that include measuring the number of changes made by IT and the impact of those changes. For development, you want to measure how many incidents were introduced by a new change or by a new feature being released.

Additionally, Zutshi recommends that companies measure performance from the customer’s perspective. That means measuring what the customer is experiencing when it comes to performance. “You need to look at how the customer is interacting with that application, what happens by introducing a new feature, and whether customer experience improves or degrades,” he said.

Focus Area 2: Are you providing more or less value over time?

The second area Zutshi recommends companies look at is the value IT is providing to the business. Zutshi and I agreed that this is one of the most difficult things to measure about the impact of your IT.

“The quantitative aspect of measuring value is really, really difficult,” he said. “One of the reasons is conflation. So let’s say you made a CRM change where you added sales stages and you introduced better discipline in sales stages and now you saw a bump in revenue.  Now, was the bump in revenue because of the process change you made, was the bump in revenue because of the technology, or was the bump in revenue because of macroeconomics? It’s hard to know for certain.”

Zutshi said that one of things he has found effective is to break up the business into three areas based on the Gartner model of Run, Transform, and Grow.  

• Run: “On the run side, we want to optimize our run costs while providing high availability, high stability, and high performance.” he said. “Across the industry, about 60 to 65% of IT spend is on run. If we reduce this to 45%, can we still provide the same level of quality? If so, we can reinvest those dollars in growth or transform opportunities.”
• Transform: With transformation, you’re tying your top three or five IT initiatives to the top business priorities and assessing the impact that IT had. This can include the effect of introducing new products to the business, to salespeople, and to customers. When this occurs, “we want to ensure that we are providing solutions on time, with quality, so that our salespeople can sell and our customers can consume the new products that we introduce. It’s an easy measurement both in terms of the backlog and whether we effectively provided those products on time to our market,” he said. While IT can’t claim credit for all value derived from a new product, if the product is successful, IT does deserve some. Additionally, he said that companies can assess IT’s transformational impact by looking at indicators such as macro level sales productivity and how the use of machine learning is helping the company be more agile.
• Grow: Zutshi acknowledged that one of the key reasons IT often falls short is not just that they are not producing value, but also because there is the perception that IT is too slow and impeding the progress of others. That’s why he focuses heavily on growth. “We work with each functional leader and determine what their initiatives are for the year and for the quarter and then prioritize those within the function, rather than having to prioritize them across all the functions.  That way, at least their business priorities are getting done, and from an IT standpoint, we are demonstrating agility and demonstrating value from that agility.” Thus, the measurement here is more of a perceptual one than a quantitative figure. But you can then measure your effectiveness as being an agile organization that is supporting the business needs. Additionally, Zutshi said he uses software from a system of intelligence vendor to measure a net promoter score for internal IT that includes IT performance with stakeholders across the company. They do this forkey stakeholders at the VP level and above as well.

Focus Area 3: Does the customer feel quality is improving?

Zutshi said this is one of the easier areas to measure. He recommended measuring:

1. The availability of systems over time
2. The number of open issues in a project or enhancement before it is put into production
3. A completeness measurement for apps, which is whether there are changes because the project was not correctly scoped from the get go
4. Technical and business debt

An agile approach is important to minimizing technical and business debt. “This is an area where we have to educate our business. From the customers’ perspective, they may see that a feature that should have been included from the beginning wasn’t,” he said. “If we are doing iterative development on a given product, customers will be accepting, knowingthat more features will be added. But if you have a project-based mentality and you finish a project and leave some things incomplete and move on to something else, then you are carrying additional technical and business debt.  A key aspect of quality is to measure the technical and business debt,” he said.

Focus Area 4: Identify app rationalization opportunities

“We are driving toward a data-driven approach to app rationalization,” Zutshi said. “On average, what I’ve heard is that most companies have 500-plus applications in each of their environments.  And while we don’t have that many in a company of our size, we still need to constantly look at rationalization.”

To do this, he recommended looking at:

• Adoption metrics for apps
• Incident and change metrics
• Application performance KPIs and metrics drawn from one or more APM tools
• Renewal cost of the application

He said he uses a system of intelligence to combine those metrics to demonstrate to the stakeholder why an application needs to be rationalized in favor of another application that might be a duplicate or the way a feature may be subsumed in another application. He said they also look at whether the work done by one app can be duplicated in another that already exists.

Focus Area 5: Measuring the impact of change

Zutshi has talked about the importance of measuring the impact of change in some of his other focus areas, but believes it deserves its own category as well. Thus, part of this would include the customer experience and satisfaction criteria outlined earlier. But it would also include measuring:

• The drivers of change failure, such as experience level
• The track record of the team implementing the change
• The type of environment that the changes are being made to
• The estimated risk assessment level

Focus Area 6: Measuring application development performance

Zutshi’s final area for companies to focus on to improve IT was measuring application development performance over time. He said that it’s key to tie app dev performance to release planning to make sure that projects that require configuration, integration, or development effort can be analyzed to reduce project execution risk, and then shorten lead times, enabling IT and its customers to realize the benefits of rationalizing.

“Agile projects benefit by directly observing the quality and performance of applications in the production environment and by enabling rapid resolution of escape defects,” he said. “We always think about how we can reduce the number of escape defects, which are the defects we didn’t observe until they became apparent in the production. They are the things that leaked through normal testing.”

How Systems of Intelligence Assist Companies in all Six Areas

Following Zutshi’s overview of these key categories to measure, I asked him about why he selected the productized analytics solution for IT to achieve these goals.

“For tracking the measurements, we realized we could build it ourselves using a Hadoop cluster with a set of dashboards in Tableau, and pull in all the data from ServiceNow and APM tools like AppDynamics. But we found that the prebuilt applications made it easier for us to deploy those dashboards faster and get people analyzing acting on the data rather than spending all their time pulling it together,” he said. “The other reason the systems of intelligence solution helps is that the entire team is utilizing those dashboards to ask their questions. In our company, each leader within IT has their set of questions to manage their department appropriately and we want to make sure that they have a single application where they can ask those questions and get answers.”

Zutshi told me that nothing that his organization did is rocket science but I think that undersells his achievement. Measuring IT in this way may be obvious, but it’s so seldomly done that it’s notable when a business takes this step. IT has been treated like an art rather than a business for a long-time, but Zutshi thinks it’s time this changed.

“I think as CIOs, we have to demonstrate vulnerability and admit that we haven’t figured all this out and work at fixing it,” he said. “From my point of view, when I have this discussion on value creation and I ask my peers, how are you doing this in your organization, I get a variety of answers.  Wouldn’t it be great if we had a really strong practice associated with how to do that in a seamless manner, in a less bureaucratic manner, and also capture it well to report it consistently to our business? But we haven’t done that until now. It’s time we did.”

data management IT management Palo Alto Networks

Author: Dan Woods

My mission: Find technology for Early Adopters. Follow me on Twitter @danwoodsearly and on LinkedIn www.linkedin.com/in/danwoodsearly/. I am a CTO, writer, and consultant. For tech vendors, I help explain their technology. For users, I help find, select, and deploy new solutions that have explosive business value. I love to speak and share ideas.