How Machine Data and Operational Intelligence Can Supercharge Business Applications, Part I

Imagine that all your business apps were people, seated in an auditorium ready to hear a speech. Here’s what I would say to them about machine data:

Imagine that all your business apps were people, seated in an auditorium ready to hear a speech. Here’s what I would say to them about machine data:

“Business apps, meet machine data. You are the stars of the show, the reason that any company has IT in the first place. Business applications capture the data that runs a company, automates processes, and fuels decisions.

“Most of you were created when information was scarce and the world moved more slowly, when real-time apps were only relevant in the defense department, on Wall Street, or in manufacturing.

“Now mobile devices, sensors, and systems of all types are monitoring our world. The data they pump out, machine data, is a rich trove of information that can help you do your jobs.

“It is time you started using machine data, but it is a different animal. It comes in huge volumes, it has a flexible structure, and it is available in real-time.

“To make this data work for you, you need a data fabric to sort it out, distill it, analyze it, recognize events, and deliver it to you. I call this type of information processing Operational Intelligence. Throw the data fabric over machine data, and it will help you become better business applications. Let’s get started.”

Machine Data: A Technology Leadership Challenge

Forgive my enthusiasm, but it has taken me the better part of a year to figure out how to explain the potential of machine data, how it will change business applications, and what a data fabric like Splunk does to help.

One of Early Adopter Research’s goals is to help CIOs and CTOs become better leaders. Making business applications more powerful is one of the best ways to lead. In the next five years, we’ll see a parade of victories in which machine data sources provide insights of a whole new variety that have significant business impact.

Machine data is an arena crying out for technology leadership. The business needs help in understanding what machine data is and how it can create value. Technology leaders have the skill to identify machine data, demonstrate its value, and provide the business the means to explore it.

What an Elevator Can Tell You

At Spunk .conf 2012, a Japanese real-estate company described what they were able to learn by reading machine data from their elevators. By tracking how often the elevator stops on each floor, the company created a predictive model for lease renewals. A decline in elevator traffic might mean a decline in business, and an increase indicates the opposite. This data provides important advance notice to leasing agents about what spaces in a building may become available. (I’m working on a project to collect examples of business use of machine data. Please send them along.)

What is Splunk? Version 2012

The leadership question that interests me is this: Is it possible to systematically improve a business by running a formal program to increase use of machine data. I think the answer to that question is Yes and Splunk provides the needed functionality.

Most people outside of the tech industry know of Splunk as a recent tech IPO that didn’t get embroiled in a post IPO kerfluffle about pricing. Unlike LinkedIn or Facebook, which were criticized for under- and over-pricing respectively, Splunk went public, provided a decent bounce for investors, and went on its merry way.

Google for Machine Data

In the tech community, especially in the data center, Splunk is known as a platform for aggregating and searching log files that is popular with system admins. “Google for machine data” is a most popular one-liner.

But the data center is just the beginning for machine data and Splunk. The biggest impact by far comes from business insights that provide a unique edge.

My team at Early Adopter Research and I have had the pleasure of getting to know Splunk over the last few years. Among other things, we helped David Carasso, Splunk’s Chief Mind, write Exploring Splunk, a book that explains Splunk’s Search Processing Language (SPL) along with some recipes for putting it to use.

At Splunk .conf 2012, it became clear to me that eventually Splunk will be known as:

• A data fabric that allows for rapid indexing, exploration, and analysis of machine data.
• A platform for creating operational intelligence applications.

In the next article in this series, I’ll explain what I mean by data fabric and operational intelligence and describe how they will transform business computing.

Author: Dan Woods

My mission: Find technology for Early Adopters. Follow me on Twitter @danwoodsearly and on LinkedIn www.linkedin.com/in/danwoodsearly/. I am a CTO, writer, and consultant. For tech vendors, I help explain their technology. For users, I help find, select, and deploy new solutions that have explosive business value. I love to speak and share ideas.