Big data is the electricity of the 21st century – a new kind of power that changes everything it touches. But big data is still a bit of a mystery. Get four CIOs in a room and you’ll have 5 opinions about what big data is, what it does, and how to make it useful. The early days of electricity were the same.In fact, at one time, it was a mystery whether electricity created in the lab was the same thing as lightning in the sky.
On a hot and humid night in 1752, Benjamin Franklin ran a now-famous experiment of flying a kite in a thunderstorm. Franklin added a wire to the kite so it stuck out the top and attached a metal key to the bottom of the kite’s wet string. When he saw fibers of the string stiffen, he moved his hand close to the key and sparks jumped from the key to his knuckles. He then captured some of this electricity in a Leyden jar (an early kind of battery), brought it home and proved the stuff he caught was the same as the electricity he had generated in his workshop.More than just curiosity drove Franklin. He was trying to solve a problem – how to prevent lightning strikes from burning up buildings. Now that he had proven lightning was the same as the electricity in his experiments, he knew how to conduct it away. The first lightning rods were installed on tall buildings in Philadelphia that summer.
Over the years, the work of many other inventors, like Nikola Tesla and Thomas Edison, turned Franklin’s discovery into one of the most useful substances humanity has ever known.The electrification of everything opened the floodgates of innovation in business, government, and private life
The datafication of our thoughts and opinions. For example, in just one minute, Facebook users share over half a million pieces of content (Source: http://www.visualnews.com/2012/06/19/how-much-data-created-every-minute/)The datafication of things through sensors collecting information from cars, medical devices, stop lights, and factory equipment. “a single jet engine can generate 10TB of data in 30 minutes. With more than 25,000 airline flights per day, the daily volume of just this single data source runs into the Petabytes.” http://www.oracle.com/us/products/database/big-data-for-enterprise-519135.pdfEven the natural world is being datafied through the use of satellite imagery and climate sensing devicesWe’ve been datafying enterprise processes for decades. Now we’re datafying personal life activities, too. You no longer have to go to the bank to deposit a check; you can use your phone. Running on the treadmill at the gym? Chances are good you might be wearing a heart-rate monitor or a Nike Fuel Band to track your physical activity
All this new data offers tremendous potential to change the way our organizations do businessWhether it be capturing the thoughts and opinions of our customers to create better marketing campaignsUsing sensors to manage buildings, capital equipment, and improve maintenance and service costsOr streamlining processes based on new data insights, the possibilities are endless
The problem is the world’s ability to produce data has outstripped most organizations’ ability to use it.One of the largest airlines in the world, employing dozens of operational research analysts, throws away most of its fleet operational data at the end of the day because it’s so big there’s nowhere to put it and analyze it. (Source: a presentation by Jim Diamond, Managing Director of Operations & Research at American Airlines. Given at the Evanta CIO event in Dallas, TX 6/7/13)The same is true for many other businesses: the information they need to improve products and services already exists, they’re just not quite sure how to use it.According to a study we conducted with The Economist Intelligence Unit, only 12% of executives feel they understand the impact data will have on their organizations over the next three years.” (Source: http://www.oracle.com/webapps/dialogue/ns/dlgwelcome.jsp?p_ext=Y&p_dlg_id=13367869&src=7634271&Act=143 )
Let’s start with what exists. You already have analytics you use to run the business – data warehouses, reports and dashboards. You carefully select and standardize the data you need to solve specific problems, like running a marketing campaign or billing customers.
This is a powerful way to reduce the time, cost and effort of standardizing and controlling processes to run the business. The data coursing through this enviroment will increase in volume and velocity – it will get bigger and arrive faster than today.This is the relational analytical environment we’re all familiar with. You’ll need more processing power to handle this, but that’s pretty straightforward.
But that’s not all. The datafication of everything opens a new possibility – the possibility to learn from the data in new ways. There’s a huge amount of available data, but it’s not always clear which data might be useful to you or what you might learn from itExamining the data in a non-relational environment and letting it tell YOU what you can learn from it, cuts the time, effort and cost of forming and testing hypotheses.
We call this the “Change the Business” approach because new ideas uncovered this way often lead companies to make changes, or pivot processes and systems in ways they wouldn’t have attempted otherwise.
The critical difference between the run-the-business and change-the-business environments boils down to one thing:To run the business, you organize data to make it do something specific; to change the business, you take data as-is to figure out what it can do for you.Relational technologies excel at the first, non-relational technologies at the second.
These two approaches are more powerful together than either alone.Combining the two gives you continuous innovation. This is Big Data At Work.This, also, has happened before. In the early days of commercial electricity, there was a huge standards war – alternating currrent vs direct current, AC vs DC. AC could be transmitted over long distances, but its high voltage made the wires deadly if you touched them. DC wires were safe to touch because of their low voltage, but could only deliver power a mile or two from the source. It was a bitter fight. But in the end, the solution was to make them work together. AC now comes out of every wall socket. DC comes out of every battery at the heart of every mobile device. The two together are far more powerful than either alone.But to bring a non-relational environment into the corporate fold, it has to have the same basic capabilities as the relational environment the company already counts onIt has to acquire, manage,and analyze, whatever data happens to be in it, just like the traditional relational environment
If you had Big Data At Work you couldGet fast answers to new questions Predict more, and more accuratelyCreate a reservoir of data for potential reuseAccelerate data-driven actionLet’s look at a couple companies realizing the benefits of big data at work
So what is the next step for your organization?Maybe you’re in need of a change the business big data environment Or perhaps you are looking to add processing power to your current run the business big data warehouseRegardless of where you find yourself, let me give you three ideas to consider as you build your big data strategy
Look beyond the value of data captured from the daily activities of your organization to the extended value chainEverything every player in your industry does produces dataWhat share of the data available does your company have access to?You might need to consider buying external dataOr extending your capabilities to capture more proprietary dataThink of this as an investment in a powerful, revenue generating source of energy for your organization
Even an investment in buying or capturing data available to your competition can give you a competitive advantageBy mixing publicly available data with your proprietary data, you make the entire collection proprietary
And using proprietary data can give you greater data market shareThis is a staple internet strategy:Google uses data from search result click-throughs to refine results for the next person who searches for the same term; Amazon uses data about customers’ past purchases to create product bundles and recommendations for other shoppersUsing data to make data opens up a first-mover advantage, and is effective in even the most traditional industriesFor example, a shipping company can use package-level sensor data to cut the cost of handling perishable goods, opening up the service to a new marketBy capturing the data exhaust from new customers using the service, they are able to refine the service to better meet customer needsThe cycle of data capture and use creates a competitive advantage that is very difficult, if not impossible for rivals to catch
In the end, we must return to where we began. To electricity.In the early days of commercial electricity, the 1893 World’s Fair was lit up with about 200,000 electric lights. No one had ever seen anything like it. This was electricity at work. But outside the fair, the world was still mostly dark. Electric cars and power grids were still in the future.For all the bright spots of big data we’ve seen so far, the world is still mostly dark. It waits for your ideas, your new uses for big data. And we look forward to creating that new world with you – a world of Big Data at Work.