It was supposed to be more fun than this.

Big Data — sweet, rich, untapped data — was supposed to be the great equalizer. McKinsey declared it the “next frontier for innovation, competition, and productivity.” Every organization would have unparalleled insight into the opportunities and risks within their business.

A decade later and the signs of frustration are mounting. A recent report found that only 41% of firms are competing based on their analytics, while 39% are managing data as an actual business asset, and merely 24% claim to have built a data-driven culture.

What happened? How did the big promises of Big Data turn into lackluster results?

There are no shortage of databases, dashboards or even alerts. Anyone who wants to visualize the state of their business can. But what do you do when those visualizations fail to show you what to do next??

All the dashboards and prediction algorithms in the world are ineffective unless they produce actual money either flowing into or kept by the business. Ostensibly, that’s what Business Intelligence and Analytics (BI&A) is for.

But stand up in a meeting armed with only charts that say “sales are down,” and you quickly recognize there’s no direct line to ROI. At the end of the day, analytics are about searching for answers to questions you’ve thought to ask, not about turning data into dollars.

That is changing. AI, particularly unsupervised learning, has emerged for the masses to provide a different way to analyze even extremely complex data. This is leading to the rise of a new category of analytics — Data Capitalization Management (DCM). Unlike traditional BI or analytics, DCM platforms are focused on the tangible, realizable value for the businesses using them by automating the analytics process .

The Complex Data Explosion

Let’s go back, way back.  One of the earliest and still widely used analytic tools, the spreadsheet, was invented in the era of the floppy disk with a whopping 1.44MB storage capacity. Fast forward a few years to business intelligence reporting and dashboard tools that arrived when 25GB of storage was revolutionary.

The exponential growth of data makes those days seem quaint. How much data depends on who you ask, but let’s go with IDC’s projections that more data will be created between 2020 and 2023 than in the previous 30 years.

That kind of volume is hard to contemplate, much less analyze. Yet, volume isn’t the problem. It’s part of the bigger problem – data complexity.

As more channels and data sources have come online, the vast majority of the data generated isn’t well-structured or well-understood. Most of the world’s data is unstructured from text, audio, images which is incredibly complex.

Plus, it’s spread out. The average company is pulling from 400 data sources. That’s average. Many are spread out across 1000 data sources.

Buried in the data are the insights needed to not only run the business, but to change the business. You’d have to dig through an exponentially growing layer of complexity to find insights— one by one.

Using traditional AI and tools, this has largely been done by an overworked and under-resourced centralized data team. Most of their time is spent on preparing the data. As much as 80% of data scientists time is spent on massaging the data, which just so happens to be the part of the job they find least enjoyable.

This generates a frustrating scenario for most organizations: the data teams are frustrated with the high quantity of demands from various teams and business teams are frustrated with the lack of insights to act on.

That’s bad enough. As data complexity exploded, something else was happening.

The Rise of the Digital-First Companies

It’s tempting to say the data complexity problem is a universal problem. The reality is that a select few organizations got ahead of the complex data curve.

These high-performing companies saw what was coming, and invested deeply in turning that complexity into their competitive edge within their respective markets. These weren’t companies with huge digital transformation projects in play; they were the ones transforming the digital world. And they’ve greatly outperformed the market.

Digital-first companies built a culture around uncovering hidden insights buried in the most complex data, permeating the entire organization — from which products they make and where they sell them to how they market and how they hire — with insights that guide human decision making, not replace it.

What gave them that leg up? Did they have better tools? Well, not really.

They created armies of data workers. There are more than 1,000 open data scientists roles at Facebook currently posted on LinkedIn. There are thousands of data scientists at Microsoft and Amazon, according to LinkedIn.

The digital-first advantage is not purely due to resources. It’s cultural. These are organizations that look at complex data as a business asset, rather than a liability where most data sits unused with the majority of the investment going into managing, storing and securing it.

The investments in data science are difficult for even large enterprises to match. But it’s the culture of ensuring each team — sales, marketing, supply chain, finance, customer experience, etc. — have real-time access to truly actionable insights that holds even the best intentioned organizations back.

There needs to be a different way.

This is where Data Capitalization Management comes in.