Mining The Data Universe: Delivering Intelligence From Chaos

Data is wonderful, but baffling. Making order from it is the key to a predictable and stress-free future.

Share this article

Share this article

Data is wonderful, but baffling. Making order from it is the key to a predictable and stress-free future.


Mining The Data Universe: Delivering Intelligence From Chaos

Data is wonderful, but baffling. Making order from it is the key to a predictable and stress-free future.

Share this article

From the number of stars in the universe to grains of sand on a beach, metaphors fail to quantify the vast amount of information that businesses have to contend with today.

With the size of our digital universe doubling every two years, even the most creative comparisons are rapidly dwarfed by yet more data. In fact, by 2020 it is estimated that the data we create will reach 44 trillion gigabytes.

Disorder on a vast scale

Individually, corporate giants like Walmart will collect upwards of two petabytes of data each hour, the digital equivalent of 50 million filling cabinets.

And that’s just the data they’re specifically pulling in themselves. Unlike filing cabinets, most of the data that’s out there on the internet – the millions of articles, tweets & blogs - isn’t structured, making it even more challenging to interrogate and extract meaning from.

File cabinet

There aren't enough filing cabinets in the world

Roughly 80% of all the data available today falls into the “unstructured” bracket. For those inside large businesses looking out, distilling timely intelligence from such a fast growing data landscape is an issue that warrants a great deal of attention.

As a result, any business that needs to develop a clear understanding of its risks and opportunities faces tough challenges. The most obvious is that, with so much data available, it is extremely tough to define and understand the full extent of any individual risk as well as the full risk landscape faced by a business.

And that’s before they come to then evaluate the significance for the brand, business model, supply chain, stakeholder relations, ability to innovate, as well as to protect and enhance reputation and shareholder value.

At worst, the unprecedented, and exponentially growing risk landscape creates a smokescreen of information in which it becomes almost impossible to distinguish the simmering issues that could become tomorrow’s headlines.

It’s a situation made  more precarious by social media, and the unpredictable and visceral way in which issues can be elevated to global attention at speed. Blind spots in this context are particularly hazardous.

Augmented humanity

But technology continues to show us that no problem is without a solution. The more complex the problem, the more intricate the solution required, but despite the size of the challenge, making sense of data is a battle businesses are winning. Here’s how.

Where technology adds the most value is through the large-scale collection, sorting, filtering and surfacing of issues, conversations and warning signs that matter, so that human intelligence can then step in, make decisions and take action to mitigate risk and potentially even transform it into opportunity.

But this is no small task as it requires an adept mastery of human linguistics in relation to risk: a careful appreciation and curation of the many words we choose to express concern, anxiety, outrage or hope in formal and informal contexts and in different languages and cultures around the world.

Road works in progress .

Computers spot the problems, humans deal with them

While algorithms have come a long way since the days when sentiment – and the presence of a smiley face or not – was the pinnacle of linguistics analysis, people and all their idiosyncrasies remain notoriously hard to interpret at a scale that makes  understanding useful.

The syncing of linguistics with big data through the application of complex algorithms means that the galaxy of unstructured data can now be distilled into intuitive real-time dashboards and visualisations that provide accessible intelligence into the issues, voices and risks that have a direct business impact.

It’s the scale that is so vital to those concerned with risk. For even if businesses had the resource to apply ten thousands brilliant minds to the task of information analysis, in the time required to work through millions of sources of unstructured data, and then translate and compare results, the issue may have already come and gone.

Instead, with the help of technology, intelligence from millions of sources of unstructured data can be distilled and delivered in real-time, and provide business with live analysis of key stakeholders and public opinion.

At the same time it allows the tracking of thousands of assets and entire supply chains for risk exposure, strategic issues trending and performance benchmarks against competitors in different languages and markets – and all just a click away

What’s more, with technology always operating at 100% capacity, little can slip, or be missed.

No more surprises

By developing a clear understanding of everything that might constitute the risk landscape for a particular business, algorithms can be levied to monitor for red flags around the clock, and alert senior decision makers whenever a hint of hazard appears, allowing humans to step in ahead of time and take the best course of action to mitigate risk or even create opportunity from uncertainty.

Technology has now reached the point that it can relegate most nasty surprises to the annals of history.

Related Articles
Get news to your inbox
Trending articles on Opinions

Mining The Data Universe: Delivering Intelligence From Chaos

Share this article