Introducing the Data-driven Society

Every year, the amount of data created by humanity increases exponentially. According to a report by the International Data Corporation, the total amount of data generated up until 2010 amounted to 1.2 trillion gigabytes. In 2020, that figure is expected to reach 40 trillion; in 2025, 463 trillion. This includes things we create manually, such as emails, photos and social media posts, but also sensor data generated by our devices, such as location tracking, temperature readings and health statistics.

To describe this new era of burgeoning access to information, computer scientist and MIT professor Alex Pentland coined the phrase “data-driven society” – that is, a society where almost every action and decision is underpinned by big data, providing potentially huge benefits to individuals as well as corporations. This vision goes far beyond how the amount of data will change society. It is also predicated on the assumption that data streams from different sources will be able to converge seamlessly, instead of information being restricted to silos and requiring humans to manually process and refine it before it becomes useful.

Effortless and secure data through algorithms

Peter Nyberg, Group Director of Risk & Credit at Dun & Bradstreet, likens the idea of a data-driven society to a car where the engine consists of the combination of data and algorithms – but where us humans remain in the driver’s seat. “I foresee a world where it’s both effortless and seamless, as well as inherently secure, to have the digital reflection we know as ‘data’ being processed by algorithms, in a way that improves individuals’ quality of life as well as empowering our society as a whole,” Nyberg says.

The end of mundane tasks

These improvements stem from algorithms taking over mundane tasks; supporting decision-making in both our personal and professional lives; optimizing the use of societal infrastructure; and maximizing the utility of scarce resources. Those mundane tasks can involve driving you to work and scheduling your meetings when you arrive, saving you all the hours you waste painstakingly syncing your calendars with other participants, only to discover the conference room has been double-booked once you arrive. When you get back home, the convergence of data from wearables and online resources will make it possible to automatically generate dietary suggestions based on what food is in season, your own preferences and health data, and availability in stores nears you.

Improved sensors in phones and wearables will also be able to monitor said health data in real time, alerting your physician if a problem arises. In the energy industry, algorithms will balance the scarcity of the resources themselves against consumers’ needs to manage the cost of their energy consumption – in addition to tasks like ensuring that an electric car battery is always charged in the optimal way in order to prolong its lifespan.

Keeping up with the shady elements

And in the financial sector, this increasingly data-driven approach will be an effective weapon in combating money laundering. “AML is not a needle in a haystack. It is worse than that. It squanders the efforts and intellects of everyone involved. And trust me, that intellect and those efforts are desperately needed to keep up with the evolving shady elements,” Peter Nyberg says. “The trick, of course, is not to assume that the data-driven society will change all of this. The shady motivations will remain. But what will change is that the effort required will be greatly reduced – if you save 99 percent of the effort where it’s not needed, you can use that energy to deep dive into the cases where it’s actually necessary. I think that is how it generally will work – humans will always do the corner cases. Also, the confidence that what needs to be done is actually being done in the right way, as well as the ability to verify this, will greatly increase.”

How to handle and interpret data correctly

However, several hurdles remain before the data-driven society can become a reality. One of the most important challenges is to ensure that all data also comes with the necessary metadata to make it possible to handle and interpret it correctly. “The world is overflowing with databases, data warehouses, extracts, transformations, data deliveries and API’s. Yet, it’s exceedingly rare that you can have just the data you need, right when you need it, in a form where it’s directly usable,” Nyberg observes. “It’s like the world ran on hundreds of millions of fundamentally different types of electricity, using hundreds of millions of different sorts of cables and no adapters.”

Quote block

Automatically tagging data with information such as where it’s from and what it’s for will enable computers to instantly understand and transform data as needed. It will also make it possible to “bring the algorithms to the data, rather than the other way around,” as Nyberg puts it. This is important for privacy reasons – consider how a company like Apple even today handles AI calculations directly on your phone, instead of shipping off your data to a remote server where you have no control over how or for how long it’s stored.

We'll get there in 3-7 years

Once these hurdles are overcome – it’s effortless to have the data needed where it’s needed, seamless to have algorithms operate on it, and inherently secure to do so – the implications are profound. And we are already well on our way to making this a reality. “On a technical level, we’ll get there in the next 3–7 years. Socially, it will first appear in smaller areas, then gradually grow and merge, over a long time period. But I’d expect my grandchildren will see the data-driven society as a natural part of their lives from an early age,” Peter Nyberg concludes.


Read the whole article "The data-driven society" by Alex “Sandy” Pentland