Original content found on https://www.linkedin.com/pulse/big-data-old-news-we-need-meaningful-robert-iati
I spoke at the FIMA Canada event on September 21-22 and heard many of the presentations and panels, with talks represented by some of the best Institutions and vendors, academics and data managers. After it was done, I thought back on what I’d heard and found that—interestingly—all the content centered on one consistent theme.
We have data. Lots of data. BIG DATA. We are all data hogs. Addicts, really. But what do we do with it all? How do we optimize its use?
Big Data. Merely the thought of that term conveys the idea that, as firms that make their money trading, we need to take in all data that is available. The more data we have, the more we can feed into our trading algorithms, risk systems and compliance reports. To steal (& alter) a phrase from Michael Douglas in “Wall Street”, “Big is good”.
Our trading institutions have indeed taken in data…and have benefitted greatly from it. The revolution of electronic trading was founded on—and has been predicated on—having access to all available data. So too has risk management, while regulatory oversight depended on using greater amounts of data to achieve optimal effectiveness. As a result, the decision makers in capital markets are bent on acquiring and using as much data as is possible. To some extent, we’ve achieved this objective. We have it all now. All the data we want is close at hand, but still we look for more. We’ve spent hundreds of millions of dollars on data and technology to make it faster and more relevant, but not necessarily more meaningful.
How’d this come about? When it comes to Big Data, we are in a seemingly endless loop. We collect more data and make more of it available through new channels, from which we then go about collecting it. For example, internet blogs and news sites generate data at staggering rates. Hundreds of cable television channels and satellite radio stations and social media sources flood us with more data, sometimes unstructured, but now used in our decision making models. We improve our technology analytics ostensibly every day and, as we get more data, we find more instances of data that is useful to us. The easy availability of this data further feeds our curiosity about the value of more data. “More will be better”, we think.
We created technology to transmit, filter and scrub this data but, think about it, the automation that helps us manage more data also creates more data! Trading algorithms create new orders and cancel others. Social media scrapers generate new trading signals. We develop different ways to aggregate data so that we have more indices and predictive metrics. In fact, the U.S. Chamber of Commerce states that 90% of the world’s data has been created in the past three years and that 40% - 50% of all data created is created by technology itself! Data will always be one step ahead of technology.
But, at this time, is more data necessarily better?
I believe that we now have reached the point where most Wall Street institutions have too much data that is without clear definition or even true purpose. As an industry, we often feel as if we are not doing well unless we know EVERYTHING about the data. It’s our nature. But taking in so much data without finely understanding its purpose can leave institutions open to inefficiencies, and to the greater risk of drawing questionable conclusions from that data that may not be accurate.
We can’t know it all and we can’t wait until we do because we never will. The pace of change is too quick. We need data but, more importantly, we need to be comfortable with all the information and our ability to find unique and differentiated data and to leverage it intelligently. To optimize the data is to draw the best insight from it. That’s what makes it meaningful.
When we look at all the data we have, and all the models we create from it, we need to ask ourselves “what is missing”? What is the data that, if we had it, we would be able to overcome our greatest obstacles?
I believe we’d find that it’s the data on the opaque securities, the private companies, and the hidden linkages between them all. To improve our trading acumen, we’re looking for unique data to provide predictive signals of movement in a market, sector or name. We search to identify some trend in the private sector that can help to predict public markets’ activity. To increase transparency and diminish risk exposure, depth of insight into counterparty relationships and linkages reduces risk and allows firms to deploy capital with confidence. For more efficient data management, having the ability to more definitively link entities more precisely with reliable standard identifiers allows for great certainty in our EDM infrastructure, which improves efficiency of our operation. This unique data is out there but, in large part, needs to be harvested more effectively to be meaningful and to maximize its value to capital markets institutions.
Big Data is great. But the real insights are extracted from meaningful data.
It’s good to be BIG. But it’s better to be meaningful.
Dun & Bradstreet is headquartered in Short Hills, NJ with offices around the globe. Learn more about our Capital Markets offerings.