Once upon a time there were a great many data marketplace companies, large and small. Sadly, almost all of them failed. They all had the excellent idea of making data visible and available for companies that need to sell and distribute their data, by facilitating the collection and marketing of said data. What an amazing concept! A replacement for failed data lakes! The hype was extreme:
Data Marketplaces: The Holy Grail of our Information Age
— hackernoon.com July 19, 2018
So, enter 2020 and a reality check on the success of these much vaunted data marketplaces. Where are the players from the early ’10s nowadays?
Sources: Data marketplaces, we hardly knew ye, Datafloq, Crunchbase
The obvious answer seems to be there simply wasn’t a market for data marketplaces, but the real answer is far more nuanced. Generally speaking, many marketplace business models fail, but there tends to be at least one significant, successful marketplace for a given commodity. However, that’s not the case with data… so what gives?
Data is a commodity with many unique characteristics including:
None of these characteristics lend themselves well to a marketplace paradigm, which relies on heavy process standardization, ease of comparison, arms length interactions between buyers and suppliers and clear-cut value transfer. But, what is perhaps most interesting is that managing these characteristics means generating very high levels of trust, which is ironic given that trust is understood to be critical for any successful marketplace.
All marketplaces tend to drive a value proposition by optimizing for discovery and fulfillment. Amazon is a great example — find everything you need, one-click checkout and same-day delivery — but trust is an issue they also need to deal with. Trust is typically about the quality of the products and trust that it will arrive. Some marketplaces (e.g. Ebay) also trust that the consumer will make payment post-fulfillment.
In data marketplaces, trust is significantly more important and challenging to resolve. Users need to trust that:
…that’s a big ask! As a result, many highly inefficient norms have been established over time to deal with the lack of trust.
A trade-off between proving value and losing ownership of an entire product. This creates significant downstream costs for data consumers to compare samples (e.g. storage and compute resources and developer time spent homogenising data models and making multiple joins with proprietary data). This also leaves suppliers unable to compete or provide hands-on support as to how to best use their product.
Data is notoriously difficult to price; the value varies dramatically between vertical, jurisdiction, customer and use case AND there is imperfect knowledge on both sides. This ultimately leads to protracted commercial discussions, potentially with multiple parties simultaneously.
Extensive legal negotiations and agreements are often required to protect against the risk for both parties due to the supplier losing ownership of their product and the potential for loss, theft, misuse, etc. The risk appetite of organizations varies dramatically but also in line with the perceived value of the product they are trying to sell or buy.
Due to the slow, expensive and risky processes outlined above, it is typical for data licenses to be for years. This run generates yet more risk for the consumer, as the cost of failure is high. Market data teams try to simplify this by championing specific data. But in the absence of understanding the specific use case or the business teams being able to properly quantify or articulate the difference in value between alternatives, this is not a solution that works.
Historically, data marketplaces have tended to fail and did not find a mechanism for sufficiently addressing the critical issues of trust. Instead they opted for a standard marketplace paradigm with little adaptation to account for the unique characteristics of data. A standard marketplace is simply not the right model for supporting transactions between data buyers and suppliers. Additionally, any marketplace has to manage issues inherent to the marketplace model, including disintermediation, commoditization, governance of negative behaviour and the cold-start problem, all of which are arguably far more significant when the commodity is data.
That said, there are aspects of the data marketplace model that bring obvious benefits to anyone that has ever worked with data. Discoverability and fulfillment are important and data marketplaces addressed those specific issues very well. Data is often curated, comparable and sometimes even standardized, which is extremely helpful. Consumers are empowered to get the data on-demand, rather than it being entirely left with the supplier. These are important aspects of what good would look like in a successful future-state. In the right market and at the right time, a data marketplace can be useful. Quandl rode the alternative data wave, launching in 2013 and getting acquired by Nasdaq in 2018 for an undisclosed amount and continues trading to this day.
Ultimately, data marketplaces are likely to continue failing because they do not sufficiently optimize the concept to address the specific characteristics of data or, put another way, they solve the wrong problems. Historically, they did not focus enough development time on creating mechanisms to iteratively build trust and the ability to interact in low-trust situations. That requires significant effort to overcome but, done well, it also helps to manage other undesirable issues such as disintermediation and commoditization, which are problematic when working with something as complex and nuanced as data.
Unfortunately, the new wave of data marketplaces continues to use the exact same paradigm but try to avoid the cold-start problem by leveraging an existing user base. Avoiding a cold start problem is good, but this introduces a new complexity. This new wave is using the data marketplace paradigm to increase the usage and stickiness of their existing cloud platforms, so they limit where the data can reside which restricts the addressable market and creates complexity for suppliers, who need to manage their offerings across multiple data marketplaces. The value proposition for data suppliers is also unclear given many have solutions that enable discovery and self-service fulfillment—the main benefits of a data marketplace—but with all of their customers and prospects and without participants they do not want to work alongside.
Ultimately, time will tell if the data marketplace paradigm can ever succeed, but the historical data is not in its favour.
Co-authored by Anthony Cosgrove (Co-founder) & John Kuo (former Head of Product) at Harbr. Originally published in May 2020.
If you’d like to learn more about the differences between public and private data marketplaces, check out our guide.
We also have written a comprehensive guide to launching an internal data marketplace.
Data commerce insights. Delivered monthly.
© Harbr Group Limited 2024. All rights reserved. Privacy policy | Cookies policy | Terms
