Digitally disrupted data production
“The collection of statistics has been digitally disrupted, along with everything else, and there are important questions about collection methods and whether or not “big data” genuinely offers promise for a giant leap forward in the productivity of official statistics.”
This statement in Financial Times’ edition of August 20th, 2015 deals with UK’s Office for National Statistics ONS. Its title: “UK needs a statistical strategy to catch up with digital disruption”. Its message: ONS (and I think all Official Statistics) has problems to keep up “with the profound changes in the structure of the economy during recent decades.”
The “Independent Review of UK Economic Statistics” by Professor Sir Charles Bean, Professor of Economics at the London School of Economics in March 2016 goes deeper and gives 24 recommendations, some of these obviously valid for statistics’ production and producers in general.
“Innovation and technological change are the wellspring of economic advancement. The rapid and sustained rise in computing power, the digitisation of information and increased connectivity have together radically altered the way people conduct their lives today, both at work and play. These advances have also made possible new ways of exchanging goods and services, prompted the creation of new and disruptive business models, and made the location of economic activity more nebulous. This has generated a whole new range of challenges in measuring the economy.” (p.71)
“Measuring the economy has become even more challenging in recent times, in part as a consequence of the digital revolution. Quality improvements and product innovation have been especially rapid in the field of information technology. Not only are such quality improvements themselves difficult to measure, but they have also made possible completely new ways of exchanging and providing services. Disruptive business models, such as those of Spotify, Amazon Marketplace and Airbnb, are often not well-captured by established statistical methods, while the increased opportunities enabled by online connectivity and access to information provided through the internet have muddied the boundary between work and home production. Moreover, while measuring physical capital – machinery and structures – is hard enough, in the modern economy, intangible and unobservable knowledge-based assets have become increasingly important. Finally, businesses such as Google operate across national boundaries in ways that can render it difficult to allocate value added to particular countries in a meaningful fashion. Measuring the economy has never been harder.” (p. 3)
And: “Recommended Action 4: In conjunction with suitable partners in academia and the user community, ONS should establish a new centre of excellence for the analysis of emerging and future issues in measuring the modern economy.” (p.118)
Disrupting Dissemination of Statistics
The rise of new technologies followed by new information behavior has also disrupted existing dissemination formats (from print to digital) and dissemination practices (from quasi-monopolistic to open and multiple).
A well-known example for disrupting dissemination is given by Wikipedia and its subject is Wikipedia itself:
“The free, online encyclopedia Wikipedia was a disruptive innovation that had a major impact on both the traditional, for-profit printed paper encyclopedia market (e.g., Encyclopedia Britannica) and the for-profit digital encyclopedia market (e.g.,Encarta). The English Wikipedia provides over 5 million articles for free; in contrast, a $1,000 set of Britannica volumes had 120,000 articles.” (Article: https://en.wikipedia.org/wiki/Disruptive_innovation)
In fact, disruptive tendencies happen on both sides: in producing and in presenting or accessing statistical information.
Some thoughts about this:
- Until the end of the 20th-century, print was the main channel for disseminating statistics. Libraries in Statistical Offices and Society had their very vital role.
- With the internet opened a new channel: Statistical Offices’ Websites appeared, access to databases and attractive data presentation (visual, storytelling, see i.e. this) were top themes and stuff for long discussions. Access to data was now simple and for everyone.
- With the open data initiatives not only accessing but also disseminating statistical information got much easier. Nearly everyone could become a data provider. License fees no longer hindered the redisseminaton of official statistics and APIs or webservices provided by statistical offices made this possible in an automated way.
Statistics can be easily integrated into websites and apps of non-official data providers, this with all the chances to enable democratic conversation and the risks of data misuse. - All this gives statistics a much more important role in communication processes. On the other hand communicating with statistics gets simpler: Letters, telephone calls and even e-mails become cumbersome seen the possibility bots (will) provide. With a stats bot in my daily used messenger, I ask for a statistical information, and the bot uses a search engine or connects me directly to a statistical expert.
“Brands that already have full-fledged apps and responsive websites can take advantage of bots’ ability to act as concierges, handling basic tasks and micro-interactions for users and then gracefully connecting users with apps or websites, as appropriate, for a more involved experience.” (Adam Fingerman, venturebeat, 20.7.2016) - What’s next? Innovation with disruption goes on, but disruption does not always mean destruction: It’s still a wise decision to keep some information in paper format. A statistical yearbook with key data lasts for centuries, not so a website, an API or a bot.
Blog about Stats 