All about Dissemination of (Official) Statistics
Once again: the New York Times presents an innovative graphic, which you always want to watch again and again.
It’s this Downhill Race at the Olympics:
Start
Run
Finish
The link to the moving graphic is below this picture:
It would be exciting to follow such visualizations, e. g. on changes in unemployment, GDP etc. of different countries from today-minus-x to today.
In a recently published EUROSTAT publication, the authors demand innovative forms of communication from public statistics in order not to lose their socially important role. Among other things, they demand ‘…. to tell stories close to the people; to create communities around specific themes; to develop among citizens the ability to read the data and understand what is behind the statistical process.’
The UNECE hackathon 
that has just been completed responds to this challenge.
‘A hackathon is an intensive problem-solving event. In this case, the focus is on statistical content and effective communication. The teams will be challenged to “Create a user-oriented product that tells a story about the younger population”. During the Hackathon, fifteen teams from nine countries had 64.5 hours to create a product that tells a story about the younger population. The teams were multidisciplinary – with members from statistical offices and other government departments. The product created should be innovative, engaging, and targeted towards the general public (that is, not specialists). There was no limit on the form of the product, but the teams had to include a mandatory SDG indicator in the product.
The mandatory indicator was “Proportion of youth (aged 15-24 years) not in education, employment or training” SDG indicator (Indicator 8.6.1).‘ (Source)
And the hackathon shows impressive results, even if only a few organisations have participated.
The four winners are:
My favourites are number 3 from the National Institute of Statistics and Geography (INEGI-Mexico) and number 2 from the Central Statistical Office of Poland.
…is aesthetically pleasing and easy to use. The interaction is left to the user and can be individually controlled by him/her in the speed.
The diagrams do not stand alone, but are explained by short texts while scrolling.
The results are not just being accepted. Rather, the concepts are explained and questioned – statistics are presented with the methodological background.
…starts with a jourmalistic approach. Here too, the interactivity can be controlled by the user at the desired speed.
At the end, the authors also seek direct contact with the users; a quiz personalizes the statistical data and gives an individual assessment of where the users stand personally with regard to these statistics.
The two applications mentioned above combine decisive user-friendly features:
– visually attractive,
– easy-to-understand navigation that can be controlled by the user according to his needs,
– the journalistic approach,
– concise and instructive explanations,
– personalization,
– hints on the methodological background.
Many of the other applications show the frequently encountered weaknesses: Too much information should be provided, no courage to leave something behind and concentrate on the most important elements. And this leads to long texts and complex navigation with the effect that users quit quickly.
In a recent paper Emanuele Baldacci, (Director, Eurostat) and Felicia Pelagalli, (President, InnovaFiducia) deal with the ‘challenges for official statistics of changes in the information market spurred by network technology, data revolution and changes in information consumers’ behaviours’ (p.3)
The status-quo or bad scenario:
‘Information will continue to be consumed via multiple decentralized channels, with new information intermediaries emerging through social platforms, digital opinion leaders, technologies that reinforce belonging to peers with similar profiles and backgrounds, including in terms of beliefs.’ … ‘Under this scenario it is likely that increased competition from alternative data providers will put pressure on the official statistics position in the information ecosystem and lead to drastic reduction of public resources invested in official statistics, as a result of the perceived lack of relevance.’ (p.8)
The ugly scenario:
‘Big oligopoly giants will emerge by integrating technologies, data and content and providing these to a variety of smaller scale platforms and information intermediaries, with limited pricing power for further dissemination. In this scenario, data generated by sensors and machines connected to the network will increasingly create smart information for individuals. However, individuals will not participate in the data processing task, but will be mostly confined to crowdsourcing data for digital platforms and using information services.’
‘In this scenario, official statistics will be further marginalized and its very existence could be put in jeopardy. More importantly, no public authority with significant influence could be in charge of assessing the quality of data used in the information markets. Statistics as a public good may be curtailed and limited to a narrow set of dimensions. … Official statisticians will appear as old dinosaurs on the way to extinction, separated from the data ecosystem by a huge technology and capability gap.’ (p.9)
The good scenario:
The authors do not stop here. They also see a good scenario, but a scenario that implies a huge engagement.
This scenario is ‘predicated on two major assumptions.
First, the information market will be increasingly competitive by sound regulations that prevent the emergence of dominant positions in countries and even more important across them.
Second, official statistics pursue a strong modernization to evolve towards the production of smart statistics, which fully leverage technology and new data sources while maintaining and enhancing the quality of the data provided to the public.
In this scenario, official statistics will generate new more sophisticated data analytics that cater to different users by tailored information services. It uses network technologies (e.g., blockchain, networks) to involve individuals, companies and institutions in the design, collection, processing and dissemination of statistics. It engages users with open collaborative tools and invests heavily in data literacy to ensure their usability. It strengthens skills and capacity on statistical communication to help users understand in transparent manners what are the strengths and limitations of official statistics.’ (p. 9/10)
The good scenario already depicts some needed actions to be taken by official statisticians. The authors conclude with proposals that are not really new, ideas that have been on the table for some time but are not so easy to implement.
‘It is important to change mindsets and practices which have been established, in order to put in contact the citizens with official statistics, to make data accessible, to expand the understanding of their analysis, to support individuals, business and institutions in the decision-making process.
The key issue is how to be authoritative and to develop quality knowledge in the new and changing information market. It is important to know the rules and languages of the media platforms used for communication; to overcome the technicalities; to tell stories close to the people; to create communities around specific themes; to develop among citizens the ability to read the data and
understand what is behind the statistical process. In summary, put people at the center (overused phrase, but extremely valuable):
⎯ communicate statistics through engaging experiences and relevant to the people who benefit from them;
⎯ customize the content;
⎯ adopt “user analytics” to acquire the knowledge of the “users” through the analysis of data (web and social analytics) and the understanding of people’s interaction with the different platforms.’ (p.11)
And the concluding words call for external assistance:
‘It will be essential for statisticians to build more tailored data insight services and team up with communication experts to play a more proactive role in contrasting fake news, checking facts appropriately and building users’ capacity to harness the power of data.’ (p.12)
Eurostat’s biennial scientific conference on New Techniques and Technologies for Statistics (NTTS) is over, a labyrinth of a website is online and tons of documents are somewhere published.
CBS Corporate nieuws summarizes the important trends discussed:
1) New data sources and the consequences
2) The importance of a proactive communication
3) Big Data and algorithms in official statistics
CBS, 06-06-2017 Miriam van der Sangen
Why taking this information just from CBS (the Dutch Statistical Office)? Because CBS Corporate nieuws is an excellent example of the second trend: proactive communication, proactivity in delivering (statistical) information to users. The website makes corporate information public and gives insights into activities of CBS and statistics. You see topics …
… and the people behind it.
The target public of this corporate website are enterprises, administrations, journalists, students and whoever may be interested.
A shorter English version is integrated into the CBS website.
Corporate websites like CBS’ are not quite usual. They are resource consuming but are probably very good in helping to understand statisticians’ mission and work .. and in motivating employees.
The New York Times did it after the election, in January 2017: You Draw It, Learning Statistics by drawing and comparing charts.
‘Draw your guesses on the charts below to see if you’re as smart
as you think you are.’
And Bayerischer Rundfunk did it before the election, in April 2017.
This kind of giving information is an excellent strategy to foster insights and against forgetting. And it’s an old tradition in didactics. 360 years ago Amos Comenius emphasized this technique in his Didactica Magna:
“Agenda agendo discantur”
Twitter reminded me that there’s #NTTS2017 going on, Eurostat’s biennial scientific conference on New Techniques and Technologies for Statistics (NTTS).
‘We are extremely sad to announce that Professor Hans Rosling died this morning. Hans suffered from a pancreatic cancer which was diagnosed one year ago. He passed away early Tuesday morning, February 7, 2017, surrounded by his family in Uppsala, Sweden.’ Anna R. Rönnlund & Ola Rosling, Co-founders of Gapminder. He died aged 68.
In 2009, the Swiss Statistics’ Meeting took place in Geneva, Switzerland. Hans Rosling was there and his talk’s topic: ‘Unveiling the beauty of statistics’. He wanted data to be free, free from legal and technical barriers. His ambition – and his success – was to disseminate these data beautifully … in order to change the world.
A difficult task. In an interview in the Guardian, in 2013: “It’s that I became so famous with so little impact on knowledge,” he says, when asked what’s surprised him most about the reaction he’s received. “Fame is easy to acquire, impact is much more difficult. …. He’s similarly nonplussed about being a data guru. “I don’t like it. My interest is not data, it’s the world. And part of world development you can see in numbers.” (Taken from the Guardian interview 2013).
And that’s why statistics and the world need more people like Hans Rosling – more than ever!
‘Fake-news’ and ‘post-truth’ (postfaktisch) are the words dominating today many discussions about truth in communication.
' ... in post-truth [post] has a meaning more like ‘belonging to a time in which the specified concept [truth] has become unimportant or irrelevant’' (https://www.oxforddictionaries.com/press/news/2016/11/15/WOTY-16).
False information or even lies are not new in the information business. And therefore many, and many more websites help to separate wrong from right:
The Reporters’ Lab maintains a database of global fact-checking sites.
And Alexios Mantzarlis ‘collected 366 links, one for each day of the year … to understand fact-checking in 2016′.
Officials Statistics collect, analyze and disseminate statistical information since long and are also confronted with wrong citations, misuse of statistics and lies. Many of the ethical codices of official statistics recommend acting against such false information.
‘In 1992, the United Nations Economic Commission for Europe (UNECE) adopted the fundamental principles of official statistics in the UNECE region. The United Nations Statistical Commission adopted these principles in 1994 at the global level. The Economic and Social Council (ECOSOC) endorsed the Fundamental Principles of Official Statistics in 2013; and in January 2014, they were adopted by General Assembly. This recognition at the highest political level underlines that official statistics – reliable and objective information – is crucial for decision making.’
Two paragraphs are of special interest:
‘ 2. Professional standards and ethics
To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.’
AND:
‘4. Prevention of misuse
The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.’
The European Statistics Code of Practice says in principle 1:
1.7: The National Statistical Institute and Eurostat and, where appropriate, other statistical authorities, comment publicly on statistical issues, including criticisms and misuses of statistics as far as considered suitable.
N.B: Wikipedia’s page on Misuse of statistics presents a broad view how readers can be fooled by many types of misuse.
False – and especially deliberately false – information as a weapon in manipulating decisions isn’t new either. But new is how such information spreads: with the help of social media dissemination gains a new level (some say like earlier Gutenberg’s printing press ).
Anne Applebaum gives a practical illustration of how it can work:
It was a peculiar experience, but I learned a lot. As I watched the story move around the Web, I saw how the worlds of fake websites and fake news exist to reinforce one another and give falsehood credence. Many of the websites quoted not the original, dodgy source, but one another. There were more phony sites than I’d realized, though I also learned that many of their “followers” (maybe even most of them) are bots — bits of computer code that can be programmed to imitate human social media accounts and told to pass on particular stories.
….
But it is also true that we are living through a global media revolution, that people are hearing and digesting political information in brand-new ways and that nobody yet understands the consequences. Fake stories are easier to create, fake websites can be designed to host them, and social media rapidly disseminates disinformation that people trust because they get it from friends. This radical revolution has happened without many politicians noticing or caring — unless, like me, they happened to have seen how the new system of information exchange works.’
May 2017 become the year of people who know about the power and the dangers of misleading information!
My best wishes to the colleagues in Official Statistics and their professional producing and disseminating information …. and perhaps statistical dissemination will need to be more active on social media, too.
Visualising data helps understanding facts.
Sometimes it’s very easy to understand a graph; sometimes it’s necessary to read it and to study it to discover unknown territory.
Such graphs are little masterpieces. Here’s one of these and I am sure the authors had more than one iteration and discussion while creating it.
The graph tells the story of the average disposable income and savings of households in Switzerland, published by the Swiss Federal Statistical Office FSO.
The authors kindly give a short explanation:
How to read this graph. In one-person households aged 64 or under, the upper-income group has a disposable income of CHF 8487 per month and savings of CHF 2758 per month. Representing 4.0% of all households, this income group corresponds to a fifth of one-person households aged 64 or under (20.1%)
There’s another nice graph, a little bit less elaborated, also explained by the authors:
But there’s one thing that is not explained:
the confidence interval!
‘A confidence interval gives an estimated range of values which is likely to include an unknown population parameter, the estimated range being calculated from a given set of sample data,‘ and the above poverty data are from a sample of ‘approximately 7000 households, i.e. more than 17,000 persons who are randomly selected…’.
Or:
‘The confidence intervals for the mean give us a range of values around the mean where we expect the “true” (population) mean is located (with a given level of certainty, see also Elementary Concepts). ….. as we all know from the weather forecast, the more “vague” the prediction (i.e., wider the confidence interval), the more likely it will materialize. Note that the width of the confidence interval depends on the sample size and on the variation of data values…..’
Khan Academy gives lectures about topics like confidence intervals, sampling, etc.
.
The above graphs use just one of multiple possibilities for visualising data.
Severino Ribecca’s Data Visualisation Catalogue is one of many websites trying to give an overview. And there’s the risk to get lost in these compilations.
© listverse.comMassive Open Online Courses (MOOC) are available worldwide and offer tons of topics, also about Linked Open Data (LOD). An easy way to enter the semantic web.
Two examples:
The Hasso Plattner Institute, Potsdam provides, for some years now, a course in Linked Data Engineering with a certificate. I did it some years ago and enjoyed it.
The French platform FUN offers a LOD course, too. (Thanks to Adrian at zazuko.com for the hint)
Step by step Bob DuCharme introduces RDF, SPARQL, LOD …
.
The European Commission (Directorate General for Communications Networks, Content and Technology) just published the second Open Data Maturity Report.
‘Open Data refers to the information collected, produced or paid for by public bodies which can be freely used, modified, and shared by anyone for any purpose.’ (all pages from the report, p.6)
‘The two key indicators used to measure Open Data Maturity are Open Data Readiness and Portal Maturity.
— The first key indicator, Open Data Readiness, assesses to what extent countries have an Open Data policy in place, licensing norms and the extent of national coordination regarding guidelines and setting common approaches.
— The second key indicator, Portal Maturity, explores the usability of the portal regarding the availability of functionalities, the overall re-usability of data such as machine readability and accessibility of data sets, for example, as well as the spread of data across domains.
The two key indicators as well as the sub-indicators are shown in the table below.
(pp.6 and 10)
Overview by countries:
Results from the Open Data Maturity Assessment for the EU28+ countries for 2016 (p.59)
From European Data Portal, all the details here.
(p.59)
And one more result blog about stats enjoys:
One of the criteria for open data maturity is the re-usability of data and especially machine readability of data. Six questions focus on this item:
‘When looking at the data on the European Data Portal, over 49 different file formats are used. The most used data formats are CSV, HTML and WMS. The fourth most used data format is PDF. PDF is one of the few data formats that is not machine-readable. The following most frequent distributions are ZIP, JSON, XLS and XLSX, followed by WFS and XML. Numbers range from nearly 49,000 CSV formats to just over 23,000 JSON formats to the least used 263 shape formats. Most data formats are or are related to a spreadsheet, which enables to analyse the data more swiftly.’ (p. 49).
That is not enough and the report recommends:
‘On the more technical side, some improvements are still necessary. To further develop automated processes each national portal should have an API in combination with a complete metadata profile. This allows a portal to share the data with data users more easily. This can for instance enable harvesting data directly from public administrations in an automated fashion, saving efforts in manual uploading of data and limiting errors when editing data and metadata manually. … Typos or different spellings can limit the discovery of data. Here activities conducted at EU level on controlled vocabularies can be of interest to learn from in order to increase semantic interoperability. ‘ (p.63)
PDF is poor, XLS is better, CSV even better, also JSON and APIs; and metadata are of crucial importance. The European Data Portal gives a good example: it organizes the datasets in the triple format (RDF) and offers an SPARQL search.
But there is more.
Not only the datasets could foster semantic interoperability but also the data in these sets. Linked data and adequate formats can assure this interoperability and with this extended machine readability and use of data. So why not add this criterion to the questionnaire (Question 7.6 +) and lead the national portals in this direction?
Linked data? Tim Berners-Lee explains
And related: https://blogstats.wordpress.com/2016/02/03/open-data-portals-news/
Blog about Stats 