Blogstats: BLOG ABOUT STATS.
All about dissemination of Official Statistics.
The views expressed on this weblog are those of the authors alone and do not necessarily reflect the views of employers.
Today Statistics Germany published their latest population projection until the year 2060. Together with this data the animated population pyramid was updated as well.
Most notable is a new layout that will put the assumptions right beside the pyramid and will let you switch between four different scenarios for the future (different assumptions for: fertility, life-expectancy, net-migration).
Thanks to the SVG Web library it will work in any browser and takes full advantages of open web standards, namely Scalable Vector Graphics (SVG). Watch a short screencast to see all of the functionality.
Then check it out for yourself, it’s available in english, french, german and russian at http://www.destatis.de/bevoelkerungspyramide/
Internet Explorer will need the Flash plugin to make this happen, all other browsers don’t.
When we first launched public data on Google.com, we wanted to make statistics easier to find and to encourage debate based on facts rather than intuition. The day after we launched, a friend who worked at the World Bank called me, her voice filled with enthusiasm, “Did you know that the World Bank also just released an API for their data?” Excited, I checked it out, and found an amazing treasure trove of statistics for most economies in the world. After some hard work and analysis, today we’re happy to announce that 17 World Development Indicators (list below*) are now conveniently available to you in Google search.With today’s update, you can quickly access more data with a broad range of queries. Search should be intuitive, so we’ve done the work to think through queries where public data will be most relevant to you. To see the new data, try queries like [gdp of indonesia], [life expectancy brazil], [rwanda's population growth], [energy use of iceland], [co2 emissions of iceland] and [gdp growth rate argentina]. For example, if you search for [internet users in the united states], you will see the following chart at the top of the results page:
NCVA (National Center for Visual Analysis) at Linköping University develops dynamic visualization publishing – a “statistical-story-telling” approach based on dynamic visualization “booklets” that are embedded into Web pages as HTML code. NCVA collaborates with its partners OECD, Eurostat, Statistics Denmark, Statistics Sweden and the city of Göteborg.
The software used is eXplorer, already used by e.g. OECD as “OECD eXplorer” and by Statistics Sweden as “SCB eXplorer“, to animate visualization of statistics at different geographical regional levels. Also the City of Linköping has made small-area statistics available in eXplorer.
The eXplorer software is made available now also in a format of “booklets” to facilitate use in regular official publishing and in commission work based on different international, national , regional and local (small-area statistics) databases.
SDMX (Statistical data and metadata exchange) is a widely accepted standard for communicating data in the realm of official statistics.
‘SDMX is an initiative to foster standards for the exchange of statistical information. Sponsored by BIS – ECB – EUROSTAT – IMF – OECD – UN – World Bank.’
More and more applications accept and use this standard for data input as well as for data output: So for instance the ECB Inflation dashboard or EUROSTATS plans to offer sdmxed data out of its database or the activities in PC-Axis tabulation software.
RDF, scovo
But official statistics are not the only ones who discuss and conceptualise statistical data exchange. scovo (‘The Statistical Core Vocabulary’) is another attempt, it ‘ specifies an [RDF-Schema] vocabulary for representing statistical data on the Web’.
Behind this we find – among others – the people who launched RIESE (RDFizing and Interlinking the EuroStat Data Set Effort).
Planetary conjunction
SDMX and RDF are beginning to approach each other. And this may lead to a new kind of open data: Already open statistical data become accessible as linked open data.
Such activities can be seen in the UK where the UK Government Data Developers (a Google Group, ONS is participating) are working on this topic.
Jenni Tennison describes this in his blog: ‘One of the things that we’ve been discussing over on the UK Government Data Developers mailing list is how best to represent the vast quantities of statistical data that the government produces, in RDF. This is what we’ve come up with: 1. We’ll use SCOVO (scv: <http://purl.org/NET/scovo#>)as our main vocabulary. ….’
and later on: ‘One source of sub-properties of scv:dimension (and subtypes of scv:Dimension) is SDMX (Statistical Data and Metadata eXchange). This provides standard ways of indicating things like the area and time that a statistic applies to…. ‘.
The work goes on and I hope that more of the specialists in statistical offices will enter the discussion.
At the IMAODBC Conference 2009 in Toronto Gareth Mc Guinness from the IMF presented a paper about SDMX and Web 3.0: That’s just semantics.
World bank launches an interactive visualizing tool resembling gapminder, it’s called Data Visualizer.
‘The time series used in Data Visualizer is a subset of 2009 World Development Indicators database. It contains 49 indicators for 209 countries and 18 aggregates from 1960-2007. Data includes social, economic, financial, information & technology, and environmental indicators.’
Tools allow to change the title and to export the graph as a jpeg. See this example:
Google’s interest in statistics is steadily growing. And this not least because Ola Rosling now works with Google and pushes visualization tools like motion chart and the public-data project.
In his presentation at the Gov 2.0 in Washington (September 2009) Ola Rosling presents theese tools: Ola Rosling, “Seeing Data as Change Over Time”
Accessing the data
How will Google access public data? The preference is bulk downloads:
Modes of Data Access
Really? We all know the risks of this: It’s manual work (isn’t it?), it needs resources, it’s repetitive … . With APIs and a standardized format like RDF and open silos (!) the future could begin … . See the remarks in this article ‘Official statistics beyond Web 2.0: Challenges, rewards and risks to come’.
Sources for Google’s public data are the U.S Census Bureau and the Bureau of Labor Statistics, also EUROSTAT is mentioned by Ola Rosling. What’s next? OECD and UN for much more country data? Will Google with its excellent internet presence become an important provider of international public data? So welcome Stats Google in the family….
And about the public-data project there is a good description by Ola Rosling in the Googleblog ‘Adding search power to public data’
In addition, there is Google Internet Stats
On September 9th 2009 Google UK quietly launched a microsite with statistics about the internet from a variety of sources. The statistics are divided into five sections: Technology, Macro Economic Trends, Media Landscape, Media Consumption and Consumer Trends.
Google-Internet-Stats
There are various sources: B2Bonline.com, BARB, BusinessWeek, Coke, Commission of the European Communities, Comscore, Core Metrics, Datamonitor, Deloitte, The Economist, eMarketer, Enders Analysis, Eurostat, Film Distributors Association, Financial Times, Forrester, GFK, Google Insights for Search, Greenbee.com, Guardian, HarvardBusiness.org, Hitwise, IAB, IFPI, IMF, Internet Retailing, Internetworldstats.com, JP Morgan, KMPG, Media & Marketing, Mediascope Europe, Mindshare, Motorola, Net Imperative, New York Magazine, Nielsen, NMA, Ofcom, Ipsos MediaCT, PriceWaterhouseCoopers, QuickPlay Media Inc., Retail Week, Reuters, TGI Net, Times Online, TNS, Verdict Research, Wall Street Journal, WARC, YouTube, ZenithOptimedia, GM
In a interview given at a technology symposium at the Embassy of Finland in Washington (15 October 2009), ‘Berners-Lee speaks about the importance for governments to place great amounts of data on the Web and the emergence of the semantic Web. He cites successful examples in Britain.’
‘In all this Semantic Web news, though, the proof of the pudding is in the eating. The benefit of the Semantic Web is that data may be re-used in ways unexpected by the original publisher. That is the value added. So when a Semantic Web start-up either feeds data to others who reuse it in interesting ways, or itself uses data produced by others, then we start to see the value of each bit increased through the network effect.’
‘DARREN GERSH: So it’s been 20 years since you invented the worldwide web. What is the biggest challenge for the Internet in the next 20 years?
BERNERS-LEE: Well, I think looking forward the most difficult thing that we’re not doing is we’re not actually studying the web. The web’s huge. It’s a huge system. There are actually more web pages out there than there are neurons in a person’s brain. So there are a lot of nerve cells (ph) in a person’s brain, but we are starting to figure out how the brain works but we really don’t study how the web works. It’s humanity connected, all these people making links, following links, exchanging ideas, trying to put together new forms of democracy, new social networking systems. We don’t really understand what is going to work and what isn’t. We don’t really know what are the dangers. Could it become unstable? What are the really huge opportunities? So studying the web is really important. I think the danger is we don’t study it and then suddenly, something happens like the financial downturn, like spam coming along, like one of these things where whoa! Oops, didn’t plan for that. Now what went wrong? So we shouldn’t be looking back and thinking what went wrong? We should be looking forward and thinking, OK, what are the things that could happen? What would be likely to happen? How can we tweak the web? After all the web, unlike the brain, is something that we designed. It’s an engineering thing.’
And he confesses that // in the http adress is in fact unnecessary .
There are specialized search engines like sig.ma using semantic technology (but working in niches), and there are the big search machines slowly starting to use such an approach. Yahoo does it for some time already and Google goes RDFa and microformats.
Google looks for markup formats (microformats and RDFa) and displays reviews and ratings and also information about persons in Rich Snippets. New types of data beyond reviews and people will follow.
‘Imagine that you have a review of a restaurant on your page. In your HTML, you show the name of the restaurant, the address and phone number, the number of users who have provided reviews, and the average rating. People can read and understand this information, but to a computer it is nothing but strings of unstructured text. With microformats or RDFa, you can label each piece of text to make it clear that it represents a certain type of data: for example, a restaurant name, an address, or a rating. This is done by providing additional HTML tags that computers understand. These don’t affect the appearance of your pages, but Google and any other services that look at the HTML can use the tags to better understand your information, and display it in useful ways—for example, in search results.’
The session ‘Statistics and the Internet’saw 4 presentations:
Managing the Internet in a statistical institution Leon Oestergaard, Statistics Denmark
Influence of the Internet on dissemination of official statistics Tomaz Smrekar, Statistical Office of the Republic of Slovenia
Official statistics beyond Web 2.0: Challenges, rewards and risks to come Armin Grossenbacher, Swiss Federal Statistical Office, Switzerland
Evaluating the public’s perception of NSO websites using a non- standard mixed methodological approach Derek Bond and Elaine Ramsey, University of Ulster at Coleraine, UK
Julia, thanks for the link in your comment of the previous post. I hope you do not mind I display your site in this way as well!/Best regards Alf
What is ABS BetaWorks?
ABS βetaWorks is a development environment for new designs and concepts for the ABS website. βetaworks allows ABS Designers to showcase what we’re working on and find out what you think. Using your insights and ideas, we can continue to improve the ABS website experience for all.
On this blog we usually showcase best practices of how to communicate statistics and keep the technological aspects of it in the background – which is the right way to do. But we also never get tired of mentioning how statistics is a basis for informed decision making and therefore a foundation for democracy. To live up to these standards we make sure our methods are well documented and I would argue we should also give some thoughts on what technology we use.
There are two reasons for this: We should allow our users to learn from our applications on the web, build upon them, mash them up with other stuff we didn’t imagine or even improve them. And secondly when we talk about archiving in the digital age, we are well advised to use open standards. Everyone of us who recently tried to open some old Word 2.0 documents will understand what I mean.
The topic comes up as several statistical offices have moved from the SVG format for interactive statistical graphics to Flash. See the latest population pyramid from ONS or the election atlas in Germany. While SVG is an open graphics standard just like HTML, you can think of Flash more like Word documents, those are closed binary files. Users cannot look behind the scenes and if the source code gets forgotten or the technology changes dramatically, all is lost.
Now there is a flipside to it: Just like Microsoft Word, Flash is ubiquitous, works really well and works the same way across all supported platforms. SVG on the other hand had its ups and downs. Since 2008 it is very well supported on modern browsers such as Opera, Firefox, Safari and Google Chrome but even Internet Explorer 8 doesn’t handle it at all. There was a plugin for Internet Explorer, but that never had the significant market share that Flash enjoyed (no YouTube without Flash!) and was “end of lifed” in January 2009, meaning SVG support on Internet Explorer was gone at the beginning of this year. So why all the bemoaning, free and open doesn’t allways win.
Well, things are changing right now. With the help of Google an open source project aims at adding SVG support to Internet Explorer through the Flash plugin. And they are aiming high, want to implement it in Wikipedia, which uses SVG as a base format for all their graphics and maps. It’s called the SVG Web project, and already it works well enough to support our use-cases. I’ve put up an animated population pyramid and an interactive map with it and couldn’t be happier.
Even in Internet Explorer you can right click in the graphics and “view source”, see how everything was done, adapt it, improve it … And when your are using Firefox or Safari you can print these graphics into PDF and get print quality vector graphics.
Give it a look, talk about it with your tech people and let me know what you think. We have comments here for a reason.
Statistical graphics are most convincing when they allow for interesting comparisons. A pie- or bar-chart allows comparisons in one data dimension as does one map, it shows how one variable varies in different regions. But data analysis shouldn’t stop here. Diagrams like the animated population pyramid or the gapminder/trendalyzer allow comparisons in more than one dimension where one of the dimensions is usually time which is depicted through animation.
Comparing regional patterns is a little trickier. A standard use case could be the question if people have more children in regions where conservative votes are higher. This would statistically be done by calculating correlations. However regression analysis is not for everybody. At least it would be nice to show two related patterns side by side and give people an idea what correlated variables would look like. Below is an example of how this could be implemented:
The Open Data Initiative is a Web 2.0 site for disseminating public data. Users discover and explore data in a rich, interactive, and intuitive application, rather than browse or read large documents of published tables and charts. The end user can select and visualize any combination of data. It can be exported, printed, linked to, and shared in collaboration environments.
The Open Data Initiative is a freely available online service for the creation and dissemination of data for public consumption. You have the data; we have the service to disseminate it to the public.
The Open Data Initiative is hosted on the Google AppEngine Cloud, enabling providers of public data to create engagingand rich Web 2.0 experiences built on top of Space‐Time Research’s SuperVIEW product suite. This provides transparent, lightning‐fast web traffic responsiveness, scalability and built in redundancy no matter where in the world you are.
Data types suitable for the Open Data Initiative: Health, Transport, Education, Agriculture, Population Statistics, Labour Force, etc.
The documentary, by the journalist Pär Fjällström, follows Hans Rosling preparing his lecture at the 2009 TED conference. The one hour documentary, which has aired several times on Swedish television will be available at svt.se for 30 days (until September 17).
Follow Hans Rosling on Twitter
You can now follow Hans Rosling on Twitter. On his Twitter page you can read his “tweets” about interesting facts and graphics and, if you have a Twitter account of your own, become a follower.
Gapminder is a non-profit foundation based in Stockholm, Sweden. We are promoting sustainable global development and the achieve- ment of the United Nations Millennium Development Goals by unveiling the beauty of statistics for a fact based world view.
Forward this email – do you know someone who might be interested in receiving the Gapminder newsletter?
Not yet subscribed to the newsletter? Subscribe here | Not interested anymore? Unsubscribe here
The conference “Statistics – Investment in the future 2″ will take place in Prague from 14 to 15 September 2009 at Congress Centre of the Czech National Bank. The conference is held by the Czech Statistical Office, the Czech National Bank and the Prague University of Economics on the occasion of the 90th anniversary of Czech official statistics.
Hans Rosling is a professor in international health, who has made an unlikely global success. His presentations on global development evokes laughter, rejoice and reflections. People with power, like Al Gore or Bill Clinton, ask for his advice. He wants everyone to question their prejudices about the world – as he himself has needed to do. A documentary by Pär Fjällström, SVT.
NCVA (
.
Recent Entries
Recent Comments
