The Cui-bono Approach to Open Data

What’s the problem? Which data are needed to solve it? Who gets an advantage of it?

These few questions are valuable key for implementing the open data culture. Open data not as ‘l’art pour l’art’ but in a pragmatic approach, demonstrating that the ‘proof of the pudding is in the eating’.

2015-07-03_opendatatriangle

It seems to work very well as Ton Zijlstra showed in his presentation at the Swiss Opendata Conference 2015.

He gives some examples of situations where open data helped to provide a solution to a problem and where stakeholders got an answer to their issues.

2015-07-03_Zijlstra

link ti

Next Step after OGD: Government’s Big Data Scientist

Open Government Data (OGD) Initiatives have been important steps helping to give broader access to administrative data.

But there was some disappointment because OGD didn’t bring up the mass of apps many hoped. And meanwhile big discussions about using Big Data emerged.

Now the US make a step forward going for a Big Data Initiative: President Obama just nominated a Chief Data Scientist in his Office, DJ Patil.

https://m.whitehouse.gov/blog/2015/02/18/white-house-names-dr-dj-patil-first-us-chief-data-scientist

‘Patil’s new role will involve the application of big data to all government areas, but particularly healthcare policy.’ (Source)

2015-03-19_Patil-Q&A

Open Data Index

There are lots of indexes.
The most famous one may be the  Index Librorum Prohibitorum  listing books prohibited by the cathoilic church. It contained eminent scientists and intellectuals (see the list in Wikipedia) and was abolished after more than 400 years in 1966 only.

Open Data Index

One index everybody would like to be registered in and this with a high rank is the Open Data Index.

‘An increasing number of governments have committed to open up data, but how much key information is actually being released? …. Which countries are the most advanced and which are lagging in relation to open data? The Open Data Index has been developed to help answer such questions by collecting and presenting information on the state of open data around the world – to ignite discussions between citizens and governments.’

2013-10-29_odindex-overall

‘The Open Data Index is an initiative of the Open Knowledge Foundation based on contributions from open data advocates and experts around the world. …. The Open Data Index is a community-based effort initiated and coordinated by the Open Knowledge Foundation with participation from many different groups and individuals. The Open Data Census, upon which the Open Data Index is based, was launched in April 2012 to coincide with the OGP meeting in Brasilia.’
See also https://blogstats.wordpress.com/2013/06/15/open-data-census/

‘The 2013 Open Data Index launches just before the Open Government Partnership summit in London, at a time when governments and civil society meet to make commitments, monitor progress, and plan for greater open government and transparency around the world.’ (more).

Country Comparison

2013-10-29_odindex

.

Country Details: Switzerland

2013-10-29_odindexCH

.

What criteria matters in the assessment of the datasets?’

‘When submitting a dataset, there is a list of questions to answer about the availability and openness of the datasets. These answers appear in the Country overview page for each country:

Question Details Weighting
Does the data exist? Does the data exist at all? The data can be in any form (paper or digital, offline or online etc). If it is not, then all the other questions are not answered. 5
Is data in digital form? This question addresses whether the data is in digital form (stored on computers or digital storage) or if it only in, for example, paper form. 5
Publicly available? This question addresses whether the data is “public”. This does not require it to be freely available, but does require that someone outside of the government can access it in some form (examples include if the data is available for purchase, if it exist as PDFs on a website that you can access, if you can get it in paper form – then it is public). If a freedom of information request or similar is needed to access the data, it is not considered public. 5
Is the data available for free? This question addresses whether the data is available for free or if there is a charge. If there is a charge, then that is stated in the comments section. 15
Is the data available online? This question addresses whether the data is available online from an official source. In the cases that this is answered with a ‘yes’, then the link is put in the URL field below. 5
Is the data machine readable? Data is machine readable if it is in a format that can be easily processed by a computer. Data can be digital but not machine readable. For example, consider a PDF document containing tables of data. These are definitely digital but are not machine-readable because a computer would struggle to access the tabular information (even though they are very human readable!). The equivalent tables in a format such as a spreadsheet would be machine readable. Note: The appropriate machine readable format may vary by type of data – so, for example, machine readable formats for geographic data may be different than for tabular data. In general, HTML and PDF are not machine-readable. 15
Available in bulk? Data is available in bulk if the whole dataset can be downloaded or accessed easily. Conversely it is considered non-bulk if the citizens are limited to just getting parts of the dataset (for example, if restricted to querying a web form and retrieving a few results at a time from a very large database). 10
Openly licensed? This question addresses whether the dataset is open as per http://opendefinition.org. It needs to state the terms of use or license that allow anyone to freely use, reuse or redistribute the data (subject at most to attribution or sharealike requirements). It is vital that a licence is available (if there’s no licence, the data is not openly licensed). Open Licences which meet the requirements of the Open Definition are listed at http://opendefinition.org/licenses/. 30
Is the data provided on a timely and up-to-date basis? This question addresses whether the data is up-to-date and timely – or long delayed. For example, is election data made available immediately or soon after the election, or is it only available many years later? Any comments around uncertainty are put in the comments field. 10
URL of data online? The link to the specific dataset if that is possible. Otherwise to the home page for the data. If that is not possible, then the link to main page of site on which the data is located. Only links to official sites are eligible, not third party sites. When it is necessary for submitters to provide third party links, then they are put in the comments section.
Date the data became available? This question describes when the data first became openly available (online, in digital form, openly licensed etc). Sometimes this is approximate. For example, “2012” or “Jan 2012”. If there is a precise date, then they are typed in in a yyyy-mm-dd format.
If the data is not open, then this question will instead describe the date the data first became available at all. (Note: some open data will have been available in other forms previously, so the date specified here is the date it became openly available).
Format of data? This question describes the form that the data is available in. For example, for tabular data it might be: Excel, CSV, HTML or even PDF. For geodata it might be shapefiles, geojson or something else. If available in multiple formats, the format descriptors are listed separated with commas. Any further information is put in the comments section.’

For Switzerland Timetables (of major government operated (or commissioned) *national-level* public transport services (specifically bus and train))
and National government budget (at a high level (e.g. spending by sector, department etc)) are less open.
Data from swisstopo and Statistics Switzerland (partially thanks to the new opendata.admin.ch/ portal) have most criteria in green, the main question lies in licensing (not freely available, not free for commercial use).

.

Featured Visualisation: An example how to present Open Data

NYC Open Data Site Finder
This interactive graphic, inspired by Chris Whong’s d3.js network diagram, allows users to access every link in the NYC Open Data site. Hover over a circle in the packed bubble chart to see link info, and click on a circle to access the site in a new browser tab. Use the bar charts and filters to focus your view.’

2013-10-29_nyc-finder

OGD: Oil, Gold, Democracy ?!

Open Government Data (OGD) are seen by many people as the new gold of the digital age. So Neelie Kroes (Vice-President of the European Commission responsible for the Digital Agenda Data) in her Opening Remarks at the Press Conference on Open Data Strategy in Brussels, 12th December 2011:

“Just as oil was likened to black gold, data takes on a new importance and value in the digital age. … Public data, generated by all administrations in Europe will become automatically re-usable. It will feed new applications and services. … We calculate that public sector information already generates €32 billion of economic activity each year. This package would more than double that – – to around €70 billion.”

OGD Portal

Since September 16th 2013 Switzerland has its Open Government Data Portal, too. It’s based on CKAN and is filled with about 1600 files, mostly statistical data.

2013-10-14_OGDPortalCH
.

OGD: Economic Impact

And Switzerland has its own report (by Adelheid Bürgi-Schmelz ) assessing the economic importance and impact of OGD. It’s a broad evaluation of methods and results (in German) with a long summary in English:
.
2013-10-14_OGD-CH-Summary

‘Conclusion
From an economic perspective, Switzerland would benefit from an introduction of OGD. The Swiss federal administration would obtain efficiency gains provided the compensation issue for federal offices can be settled. But ultimately, the issue whether OGD should be introduced in Switzerland is subject to political decision making.’ (p.14)  “Information is the currency of democracy.” Thomas Jefferson (p.8)

2013-10-14_ImpactOGDCH

 .

Ex post?

Are the now published data by several governments the data needed? What’s their impact after first evaluations?
These questions are more and more of interest. And governments are starting initiatives to ‘sell’ open data with specific actions. So the White House in January 2013 with its showcasing the best government data sources (and here):

‘We at White House Office of Science and Technology Policy ‘had an idea: create an online showcase, highlighting the very best Open Data resources and how they are already being used by private-sector entrepreneurs and innovators to create new products and services that benefit people in all kinds of ways—from empowering patients to find the best healthcare right when they need it; to helping consumers detect credit card fraud; to keeping kids safe by notifying parents when products in their home are recalled’.

2013-10-14_alphadatagov

OGDalphatopics

Evaluation by ODI ?

2013-10-14_odi

The website of the Open Data Institute in London provides some information about how open data are used. An interesting guest blog shows the situation on the local level.

And a press release of ODI (December 2012) gives an example in the field of prescribing practices:

MILLIONS OF POUNDS IN PRESCRIPTION SAVINGS IDENTIFIED IN OPEN DATA

‘…. This project is an example of how open data can be used to help services run more effectively and efficiently. In using open data to highlight trends, we see where we can do better and make improvements. Inefficiency in any system often results from poor or incomplete information about the overall picture. This is something that open data can address as Prescribing Analytics so dramatically illustrates. The analysis should be required reading for all GPs as they seek to make best use of their resources. …’. (ODI’s Chairman, Nigel Shadbolt).

To do

These are just some examples of evaluation work. It’s a fascinating and important work to be done in this field. And it’s worth to be done systematically.

See also https://blogstats.wordpress.com/2013/10/15/open-data-literacy/

Open Data Literacy

Governments opening data is one step, the public making use of these data is the other one.
And not a guaranteed one. As a guest blog on Harvard Business Review’s Blog says: ‘the goal is for this data to become actionable intelligence: a launchpad for investigation, analysis, triangulation, and improved decision making at all levels….. While the “opening” has generated excitement from development experts, donors, several government champions, and the increasingly mighty geek community, the hard reality is that much of the public has been left behind, or tacked on as an afterthought. So how can we support “data-literacy” across the full spectrum of users, including media, NGOs, labor unions, professional associations, religious groups, universities, and the public at large?’

OGD Toolkit

The World Bank adresses this question and provides an Open Government Data Toolkit: ‘So, now that this data has been ‘opened’, how can it capture attention and imaginations of the full spectrum of users? How can we focus on the other side – the demand side – of the open data phenomenon? How can we grow communities of data users, and encourage data ‘ownership’ by the media, civic hackers, community groups, NGOs, labor unions, professional associations, universities, and more?’

WorldBnkOGDTools

 

G8 and Open Data

Open Data is a top priority now:

g8-2013-LoughErne

G8 leaders signed the Open Data Charter on 18 June 2013.

‘The Open Data Charter sets out 5 strategic principles that all G8 members will act on. These include an expectation that all government data will be published openly by default, alongside principles to increase the quality, quantity and re-use of the data that is released. G8 members have also identified 14 high-value areas – from education to transport, and from health to crime and justice – from which they will release data. These will help unlock the economic potential of open data, support innovation and provide greater accountability.’

Screenshot-3Action 2 among the planned collective actions is the ‘Release of high value data.’

‘Action 2: Release of high value data

  • We recognise the following as areas of high value, both for improving our democracies and encouraging innovative re-use of data.
Data Category* (alphabetical order) Example datasets
Companies Company/business register
Crime and Justice Crime statistics, safety
Earth observation Meteorological/weather, agriculture, forestry, fishing, and hunting
Education List of schools; performance of schools, digital skills
Energy and Environment Pollution levels, energy consumption
Finance and contracts Transaction spend, contracts let, call for tender, future tenders, local budget, national budget (planned and spent)
Geospatial Topography, postcodes, national maps, local maps
Global Development Aid, food security, extractives, land
Government Accountability and Democracy Government contact points, election results, legislation and statutes, salaries (pay scales), hospitality/gifts
Health Prescription data, performance data
Science and Research Genome data, research and educational activity, experiment results
Statistics National Statistics, Census, infrastructure, wealth, skills
Social mobility and welfare Housing, health insurance and unemployment benefits
Transport and Infrastructure Public transport timetables, access points broadband penetration
  • In accordance with the principles of “open by default” and “quality and quantity” we will work towards the progressive publication of these data.
  • As a first step, we will collectively make key datasets on National Statistics, National Maps, National Elections and National Budgets available and discoverable (from June 2013), and we will work towards improving their granularity and accessibility (by December 2013)