Will open government data cause better public policy?

There’s a pretty big movement afoot in Canada. It’s called the open data movement and several levels of government are getting on board.

It’s the movement for governments to make large datasets freely available to the public.

It’s pretty rough going right now. I’m reading reports that the sets frequently lack a data dictionary and suffer from some pretty bad accessibility issues. The early efforts are to be commended. I’ve spoken to several government statisticians who are both excited and frustrated by what they’re able to share with the public, and where they’re totally blocked. They’re bullish on this movement.

These pains in the public sector mirror those in the private sector.

Will open data cause better public policy, and by extension, a better society?

I’m optimistic that it will in Canada.

For one, the evidence borne from the scientific method, as applied to datasets that everybody can see, are likely to be accepted for what they are, as evidence. We have a fairly well educated urban society that believe in pragmatic, incremental testing and an incredibly well developed rural society that uses science every day to build better livestock, soils, and crops. Nearly every sector of Canadian society is educated.

For two, evidence still matters in making many public policy decisions, at least, in many municipalities and provinces. Politicians will seek evidence, regardless if it’s really about convenient reasoning, to back up decisions.

For three, more eyes on the same data, subjected to more analytical scrutiny, ought to generate more evidence and better insight. A generation of companies, emergent in my neighborhood, are motivated to generate new algorithms and services to interpret that data to generate profit first, and social goods second. More evidence, disseminated through to the public, ought to create a more informed public and better decision making.

Skepticism

If there’s a desire for better public policy on the part of constituents, a desire for better evidence by government officials and representatives, and interest from analytics practitioners and entrepreneurs from the private sector crossing into this space, what could go wrong?

Many things.

For one, selling to the public sector is extremely risky. If practitioners can’t imagine alternative niche markets to buttress their risk profile and ensure sufficient margin, then there will be less investment in that sector.

For two, spending on making open data more open may screech to halt owing to the deep spending cuts that will follow in an effort to correct nasty structural deficits that have developed.

For three, the chain of causality between open data, new value creation, better public policy, and a better society, is extremely long. Chains that exceed one link are hard enough to follow, little though prove. There’s an awful lot of opportunity for unseen factors to disprove the theory.

On balance

The origin of many of the methods common in analytics goes back to the public policy debate around temperance. Evidence was important then. Evidence is important now. There are good indicators that open data can really work in Canada.

It’s a great time to be in analytics and for public policy.