There are really big problems in education, health, and energy that could benefit from advanced machine learning techniques made possible by the suite of so-called big data technologies.

Why is it possible to solve them now? Why aren’t they solved yet?

It’s because technologies for distributed storage and processing, pioneered and open-sourced by companies like Google, are available. Distributed computing systems, like but not restricted to ‘the cloud’, have brought down the cost of such operations. Finally, enough people have spent enough time trying, failing, and succeeding, to be able to use those technologies successfully.

In other words, there are very good physical technologies and social technologies that are now in place.

Ontario has very huge, centralized, repositories of very big data because the government dominates all three of those fields.

This is unlike, say, California’s fragmented HMO system, or, several energy markets in the United States.

Ontario, and Canada more generally, with its system of public health, public utilities, and public education, are fairly well positioned to take advantage of these technologies at scale.

Smaller governments will need to be smarter if they’re to maintain the same level of effectiveness.

2 thoughts on “Tackling the big problems

  1. As with most problems in the public sector, technology isn’t the problem – people are the problem. Unless there’s enough political will to push through initiatives to make our data (in Ontario, it really is OUR data) open, readable, searchable, downloadable etc., it just won’t happen.

    Case in point. For a project I did in my MBA class, I needed stats from the Ministry of Health. I had to fill out a form stating which stats I wanted & why. The Ministry then sent me a CD via courier. On that CD was one Excel file, which had 5 rows and 2 columns.

    We need better people – the machines have done their part.

  2. Hi Martin,

    Great insight as usual.

    It’s a social social problem.

    What do?

Comments are closed.