Tackling the big problems

There are really big problems in education, health, and energy that could benefit from advanced machine learning techniques made possible by the suite of so-called big data technologies.

Why is it possible to solve them now? Why aren’t they solved yet?

It’s because technologies for distributed storage and processing, pioneered and open-sourced by companies like Google, are available. Distributed computing systems, like but not restricted to ‘the cloud’, have brought down the cost of such operations. Finally, enough people have spent enough time trying, failing, and succeeding, to be able to use those technologies successfully.

In other words, there are very good physical technologies and social technologies that are now in place.

Ontario has very huge, centralized, repositories of very big data because the government dominates all three of those fields.

This is unlike, say, California’s fragmented HMO system, or, several energy markets in the United States.

Ontario, and Canada more generally, with its system of public health, public utilities, and public education, are fairly well positioned to take advantage of these technologies at scale.

Smaller governments will need to be smarter if they’re to maintain the same level of effectiveness.