Geoffrey Hinton, the father of deep learning, said a few things at the ReWork Deep Learning Summit in Toronto last week. Hinton often looks to biology as a source for inspiration. I’ll share and expand in this post. Hinton started off with an analogy. A caterpillar is rally a leaf eating machine. It’s optimized to eat leaves. Then it turns itself into goo and becomes something else, a butterfly, to serve a different purpose. Similarly, the planet has minerals. Humans build an infrastructure to transform earth into paydirt. And then a different set of chemical reactions are applied to paydirt to yield gold, which has some purpose. This is much the same way that training data is converted into a set[…]

There’s a quote from The Office (US) [Season 6, episodes 5/6, “Launch Party”]: Michael: Okay, okay, what’s better? A medium amount of good pizza? Or all you can eat of pretty good pizza? All: Medium amount of good pizza. Kevin: Oh no, it’s bad. It’s real bad. It’s like eating a hot circle of garbage. The launch in that episode was the ill fated “Dunder Mifflin Infinity”, and while the reference in the passage is to the pizza that Michael Scott had ordered, it may as well been referring to the website. For many reasons, people tend to build all you can eat hot circles of garbage, instead of a medium amount of pretty good pizza. Minimum Viable Product and[…]

Jon Evans wrote a piece for Techcrunch entitled: After the end of the startup era. In it, Evans writes: We live in a new world now, and it favors the big, not the small. The pendulum has already begun to swing back. Big businesses and executives, rather than startups and entrepreneurs, will own the next decade; today’s graduates are much more likely to work for Mark Zuckerberg than follow in his footsteps. And, Because we’ve all lived through back-to-back massive worldwide hardware revolutions — the growth of the Internet, and the adoption of smartphones — we erroneously assume another one is around the corner, and once again, a few kids in a garage can write a little software to take[…]

It was a treat to see these three – Yoshua Bengio, Yann Lecun, and Geoffrey Hinton – for an afternoon. Easily the best three consecutive hours I’ve ever seen at a conference. They remarked that Canada continues to invest in primary research. And this is a strength. Much of the exploratory work these three executed in the 80’s, 90’s and naughties was foundational to industrial applications which came after. Much of reinforcement and deep learning has moved on into industrial application. For the three grandfathers of deep learning, all of these algorithms and methods move into the realm of solved problems. For those of us in industry, there remains a lot of work to realize the benefits of deep learning.[…]

Hinton is quoted as saying, with respect to back propagation, “I don’t think it’s how the brain works”. You can read the full article here. Back Propagation To oversimplify, in Back Propagation, the influence of each neuron is rewarded based on how well it predicts something. Accurate predictions are rewarded with more influence. Bad predictions are punished with less. This is how the machine learns. And there’s a lot of optimism about Back Propagation. It’s really useful and generates fairly predictable machines. As data scientists, we like this. And as data scientists, we should also like what Hinton is hinting at. Kuhn It’s much more likely than not that we’re approaching a local maxima on this thread of research. I’m[…]

The other I likened the process for taking apart a Job To Be Done to taking a part a lobster. There’s a very effective way to decompose any problem with enough energy. And then I watched The Founder on Netflix and admired the McDonald brothers using a classic technique in management science to refine a system on a tennis court. And I loved it. They really refined hamburger and frenched fry delivery. And then this morning I read that Andrew Ng in working on a new coursera course for AI. And I’m thankful for his initiative and optimism. Out of those three threads, this one post. The Assembly Line The assembly line was an American invention for Americans. It could[…]

Earlier in the month, I dined under the space shuttle Endeavour with some of the best minds in marketing science. One mind remarked: “That’s why I bring a glossary with me, oh, you want to do supervised learning? Oh you mean regression? Oh, okay, now we can talk… We’ve been talking to managers about these methods for decades, but it’s just suddenly sexy because it’s all machine learning and deep learning and reinforcement learning.” A lot of the math that underlies much of machine intelligence and artificial intelligence is indeed remarketed marketing science. And, hipsterism aside, the annoyance is understandable. Marketing science started out a bit of a revolt against the Mad Men. Some of the early stories feature post-war[…]

“A study at Ball State University’s Center for Business and Economic Research last year found that trade accounted for just 13 percent of America’s lost factory jobs. The vast majority of the lost jobs — 88 percent — were taken by robots and other homegrown factors that reduce factories’ need for human labor.” – AP Canada’s labour force is around 19.6 million people, of which 18.2 million people are employed. Together, they worked something like 2.4 billion hours that month. In December 2016, something like 1.7 million Canadians worked about 240 million hours in manufacturing.  Roughly. Because of seasonal adjustments and different data at different times. And error. In terms of our working lives in Canada, collectively, manufacturing is about 10% of[…]

A great mind in public policy told me, just this last September, that people are really bad at judging the rate of technological change and when it’ll affect them. It’s like standing on a railway. You can see the train out there. Some people assume that the train is going to hit them very soon. They get off the tracks. Then, when the train is getting very close, others misjudge the speed and assume that it’s still a far way. And then they get hit. It’s a great analogy because it combines prediction with decision. The rate of technological change is actually quite difficult to predict. If it was easy there’d be a lot more successful startups. One Heuristic Start[…]

You’re going to hear a lot more about Artificial Intelligence (AI) more generally, and Machine Intelligence more specifically. Valuation is the core causal factor. Here’s why: We’ve gotten pretty good at training a machine on niche problems. They can be trained to a point to replace a median-skilled/low-motivated human in many industries. Sometimes they can make predictions that agree with a human’s judgement 85 to 90% of the time, and sometimes, it’s the human that’s causing the bulk of the error to disagree with the machine. We’re confident that we can train a machine to learn a very specific domain. And these days we’re in the midst of that great automation revolution. Most of the organization that build those machines can[…]