Geoffrey Hinton, the father of deep learning, said a few things at the ReWork Deep Learning Summit in Toronto last week.┬áHinton often looks to biology as a source for inspiration. I’ll share and expand in this post. Hinton started off with an analogy. A caterpillar is rally a leaf eating machine. It’s optimized to eat leaves. Then it turns itself into goo and becomes something else, a butterfly, to serve a different purpose. Similarly, the planet has minerals. Humans build an infrastructure to transform earth into paydirt. And then a different set of chemical reactions are applied to paydirt to yield gold, which has some purpose. This is much the same way that training data is converted into a set[…]

It was a treat to see these three – Yoshua Bengio, Yann Lecun, and Geoffrey Hinton – for an afternoon. Easily the best three consecutive hours I’ve ever seen at a conference. They remarked that Canada continues to invest in primary research. And this is a strength. Much of the exploratory work these three executed in the 80’s, 90’s and naughties was foundational to industrial applications which came after. Much of reinforcement and deep learning has moved on into industrial application. For the three grandfathers of deep learning, all of these algorithms and methods move into the realm of solved problems. For those of us in industry, there remains a lot of work to realize the benefits of deep learning.[…]