Category: Deep Learning

The Generative Era

Posted on by Christopher Berry

I want to believe that the current generation of applications powered by Large Language Model (LLM) don’t represent the height of the state of the art for prediction machines and that no single firm will reach 80% market share and go onto dominate the generative era. I want to believe that the future is quite open, and that these early returns we’ve made in applied machine learning can compound. It’s because I want to believe so much that it’s worth questioning the assumptions. What might cause the future to become closed? Does OpenAI scan its API logs for good ideas? In 2023, a surge of founders developed skins for OpenAI’s ChatGPT. Some based their startup on enabling a user to[…]

On Synthetic Media, Synthetic Audiences, And Facts

Posted on by Christopher Berry

Generative Pre-trained Transformers (GPTs) have captured the public’s imagination. There’s a lot of fear. The relevance of that fear, as always, depends on who you are. The technology first caused a surge of panic in December 2022 among some communications professionals. They fear a surge of supply, a massive increase in synthetic media, along with all of the misinformation that goes along with it. Because attention is inelastic, the price for content will collapse, and it’ll take their wages down with it. They aren’t the only ones likely to be affected [1]. It’s kind of curious that you aren’t reading too much publicly from developers and their experiences with the technology. GPT’s are a tool. A good GPT is capable[…]

Generative Media

Posted on by Christopher Berry

You may have noticed a lot of sharing of Open AI’s Generative Pre-trained Transformer 3 (GPT-3) and ChatGPT screenshots. The most popular tend to be funny. I asked ChatGPT to describe in a style that Stephen King would write, a scene describing somebody struggling to decide which brand of toilet paper to buy. The result is fun. Here’s one about writing a script in Seinfeld style: In the sense that you can recognize the blend of styles. Elements of relevance and divergence are provided by the prompts. The evaluation, on the part of this author, is entirely subjective. And in some ways, carbon dated. Years from now, the bubble sort will be around. Probably. The knowledge of what a typical[…]

Deep Learning: Evolution or Revolution?

Posted on by Christopher Berry

Is what is happening in analytics, in industry, an evolution or a revolution? What is Analytics is the science of data analysis. Those who practice analytics self-identify as analyst, digital analyst, marketing scientist, data engineer, researcher, among many others. Tukey (1962, The Future of Data Analysis, The Annals of Mathematical Statistics, (33), 1) called them all practitioners. The goal of the practitioner depends on their context. That context largely, but not always, depends on the state of knowledge, state of the culture, or sometimes, normatively, the state of maturity, of the group they belong to. Large organizations can have a large amount of difference within them. It’s not uncommon for an operations department to be extremely mature and for its[…]

Hinton’s Biological Insight

Posted on by Christopher Berry

Geoffrey Hinton, the father of deep learning, said a few things at the ReWork Deep Learning Summit in Toronto last week. Hinton often looks to biology as a source for inspiration. I’ll share and expand in this post. Hinton started off with an analogy. A caterpillar is rally a leaf eating machine. It’s optimized to eat leaves. Then it turns itself into goo and becomes something else, a butterfly, to serve a different purpose. Similarly, the planet has minerals. Humans build an infrastructure to transform earth into paydirt. And then a different set of chemical reactions are applied to paydirt to yield gold, which has some purpose. This is much the same way that training data is converted into a set[…]