AI's Astonishing Progress - The Context S04E23

What are the highlights of Artificial Intelligence development in the past year or so? I define Artificial Intelligence as a Jolting Technology. Rather than a constant rate of doubling, like the two years of Moore's Law, AI progresses with a shrinking rate of doubling. Its rate of acceleration is increasing. Stanford University with open AI a few years ago published their study, looking at AI doubling in power every four months, starting with 2008. Thenthe CEO of Nvidia published their data, according to which AI is now doubling in power every two months.

Two applications of AI deserve special mention.

DeepMind released AlphaFold, which is able to predict the shape of proteins, and published the full database of the human proteome. Next year they expect to publish the shape of all the 100 million known proteins. The database is free, including for commercial applications.

Microsoft trained a special version of GPT-3 Codex, called Copilot, to assist programmers. Already over 30% of the new code written on GitHub is created with the help of Copilot.

The most important development in terms of AI platforms is the emergence of AutoML. The ability of machine learning systems to configure themselves in an unsupervised fashion.

Neural networks require huge amounts of carefully curated data, the selection of the appropriate algorithms to analyze the data, stages of fine-tuning. And then closing the feedback loop of how to introduce new sets of updated data to specifically tailor the neural network for a given application or another. An entire industry was born to support this called MLOps, Machine Learning Operations, which is how the operations for employing and deploying neural networks and machine learning should work.

Automatic machine learning is the application of neural networks to the task of analyzing fine-tuning, deploying, upgrading, and maintaining neural networks for machine learning tasks. And it will dominate the field going forward.

An important consequence is going to be that of democratizing the access to tools that previously were available only for companies with billion-dollar budgets. In the future, everyone can be, if they want to, an Artificial Intelligence engineer.