The State of AI in 2021

We take a deep dive into the key issues of the annual State of AI report and the winners and challenges for the next year.

Did the State of AI’s predictions come true in 2019?

2019 Predictions:

  1. New natural language processing companies raise $100M in 12 months.


AI research is less open than you think: only 15% of papers publish their code.

What tools and resources are researchers using?

Benaich and Hogarth used conference papers to determine the most used framework. Of those that specify the frameworks used, 75% cite the use of PyTorch but not TensorFlow. However, TensorFlow, Caffe and Caffe2 are still the workhorse for production AI. PyTorch offers greater flexibility and a dynamic computational graph that makes experimentation easier. JAX is a Google framework that is more math friendly and favoured for work outside of convolutional models and transformers.

A new generation of transformer language models are unlocking new NLP use-cases

GPT-3, T5, BART are driving a drastic improvement in the performance of transformer models for text-to-text tasks like translation, summarization, text generation, text to code. For example, an unsupervised machine translation model trained on GitHub projects with 1,000 parallel functions can translate 90% of these functions from C++ to Java, and 57% of Python functions into C++ and successfully pass unit tests. No expert knowledge required, but no guarantees that the model didn’t memorize the functions either.

Biology and Health sciences are repeating the benefits

Biology is experiencing its “AI moment” with over 21,000 papers in 2020 alone involving AI methods (e.g. deep learning, NLP, computer vision). Papers published since 2019 account for 25% of all output since 2000.

The COVID Moonshot project

Drug discoveries have gone open source to tackle COVID-19. Benaich and Hogarth assert this is a rare example of where AI is actively used on a clearly-defined problem that’s part of the COVID-19 response. An international team of scientists are working pro-bono, with no IP claims on a project called COVID Moonshot to crowdsource a COVID antiviral.

Federated learning is booming

Kicked off by Google in 2016, federated learning research has experienced almost 5x growth in the number of papers that mention federated learning from 2018 to 2019. More papers have been published in the first half of 2020 than in all of 2019.

Improved validity in AI research

A review of 20,000 recent AI-based medical imaging studies found that less than 1% of these studies had sufficiently high-quality design and reporting. Studies suffer from a lack of external validation by independent research groups, generalizability to new datasets, and dubious data quality. This year new international guidelines were drafted for clinical trial protocols and reports that involve AI systems in a bid to improve both quality and transparency. New requirements include:

  • “State which version of the AI algorithm will be used.”
  • “How was input data acquired and selected.”
  • “How was poor quality or unavailable input data assessed and handled.”
  • “Was there human-AI interaction in the handling of the input data, and what level of expertise was required?”
  • “Describe the onsite and offsite requirements needed to integrate the AI intervention into the trial setting.”
  • “How can the AI intervention and/or its code be accessed, including any restrictions to access or re-use.”

AI and Machine learning have gone mainstream

Benaich and Hogarth offer compelling evidence as to the mainstreaming of AI ideas and technologies. They cite the example of the rise of MLOps (DevOps for ML) as a signal of an industry shift from technology R&D (how to build models) to operations (how to run models).

Ethics on the front line

AI Ethics remains a hot topic in research, media and social commentary. 50% of the world currently allows the use of facial recognition. Only three countries (Belgium, Luxembourg, and Morocco) have partial bans on the technology that only allow it in specific cases.

8 AI predictions for the next 12 months

Before closing this article, here are the 8 most significant prediction that we should expect to happen in 2021:

  1. The race to build larger language models continues and we see the first 10 trillion parameter model.
  2. Attention-based neural networks move from NLP to computer vision in achieving state of the art results.
  3. A major corporate AI lab shuts down as its parent company changes strategy.
  4. In response to US DoD activity and investment in US-based military AI startups, a wave of Chinese and European defence-focused AI startups collectively raise over $100M in the next 12 months.
  5. One of the leading AI-first drug discovery startups (e.g. Recursion, Exscientia) either IPOs or is acquired for over $1B.
  6. DeepMind makes a major breakthrough in structural biology and drug discovery beyond AlphaFold.
  7. Facebook makes a major breakthrough in augmented and virtual reality with 3D computer vision.
  8. NVIDIA does not end up completing its acquisition of Arm.



We help tech communities to grow worldwide, providing top-notch tools and unparalleled networking opportunities.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store

We help tech communities to grow worldwide, providing top-notch tools and unparalleled networking opportunities.