AI Watch: Assessing Technology Readiness Levels for Artificial Intelligence

This AI Watch report “Assessing Technology Readiness Levels for Artificial Intelligence” aims to define the maturity of an illustrative set of AI technologies through the use of Technology Readiness Level (TRL) assessment.

Key points

• Novel exemplar-based methodology to categorise and assess the maturity levels for several AI research and development technologies, by mapping them onto Technology Readiness Levels (TRL).
• Interpretation of the nine TRLs (introduced by NASA and adapted by the EU) in the context of AI, and systematic application to different categories in AI, by choosing one or two exemplars in each category.
• Development of new bidimensional plots (readiness-vs-generality charts) as a trade-off between how open-ended a technology is versus its readiness level.
• Discussion about the future of AI as a transformative technology and how the readiness-vs-generality charts are useful for short-term and mid-term forecasting.

What are the key policy-relevant findings? 

• Not only do we lack the tools to determine what AI achievements will be attained in the near future, but we even underestimate what various technologies in AI are capable of today.
• AI research breakthroughs do not directly translate to a technology that is ready to use in real-world environments.
• This report provides a novel example-based methodology to categorise and assess several AI research and development technologies, by mapping them onto Technology Readiness Levels (TRL) (representing their maturity and availability). 
• The AI technologies analysed are used as exemplars that work as practical guidelines for anyone interested in analysing other AI technologies using a similar methodology. 
• The examples selected in this report are sufficiently representative for a discussion about the future of AI as a transformative technology and how these charts can be used for short-term and mid-term forecasting. 

What has been the most interesting observation with your research so far?

• Methodologically, the AI technologies analysed have served to illustrate the difficulties of estimating the TRLs, a problem that is not specific to AI. The use of layers of generality, however, has helped us be more precise with the TRLs than would be otherwise.
• In the analysis of the charts for the different AI technologies we see that higher TRLs are achievable for low-generality technologies focusing on narrow or specific abilities, while low TRLs are still out of reach for more general capabilities.
• High TRLs for high layer generalities should indicate potential short-term or mid-term massive transformative power.
• The shapes of the readiness-vs-generality charts are informative about where the real challenges are for some technologies.
  • A very steep curve suggests that there may be a long way to go from one layer of the technology to the next one.
  • A flatter curve may correspond to situations where the fundamental ideas are already there, and progress could be smoother.
• We may use the dynamics of several AI technology exemplars at different generality layers and moments of time to forecast some short-term and mid-term trends for AI.
• We must not confuse readiness with success of a technology since the latter is an even more difficult variable to estimate, as many social and economic factors may interplay.

Details on JRC Publications Repository