This dimension of the AI Watch Index addresses the evaluation of the technical progress in an illustrative set of AI tasks found in different subareas (including image classification, face recognition, speech recognition, text summarisation, etc.).
The aim of this dimension is to provide information about the technological evolution that the domain is experiencing. In addition, the technological dimension is also investigated with regard to the standardisation work that is carried out by Standardisation Development Organisations supporting interoperability, promoting consistent application of best practices in the development of AI products and services, and ensuring proper consideration is given to their potential risks in terms of safety and fundamental rights.
The results show an overall improvement in performance of all analysed AI tasks from 2016 to 2021, with higher growth rates for the tasks that start with lower levels (Computer Vision – Video and Natural Language Processing – Language Reasoning Skills). The best performing AI task is Natural Language Processing – Speech.
In the standardisation sphere, intense activity is observed both nationally and internationally. Indeed, many standardisation deliverables regarding AI (including standards, as well as technical specifications, technical reports and certification criteria) have been identified. Importantly for the EU, a substantial number of these are significant in the context of the requirements for AI systems laid down in the European AI Act. This is a promising sign for the harmonisation of further innovations in the field.
AI Technology indicators
T1: Performance of AI research
This indicator addresses evaluation of technical progress in an illustrative set of AI tasks belonging to different sub-areas using a combination of quantitative measurements, such as in popular AI benchmarks and prize challenges.
T2: Standardisation activity engagement
This indicator provides an overview of the AI standardisation landscape across multiple international and European standards development organisations (SDOs).