Konstantinos Voudouris (Helmholtz AI, University of Cambridge), “ What Are AI Capabilities and How Can We Measure Them?”

What can AI systems do? Answering this question requires us to model their capabilities, but this first demands a clear conception of what capabilities are and which tools we can use to measure them. I advance a dispositional account of capabilities, understanding them as a system’s propensity to behave in certain ways under certain conditions. I then survey the tools we have at our disposal to measure capabilities, and what the nascent field of AI Evaluation can learn from the broader cognitive sciences.

Photo by Stefan Cosma on Unsplash