Safe AI: Distinguishing Performed Competence from Reliable Behavior

Benchmarking has become a dominant driver of progress in AI and machine learning – rewarding systems that perform well on standardized tasks. Yet strong benchmark performance does not necessarily indicate reliable or safe behavior. In this keynote, I use fairness-aware machine learning as a lens to examine how benchmark-driven evaluation can conflate performed competence with genuine understanding and reliability. I show how framing fairness as a black-box optimization problem – balancing fairness metrics against accuracy – can be misleading, produce unintended side effects, and in some cases harm the very groups such interventions aim to protect. I argue for a shift away from competitive benchmarking toward evaluation practices that support deeper understanding: approaches that reveal how AI systems behave across contexts, where and why they fail, and what their limitations imply for safety and trustworthiness.