Artificial intelligence, once the subject of science fiction, has quietly become part of everyday life. Voice assistants book appointments, algorithms recommend what we watch and read, and specialised programs diagnose medical conditions and drive experimental cars. The remarkable progress of recent years has been driven largely by machine learning, in which computer systems improve their own performance by examining vast quantities of data. Yet the very capability that makes these systems powerful also raises difficult questions about how much human oversight they require, and how much they should be permitted to operate on their own.
The first category of concern is practical. Machine-learning systems are often described as "black boxes" because even their designers cannot explain, step by step, why a given decision was reached. When such systems approve loans, filter job applications or help judges set prison sentences, their invisible reasoning can reproduce, and sometimes amplify, the biases hidden in the data on which they were trained. Several high-profile cases have shown that an algorithm that looks neutral on its surface can in practice disadvantage particular groups of people.
A second set of concerns is economic. If intelligent systems become capable of performing a wide range of professional tasks, from legal research to radiology, the employment picture in many industries may change dramatically within a generation. Supporters argue that new kinds of work will emerge, as they have after previous waves of automation, while critics suggest that the speed and scope of the change may exceed anything seen before.
Most serious of all, in the view of some researchers, is the longer-term challenge posed by systems that approach or exceed human-level abilities in general reasoning. Even if such systems remain decades away, they argue, the question of how to align an artificial agent's goals with human values is worth thinking about now. History suggests that technologies developed without careful safeguards have sometimes proved difficult to control once widely adopted.