Imagine: a digital assistant, in order not to exceed the budget, cancels an already planned trip itself - even though you did not ask it to. Who is to blame? This example from Giacomo Zanotti's research illustrates the main ethical problem of so-called agentic artificial intelligence (agentic AI) - systems that are capable of setting goals and independently choosing actions to achieve them, without requiring human participation at every step.
The interdepartmental project META - Social sciences and humanities for science and technology at the Polytechnic University of Milan studied how the development of such systems is changing the relationship between humans and machines. One of the key conclusions: the more autonomous AI becomes, the harder it is to determine who bears responsibility for its actions. Zanotti's research revealed a fundamental tension between the growing independence of systems and the possibility of assigning blame to someone for their decisions.
Another work, by Camilla Quaresmini, concerns the production and evaluation of knowledge. In hospital settings, AI can analyze medical reports and test results, offering a clinical assessment. A question arises: will the algorithm's judgments receive greater weight than a doctor's observations or a patient's account? The research identified three roles of AI in cognitive processes: systems that organize information, systems that make assessments, and systems that actively participate in research. It was separately noted: maintaining human control in itself does not guarantee that people's opinions will be trusted more than the conclusions of artificial intelligence.
The third area is Fabio Iapaolo's work on self-driving cars, a striking example of "physical agentic AI," which acts not in a digital environment but directly in the material world, including cities. The research calls into question the idea that a decision can be attributed to a single subject - a human or a machine. In practice, a decision is formed in the interaction of sensors, algorithms, control devices and the surrounding environment. Such a view requires expanding ethical reflection: taking into account not only individual decisions and their consequences, but also the technical, material and social connections through which systems act in the world.
"In conditions where agentic AI systems are used in increasingly diverse areas - from healthcare to mobility and the organization of work - three studies show that the growing autonomy of artificial systems raises questions that go far beyond the automation of tasks and decisions," explain Zanotti, Quaresmini and Iapaolo. The project offers tools for making sense of these changes and their ethical and social consequences.