Berlin (dpa) - "Is this dark skin spot a benign birthmark or black skin cancer?" If you put a machine with artificial intelligence up against flesh-and-blood doctors to answer this question, statistically speaking, the AI machine is now more likely to be right. What's more, the AI's decisions do not depend on human mood swings, the machines only need seconds for the analysis and they don't get tired even after many hours of use.

However, even if the advantages of using AI are obvious, the German Ethics Council strongly warns against giving the machine too much responsibility. "AI must not replace humans," said Alena Buyx, Chairwoman of the Ethics Council.

Four areas examined

AI in medicine is one of four areas that the Ethics Council has analyzed in terms of the relationship between humans and machines. The use of AI in schools, in communication and opinion-forming and in public administration were also scrutinized. In October 2020, the then President of the Bundestag, Wolfgang Schäuble (CDU), asked the committee to draw up a statement on the relationship between humans and machines. At the presentation of the 287-page study in Berlin on Monday, Buyx sums up the results: "The use of AI must expand human development and not diminish it," says the medical ethicist.

For the medical sector, the report lists a number of reasons why the use of AI could be useful: for example, AI could be used to alleviate care bottlenecks due to staff shortages and provide more precise diagnoses. In medical care, AI tools are also being developed for diagnostics and therapy, for example for breast and prostate cancer.

At the same time, however, the Ethics Council sees dangers that should not be ignored: A loss of medical expertise must be avoided in the development and use of AI products. The privacy of patients must be reconciled with the intensive use of data in medical research. Completely replacing doctors with an AI system would endanger patient welfare and could not be justified by staff shortages.

Ban the use of AI in education?

In the area of education, the members of the Ethics Council discussed quite controversially whether or not the use of AI in schools should be banned in general. Specifically, the question was whether it should be allowed to record what happens in the classroom on video in order to use AI to analyze how the learning process works for individual students.

While some members of the Ethics Council rejected this form of "classroom analytics" in principle, others saw opportunities because it could provide teachers with continuous feedback and significantly improve teaching results. As a compromise, it was agreed that data collection and provision should serve the learning process. "It must not be misused to monitor and stigmatize learners," said Julian Nida-Rümelin, Deputy Chairman of the German Ethics Council.

AI in communication and opinion-forming - critical use case

The Ethics Council is also predominantly critical of the role of AI in public communication and opinion-forming. The polarization and brutalization of discourse has increased in social media, Nida-Rümelin lamented. It is often only possible to stand out from the "white noise" of the networks with particularly extreme views. The former Minister of State for Culture also blamed this on the commercial interest of platforms such as Facebook and Twitter in retaining users for as long as possible. This is achieved with personalized offers that are tailored to the recipients with the help of AI.

However, AI is also used to moderate problematic or even criminal content. This creates a tension between freedom of expression and the civil culture of communication, which is resolved in the hands of private digital companies according to their own rules, said Nida-Rümelin. At this point, the Ethics Council is proposing a public service alternative so that this issue is not left to private commercial companies alone. "We are not referring to public broadcasters," said Nida-Rümelin. Public-law foundation models without major state influence are conceivable.

AI in administration and homeland security

AI is increasingly being used in public administration to support decisions and ensure greater efficiency, reported Judith Simon, Professor of Ethics in Information Technology at the University of Hamburg. However, it has not been proven that its use necessarily leads to better decisions. The Ethics Council is committed to ensuring that citizens are protected from discrimination. Machine recommendations should not be followed blindly. Furthermore, case-by-case considerations as well as the rights of inspection and objection of those affected must be guaranteed.

This also applies to the use of AI to improve internal security. "If software predicts that 99% of a person will commit a crime, we can never know whether the person in front of us is exactly one percent."

AI and skills shortage - untapped potential

Research Minister Bettina Stark-Watzinger (FDP) explained that the Ethics Council illustrates "the diverse potential and opportunities" associated with the use of artificial intelligence in key areas of social application. "We absolutely must make use of these." At the same time, the report provides guidance on how possible risks such as discrimination in the use of AI can be countered.

The AI association was more critical of the report: It does not focus enough on the positive effects of AI, explained the association's managing director Daniel Abbou. The opportunities that AI will have in an economic context, particularly in relation to the shortage of skilled workers, were ignored.