In your opinion, how does the AI Act address the data protection risks arising from the use of AI systems?

Ivana Bartoletti: The AI Act is - in some respects - product-based legislation. This means that it addresses the risks in terms of potential health and safety impacts in line with existing European legislation. However, the regulation goes beyond this by defining a number of high-risk and prohibited AI systems that could have a negative impact on the fundamental rights of Europeans - in line with the EU Charter of Fundamental Rights. This is crucial, as privacy is an essential European fundamental right.

The European AI regulation covers the aspect of privacy by applying the General Data Protection Regulation (GDPR) and is considered an essential part of tools such as conformity analysis for high-risk AI. The key point here is that violations of rights in the context of AI are, in a way, violations of privacy rights. For example, people could be denied basic services - hospital treatment or transportation - if uncritical use of personal data leads to bias.
With this new law, the relevance of data protection authorities will increase in the future, as they will play a key role with regard to high-risk AI in all member states.

To what extent could the AI Act slow down the innovation process through additional regulations? Do you see any risks that the legal framework will slow down the development of new AI products?

Bartoletti: Innovation and regulation are not fundamentally opposed to each other. I even think that regulation can stimulate innovation. Just look at a highly regulated industry like the pharmaceutical industry: although products have to go through testing before being launched on the market, innovation has never been an obstacle. Safe AI means AI that can be used by people, and the more people who use it, the more demand it can generate - and ultimately growth.

How can companies master the balancing act between complying with regulations and promoting innovation?

Bartoletti: By investing in data protection and security as well as regulation by design. We have a great opportunity ahead of us to use technology to improve data protection and security. In this context, data protection authorities have an important role to play in two ways: first, by creating a safe space for innovation where companies can experiment. Secondly, by constantly challenging the way we apply laws to new products such as large language models. For example, the Hamburg Commissioner for Data Protection and Freedom of Information recently argued that the processing of tokens and vector relationships (embeddings) by LLMs does not constitute "storage of personal data" within the meaning of the GDPR. The authority's position is interesting and constructive because it advances the debate.

The AI Act calls for more transparency in the use of AI systems. What measures should companies take to make data protection more transparent and protect users' rights?

Bartoletti: Transparency is a complex term, as it can have very different meanings. If I'm a patient in a hospital, I don't need to know how a machine works - I trust the technology and, above all, expect my health to improve. But if I'm a doctor and I rely on an AI system to help me make decisions, that system needs to be transparent so that I can question the outcome with human intelligence. What I'm trying to say is that transparency is contextual. Providing complex information to users does not automatically mean transparency if it is not understandable. For me, the key point is to ensure that systems are tested and checked before they go to market, so that users know that the products have been thoroughly tested.

How important do you think it is that users are informed when they interact with an AI and what impact does this have on data protection?

Bartoletti: It is essential to inform users that they are interacting with a machine and not a human being. This also helps to promote collaboration between the fundamentally different fields of human and artificial intelligence. It is also essential to ensure that users know how to challenge automated decisions or request human intervention. This aspect is a requirement of the GDPR and is highly relevant. Data protection supervisory authorities across Europe have played a key role in safeguarding the rights of individuals in the context of automated decision-making.

What role does the responsibility of companies for the protection of personal data in AI systems play under the new AI Act?

Bartoletti: Companies play a crucial role. Implementing AI in compliance with the GDPR is essential to foster trust and innovation. Especially in high-risk areas of AI implementation, such as labor management, organizations need to ensure that the right governance and capabilities are in place. From defining clear responsibilities - for example, processing and reviewing functions - especially when using a cloud application for data protection impact assessment, to ensuring sufficient technical and organizational measures and data security tools.

Data protection teams must be at the forefront of the use of artificial intelligence. At the same time, flexibility is required and a certain amount of experimentation is needed when it comes to finding a way to implement legal requirements in such technologies without hindering innovation.

How could companies integrate the new data protection requirements into their existing data protection guidelines?

Bartoletti : By promoting training in key areas such as privacy by design - this aspect should not be neglected. The AI Act does not introduce any new data protection requirements. It merely makes it clear that the GDPR plays a decisive role when personal data is involved in the use of artificial intelligence. Upskilling in data protection measures will be of paramount importance. We need to write data protection into the code - and not build it in as an afterthought.

Do you think that the AI Act could contribute to the harmonization of international data protection standards?

Bartoletti : The AI Act is driving forward the discussion about sensible regulation of AI. Data protection is already recognized as a universal value, although it can be interpreted in different ways. In terms of AI technologies, this will be a defining issue for future innovations. If we succeed in preserving the value of privacy and protecting personal data and the dignity of people, we will have a form of artificial intelligence that benefits people worldwide.