The European Union is getting serious about regulating powerful AI models, and it looks like they’re making some progress. European countries are currently discussing potential concessions as they prepare for negotiations on the world’s first comprehensive AI rulebook. The Spanish presidency of the EU Council of Ministers recently shared a document outlining possible compromises on the AI Act, which aims to regulate AI based on its potential to cause harm.
One of the key areas being discussed is how to handle foundation models. These are large machine-learning models that are trained on massive datasets and can generate responses based on specific stimuli. The European Parliament has suggested that these models should be subject to a specific regulatory regime, and the Council seems to be on board with that idea. The Spanish presidency has proposed a definition of foundation models as AI models that are capable of performing a wide range of tasks competently. They have also suggested implementing benchmarks to assess these models’ capabilities. Transparency obligations, such as documenting the modeling process and evaluating benchmarks, would be required for all foundation models before they hit the market.
In addition to foundation models, the Spanish presidency is also proposing a new category called “very capable foundation models.” These models would be subject to additional obligations because their capabilities go beyond the current state-of-the-art and may not be fully understood yet. External red teams would regularly vet these models, and independent auditors would perform compliance controls before they are launched on the market.
Another category being discussed is general-purpose AI systems. These systems are built on foundation models and are used at scale. The obligations for these systems include regular external vetting to uncover vulnerabilities and the establishment of a risk mitigation system. Providers of general-purpose AI systems would also need to state if their system can be used for high-risk applications and take appropriate action.
Copyright is another important issue being addressed. The presidency wants foundation model providers to demonstrate that their models are trained in compliance with EU copyright law, allowing rightsholders to opt out if necessary. They also want generative AI models to have a technical solution in place to detect artificially generated or manipulated output.
On the governance front, there is a proposal to establish an AI Office that would oversee the new rules on foundation models and general-purpose AI systems used at scale. This office would define auditing procedures, carry out compliance controls, and investigate any potential violations.
There are also discussions surrounding the use of real-time biometric identification systems by law enforcement. The European Parliament wants a complete ban, but EU governments want to keep some exceptions. The proposed compromise would limit the use of these systems to specific cases, such as searching for victims of abduction and human trafficking, preventing terrorist attacks, and prosecuting the most severe crimes. Additional safeguards, including obtaining judicial authorization for using these systems, are also being considered.
Other contentious issues include the ban on emotion recognition in certain areas like law enforcement, border management, workplaces, and educational institutions. The Spanish presidency proposes a limited prohibition but with certain exemptions. They also suggest a ban on the biometric categorization of protected data, with a potential carve-out for law enforcement. Predictive policing is another topic of debate, with discussions around additional wording and the prohibition of social scoring practices.
Overall, the negotiations on the AI Act are entering their final phase, and there seems to be progress on various fronts. The EU is taking a comprehensive approach to regulating AI, aiming to balance innovation with protecting citizens’ rights and ensuring accountability. It will be interesting to see how the negotiations unfold and what the final AI rulebook will look like.
