Transparency, accountability, and ethics: what regulations for artificial intelligence?

“A new era is emerging,” explains Bruce Daley, researcher at Stanford University. According to him, “technologies modeled on the human brain, such as deep learning, are performing tasks as diverse as medical diagnosis, risk assessment, program trading, fraud detection, product recommendation, image classification, speech recognition, translation, and autonomous driving. And the first results already speak for themselves.”¹

Artificial intelligence (AI) is considered one of the most promising technologies of our time, as it offers numerous opportunities across industries such as healthcare, finance, manufacturing, transportation, and many others. However, it also raises significant concerns, particularly regarding potential risks to employment, privacy, security, and national sovereignty. Understanding both the benefits and the risks of AI is essential to maximize its value while minimizing potential harm.²

Artificial intelligence fascinates as much as it concerns

Artificial intelligence has experienced tremendous growth since the 2010s. Very quickly, the GAFAM companies (Google, Apple, Facebook, Amazon, Microsoft) invested heavily in AI research and development by creating dedicated teams focused on advancing the technology.

Since 2020, following the COVID-19 pandemic, the use of AI has accelerated across numerous sectors, including disease detection, vaccine and treatment research, outbreak prediction, as well as strengthening cybersecurity, fraud prevention, and crime detection mechanisms.

The AI market is expected to reach $11.1 billion by 2025. Gartner estimates that by 2035, AI could increase global productivity by 40%.³

The launch of certain AI technologies was initially widely praised for their relevance and broad knowledge across multiple fields. However, this admiration gradually evolved into skepticism and concern as users became more aware of the risks associated with these technologies. In December 2022, the journal Nature expressed concerns regarding researchers’ ability to distinguish AI-generated abstracts from those written by scientists, highlighting the difficulty of establishing complete trust in such systems.

In light of these developments, it is important to recognize that AI mechanisms may infringe on privacy, introduce discrimination, manipulate individuals, or even generate physical, psychological, or economic harm. It is therefore crucial to carefully assess both the opportunities and risks associated with AI to ensure its responsible use and reduce potential dangers.

Artificial intelligence: an emerging regulatory framework

There are currently few regulations specifically dedicated to artificial intelligence. However, governments and international organizations are actively working to establish rules aimed at governing AI usage and mitigating associated risks. Ensuring that AI complies with existing laws and ethical standards — particularly regarding privacy protection and non-discrimination — has become a major priority.

Although AI systems were already partially regulated through the General Data Protection Regulation (GDPR) in 2018, the European Commission adopted a proposal on April 21, 2021: the AI Act, intended to create “a legal framework for trustworthy AI.”

This future regulation, which would become the second major extraterritorial European regulation after GDPR, could come into force between 2023 and 2026. Its objectives are clear:⁴

• Position the European Union as a leading authority in AI regulation
• Build a coherent European digital strategy that respects individual freedoms and fundamental rights
• Foster cooperation among member states and prevent market fragmentation
• Facilitate the development of a single market for lawful, safe, and trustworthy AI applications

“The EU wants to establish a level playing field for all market players and become an international benchmark for AI regulation.”

Thomas Skordas, CNRS researcher

A key role for providers, distributors, and users to ensure AI governance and oversight

Providers, distributors, and users are all impacted by the deployment and monitoring of AI systems, as each plays a distinct role in their implementation and governance.

The Artificial Intelligence Act will apply to the following actors:⁵

• Providers of AI systems established within or outside the European Union who place AI systems on the EU market or put them into service within the EU
• Distributors of AI systems located outside the EU when the outputs generated by these systems are used within the European Union
• Users (legal entities) of AI systems operating within the European Union

Providers responsible for developing and supplying AI technologies will face numerous obligations before bringing AI systems to market or deploying them within the EU, including conformity assessments, quality management requirements, reporting obligations to national authorities, and technical documentation obligations.

Users (companies) leveraging these technologies to improve operations and efficiency will also have responsibilities. They must comply with providers’ requirements (such as terms of use), ensure human oversight of AI systems, verify the appropriate use of processed data, and notify providers of incidents or risks.

The Artificial Intelligence Act will not apply to:

• Public authorities of third countries or international organizations using AI systems within the framework of international police or judicial cooperation agreements with the EU or member states
• Purely private and non-commercial use of AI systems

Regulatory content and impacts for providers, distributors, and users

The proposed regulation follows a risk-based approach, classifying AI systems according to the level of risk they present to individuals. Regulators distinguish four categories of AI systems:

• Unacceptable-risk AI systems
  These include systems that manipulate behavior or lead to discrimination, particularly through social scoring mechanisms (for example, denying credit based on social behavior). Such systems are prohibited.
• High-risk AI systems
  Eight categories have been identified as high-risk, including vocational training, access to public services, border control, and more. These systems require robust risk management mechanisms, transparency toward individuals, and human oversight. Once compliant, these systems receive CE marking certifying both AI compliance and the protection of individual rights.
• AI systems subject to transparency obligations
  These systems interact with humans, detect emotions, or generate manipulated content (such as chatbots). They may be deployed provided users are clearly informed and a code of conduct is implemented.
• Minimal or no-risk AI systems
  Examples include predictive maintenance systems.

Failure to comply with the obligations established by the Artificial Intelligence Act may result in significant financial penalties and reputational damage.

As with GDPR, the AI Act introduces substantial fines for non-compliance:⁶

• Up to €30 million or 6% of global annual turnover in cases involving moral or physical harm, behavioral manipulation, discrimination, or failure to comply with risk-based principles
• €10 million or 2% of global annual turnover for misleading or inaccurate information
• €20 million or 4% of global annual turnover for other breaches of the AI Act

Beyond financial penalties, companies may also face reputational damage if they fail to comply with regulatory obligations. As with GDPR sanctions, penalties may be publicly disclosed by supervisory authorities.

The Artificial Intelligence Act reflects strong ambitions from the European Union and will significantly impact all AI stakeholders operating within the European market.

Through this proposed regulation, the European Union once again demonstrates its willingness to establish extraterritorial regulations capable of competing with dominant players from Asia and North America. After focusing on personal data protection and individual rights through GDPR, the European Commission is now regulating AI technologies themselves and their implementation frameworks to encourage both innovation and ethics while positioning Europe as a leading AI player.

Companies must therefore prepare to strengthen their risk management approaches in the coming years to ensure compliance with evolving regulations while addressing new challenges generated by the rapid evolution of technology.

Célia Allouche
Senior Data Privacy & Data Governance Consultant

Eliott Mourier
Data Compliance & Data Privacy Manager