Ensuring the security of an AI system’s development

02 July 2024

The security of AI systems is an issue too often overlooked by their designers. However, it remains an obligation to guarantee data protection both during the development of the system and in anticipation of its deployment. This how-to sheet details the risks and measures to be taken as recommended by the CNIL.

This content is a courtesy translation of the original publication in French. In the event of any inconsistencies between the French version and this English translation, please note that the French version shall prevail.

 

The security of the processing of personal data is an obligation laid down in Article 32 of the GDPR. It states that it must be implemented taking into account “the state of the art, the costs of implementation and the nature, scope, context and purposes of processing as well as the risk of varying likelihood and severity for the rights and freedoms of natural persons”. The security of processing is therefore an obligation to be implemented by risk-appropriate measures.

In practice, for the development of an AI system, a “traditional” security analysis should be combined, in particular on environmental security (including infrastructure, clearances, backups, physical security) and the security of software development and its maintenance (which includes in particular the implementation of good development practices, or the management of vulnerabilities and updates) with a risk analysis specific to AI systems and large training datasets.

This sheet details:

  • The methodological approach to manage the security of the development of an AI system,
  • The main security objectives to be pursued when developing an AI system,
  • the risk factors to be taken into account, some of which are AI-specific,
  • recommended measures to make the level of residual risk acceptable.

The methodological approach to be adopted


Security objectives related to AI development


Risk factors for the security of an AI system


Security measures to consider for the development of an AI system


Useful resources