The Draft EU AI Act seeks to deliver a regulatory framework mandating technical documentation and record-keeping requirements pertaining to high-risk AI systems. Draft Article 11 of the draft act spells out the specifics of this framework.

In an effort to comply with the likely forthcoming EU AI Act, companies should take care to ensure thorough documentation of their high-risk AI systems, including elements about the data used to train an AI system, key design choices, and even the hardware an AI system utilizes. In essence, all of the key ingredients that go into the creation and operation of the AI system should be documented. 

Article 11: Technical Documentation

Article 11 spells out that technical documentation of high-risk AI systems “shall contain, at a minimum, the elements set out in Annex IV or, in the case of SMEs and start-ups, any equivalent documentation meeting the same objectives, subject to approval of the competent national authority.” This technical documentation requirement aims to “demonstrate that the high-risk AI system complies with the requirements” of the draft AI EU Act.

Based on draft Article 11, liability could attach to an entity not only for failing to comply with other articles within the draft EU AI Act but also failing to document compliance constitutes a violation. In addition, the draft language provides that the Commission “is empowered to adopt delegated acts … where necessary to ensure that, in the light of technical progress, the technical documentation provides all the necessary information to assess the compliance of the system.” This language allows additional elements of technical documentation to be added as AI systems evolve.

Summaries of selected elements of required technical documentation are provided below. These are not all the elements but instead are a selected subset of the draft language. For the complete language and all of the elements, consult the latest amendments to the draft language here: https://www.europarl.europa.eu/doceo/document/TA-9-2023-0236_EN.html

• Section 1: “A general description of the AI system”
  • The intended purpose, name of the provider, and the version of the system in relation to its previous and potentially more recent versions.
  • The nature of data, especially personal data, that's likely to be processed by the system and the categories of natural persons and groups that will be affected.
  • How the AI system interacts with other hardware or software, including other AI systems, that are not a part of it.
  • Information about the version of relevant software or firmware, especially any requirement related to updates.
  • The type of hardware the AI system is intended to run on.
  • An easily intelligible description of the system’s main optimization goal or goals.
  • An easily intelligible description of the system’s expected output and the quality of that output.
• Section 2: “A detailed description of the elements of the AI system and of the process for its development”
  • The methods and steps of development, especially when third-party pre-trained systems or tools have been used or modified.
  • Key design and classification choices, the rationale and assumptions made regarding its intended users, and any possible trade-offs made regarding technical solutions.
  • Descriptions of training methodologies, data sets used, data provenance, and data cleaning methodologies.
  • Testing and validation procedures, especially metrics used to measure accuracy, robustness, and other requirements. This includes test logs and signed reports.
• Section 3: “Detailed information about the monitoring, functioning and control of the AI system”
  • Information on the system's performance capabilities and limitations, especially accuracy for specific groups of people, and any foreseeable risks to health, safety, and rights.
  • A description of the appropriateness of the performance metrics specific to the AI system.
  • Information about the energy consumption of the AI system during both the development phase and its expected consumption during use.
• Section 5
  • A description of any relevant change made by providers to the system through its lifecycle.