AI Impact Assessment: When and How to Conduct One

What is an AI impact assessment?

An AI impact assessment is a structured process for analysing the consequences an artificial intelligence system may produce in its real context of use.

The question should not be limited to whether the system works technically.

It is also useful to consider:

• who uses it
• which decisions it supports
• which people or groups it may influence
• what data it uses
• what errors may occur
• the consequences of an incorrect outcome
• the level of human oversight
• which measures can reduce unwanted impacts.

The aim is not to prove that the system carries no risk.

It is to understand its effects well enough to decide whether it may be used, under what conditions and with which controls. The assessment should form part of the organisation’s AI governance.

Impact and risk are not the same

Risk and impact are connected, but they are not identical.

Risk normally concerns the possibility of an unwanted event and the consequences it could produce. AI risk management helps an organisation identify, assess and treat that possibility.

Impact focuses on the actual or potential consequences for people, groups, processes, rights, the organisation or society.

For example, a recruitment system may present a risk of bias. The impact may be that certain applicants consistently have fewer opportunities to reach interview.

A useful assessment should therefore go beyond saying, “There is a risk of bias.” It should ask:

• who could be affected?
• in what way?
• how seriously?
• for how long?
• can the effect be reversed?
• what happens if nobody detects the problem?

That is where impact analysis truly begins.

When an impact assessment should be conducted

Not every use of AI requires the same depth of analysis.

A formal assessment is particularly valuable where the system:

• contributes to decisions about people
• may influence employment, education, credit, services or benefits
• processes sensitive information
• operates with a significant degree of autonomy
• may produce consequences that are difficult to reverse
• affects large groups
• may particularly disadvantage vulnerable people
• relies on models or providers that are difficult to explain
• substantially changes a process previously performed by a person
• presents material risks already identified.

It may also be useful before purchasing a solution. Assessing impact after the contract has been signed and the system deployed greatly reduces the organisation’s ability to change key decisions.

When it may be required under the AI Act

An impact assessment used as a general governance practice must be distinguished from an assessment arising from a specific legal duty.

Article 27 of the AI Act provides for a fundamental rights impact assessment for certain deployers of high-risk AI systems.

This does not mean that every business using AI must automatically conduct one.

Whether it applies depends, among other matters, on the type and classification of the system and on the deployer concerned.

Where applicable, it must be completed before the high-risk system is used for the first time.

The matters to be considered include:

• the processes in which the system will be used
• the intended period and frequency of use
• the categories of persons and groups likely to be affected
• the specific risks of harm to those persons
• human oversight measures
• measures to be taken if those risks materialise.

Before stating that an organisation must conduct a fundamental rights impact assessment, it must first establish whether it falls within the relevant scope and review the applicable AI Act obligations for businesses.

Sound governance starts with that classification.

How it relates to ISO/IEC 42001 and ISO/IEC 42005

Impact assessment also fits within artificial intelligence management systems.

ISO/IEC 42001 provides an organisational framework for managing AI systematically. Within that system, organisations need to understand their systems, risks, interested parties, effects and controls.

ISO/IEC 42005 develops the concept of AI system impact assessment specifically.

The relationship is straightforward: ISO/IEC 42001 helps establish the management system, while ISO/IEC 42005 helps structure the assessment of individual systems.

They are not interchangeable documents. They are complementary parts of a wider governance approach.

Define the system and its context of use

An impact assessment starts by establishing exactly what is being assessed.

Writing “artificial intelligence system for Human Resources” is not enough.

A more useful description would be: “A system used to rank applicants before the recruitment team’s initial review.”

That distinction matters because impact depends on real use.

The assessment should describe at least:

• purpose
• the process in which the system operates
• users
• provider or developer
• principal inputs
• outputs produced
• decisions supported
• degree of automation
• people who may be affected
• duration or frequency of use.

The AI inventory connects this information with owners, risks and controls.

Without context, impact analysis becomes too abstract.

Identify affected persons and groups

One of the most important questions is: Who could this system affect?

The answer is not always the same as the identity of the user.

A Human Resources team may use the system, but applicants are the affected persons. A financial institution may use a system internally, but the impact may fall on customers. A public body may use an administrative tool while citizens experience the consequences.

It is useful to identify:

• direct users
• people who are the subject of decisions
• people whose data is used
• potentially vulnerable groups
• third parties affected indirectly.

The more significant the decision, the more important this distinction becomes.

Identify positive and negative impacts

An assessment should not focus only on harm. It may also identify benefits such as:

• shorter processing times
• greater consistency
• error detection
• increased capacity
• improved access
• fewer repetitive tasks.

Those benefits do not automatically remove potential negative impacts, which may include:

• discrimination
• exclusion
• loss of privacy
• incorrect decisions
• lack of transparency
• difficulty challenging outcomes
• loss of autonomy
• excessive reliance on the system
• reputational harm
• financial or professional loss.

The assessment is useful precisely because it considers both sides.

Assess severity, likelihood and scale

Once impacts have been identified, they need to be prioritised.

Relevant factors may include:

Severity

What consequences would the impact have?

Likelihood

How likely is it to occur?

Scale

How many people could be affected?

Duration

Would the effect be temporary or persistent?

Reversibility

Could it be corrected easily?

Vulnerability

Are particular people or groups especially exposed?

A matrix can help, but a numerical score does not replace analysis. Two impacts with the same mathematical value may have entirely different implications.

Define mitigation measures and human oversight

An assessment does not end when impacts have been identified. The organisation must decide what to do about them.

Measures may include:

• human review
• prior authorisation
• data quality controls
• pre-deployment testing
• limitations on use
• monitoring
• periodic review
• complaint mechanisms
• additional validation for particular decisions
• user training
• contractual controls for providers
• the ability to stop the system.

These measures should align with the organisation’s AI use policy.

Human oversight must be genuine. It does not simply mean placing a person at the end of the process.

That person needs sufficient information, time, authority, the ability to challenge the output and a genuine opportunity to change or reject a decision.

How to conduct an AI impact assessment step by step

A practical method may follow this sequence:

1. Identify the system

Define the system and the version being assessed.

2. Describe the use

Explain its purpose and the process in which it operates.

3. Identify affected persons

Determine who may experience direct or indirect consequences.

4. Analyse impacts

Identify foreseeable positive and negative effects.

5. Assess impacts

Consider severity, likelihood, scale, duration and reversibility.

6. Review existing controls

Establish which measures are already in place.

7. Define additional measures

Decide how unacceptable impacts will be reduced.

8. Determine residual impact

Assess what remains after controls have been applied.

9. Make a decision

Approve, limit, modify, postpone or reject the use.

10. Assign responsibilities

Identify who will implement each measure.

11. Document

Retain sufficient evidence.

12. Review

Update the assessment when the system or context changes.

The sequence need not become excessively bureaucratic. It should be robust enough to support an informed decision.

What evidence should be retained

It should be possible to reconstruct the assessment.

A final conclusion stating “acceptable risk” is not enough.

Evidence should cover:

• the system and version assessed
• purpose and context
• owners
• affected persons
• impacts identified
• assessment
• controls
• decisions
• outstanding measures
• residual risk or impact
• approval
• date
• next review.

Where relevant, supplier documentation, testing, oversight records, minutes, technical reports, complaints and exception decisions may also be retained.

Evidence demonstrates that a reasoned process took place.

When the assessment should be reviewed

An impact assessment should not be treated as permanent.

It may need to be updated when the model, provider, purpose, data, number of affected people or level of automation changes; when an incident occurs; when new risks are identified; when regulation or the organisational process changes; or when new complaints or evidence emerge.

An assessment that no longer describes the real system no longer serves its purpose.

Common mistakes

Assessing too late

Conducting the analysis after the system has been purchased and deployed reduces the scope for meaningful change.

Turning it into a form

Completing fields without analysing the real context creates a false sense of control.

Looking only at technical risks

Model accuracy alone does not explain the impact on people.

Failing to identify affected people

Without knowing who may be affected, consequences cannot be assessed properly.

Confusing impact with compliance

Complying with a standard does not automatically mean every impact is controlled.

Failing to assign owners

A measure without an owner may remain no more than a recommendation.

Failing to review

Systems change, and the assessment should change with them.

Assess before deployment to make better decisions

An impact assessment should not be used merely to justify a decision that has already been made.

Its value appears while there is still scope to change the system, introduce controls, limit particular uses or decide not to deploy a solution.

The final question is not only: Can it work?

It should also be: What happens when it works in our real context and affects real people?

That is the difference between assessing technology and governing artificial intelligence.

References

• Regulation (EU) 2024/1689 — AI Act, Article 27.
• ISO/IEC 42001:2023 — Artificial intelligence management system.
• ISO/IEC 42005:2025 — AI system impact assessment.
• ISO/IEC 23894:2023 — Guidance on risk management.