Frequently asked questions: AI decision-support systems in armed conflict

These systems analyse large datasets to support military decisions, but they carry risks based on the way in which AI functions, and the lack of quality and context-specific data in armed conflicts.

Their use also raises questions of automation bias – the tendency of humans to over-trust machine-generated recommendations or outputs, particularly in high-pressure situations or under time constraints. This may lead users to rubber-stamp AI outputs rather than apply independent judgment.

Additionally, in adversarial environments such as an armed conflict, opponents may deliberately manipulate data or exploit system vulnerabilities. 

These systems can amplify the speed and scale of errors and risk unintended escalation, especially if users fail to exercise independent judgment.

Promoters of AI-based systems have claimed that they can enhance compliance with IHL and minimize risks for civilians. 

In certain circumstances, the careful and responsible use of AI DSS may facilitate quicker and more comprehensive information analysis, which can support decisions in a way that enhances IHL compliance and minimizes risks for civilians. 

For instance, the use of AI DSS can assist in collecting and synthesizing information from open-source repositories online about the presence of civilians and civilian objects or support weaponeering by recommending means and methods of attack that can best avoid, or at least minimize, incidental civilian harm. The effectiveness of any such tool will still be subject to inherent limitations of the technology and will depend upon access to good quality data. 

Unfortunately, many of the conflicts today in which the most technologically sophisticated weapons and systems are employed, we have not seen better outcomes for civilians but rather the most widespread and indiscriminate devastation. The qualities advertised in new technologies have led not to precision, distinction and precaution, but to acceleration, amplification and escalation of destruction, with appalling results for civilians.

Ultimately, the impact on civilians of AI DSS will depend on the way they are designed and used.

Importantly, the use of AI-decision-support systems can never ameliorate targeting methodologies and other policies that do not otherwise comply with IHL; applying AI-decision-support systems within such frameworks will serve only to replicate and likely exacerbate unlawful or otherwise harmful effects faster and on a larger scale.

The ICRC has made 16 preliminary recommendations to address the risks associated with AI decision-support systems in armed conflict. These recommendations emphasize the importance of preserving human judgment and control. Key measures include:

• ensuring rigorous testing, evaluation, verification and validation, and legal reviews;
• using reliable and high quality data;
• mitigating risks of bias and discriminatory outcomes;
• ensuring meaningful human engagement and the ability to challenge AI outputs;
• strengthening user training, including on automation bias;
• conducting after-action reviews.

The ICRC also emphasizes that certain uses of AI must be prohibited, such as in nuclear command and control systems or in autonomous weapon systems that target humans directly or produce unpredictable effects.

The recommendations are guided by a human-centred approach aimed at ensuring that AI systems support human decision-making, rather than impair or replace it.