States must address concerns raised by autonomous weapons

Statement to the Meeting of the High Contracting Parties to the Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons which may be Deemed to be Excessively Injurious or to Have Indiscriminate Effects (CCW), 13 to 15 November 2019.

The International Committee of the Red Cross (ICRC) welcomes the progress made in 2019 by the High Contracting Parties in the Group of Governmental Experts (GGE) on "Lethal Autonomous Weapon Systems" to address the legal and ethical issues raised by theses weapon. We thank the Chair of the GGE, Ljupco Jivan Gjorgjinski, for successfully guiding its work.

The ICRC also welcomes the GGE's Report (CCW/GGE.1/2019/3), notably its Conclusions in Part III, including the additional guiding principle on human-machine interaction to complement the ten guiding principles adopted in 2018, and its Recommendations in Part IV. The GGE report reaffirms the applicability of international humanitarian law (IHL) and the importance of a human-centered approach to the issues raised by autonomous weapon systems.

The ICRC's primary concern with weapon systems that can select and attack targets without human intervention is the loss of human control over the use of force. The unique characteristics of autonomous weapon systems heighten risks for civilians and raise challenges for IHL compliance and concerns about ethical acceptability.

The ICRC again calls on CCW High Contracting Parties to work to urgently address these challenges. In particular, as the ICRC recommended in its Working Paper submitted to the 2018 Meeting of High Contracting Parties (CCW/GGE.1/2018/CRP.1), the GGE must work to reach common understandings on the elements of human control over the critical functions of weapon systems needed for legal compliance and ethical acceptability. States must determine where lines should be drawn, and what form these limits should take – whether through policies, standards or new legally binding rules.

Existing IHL rules already constrain autonomy in weapon systems. They establish responsibilities for combatants, who must retain the ability to make the context-specific judgments in carrying out attacks in armed conflict. But many questions remain about how these obligations should be interpreted and applied in light of the unique characteristics of autonomous weapon systems. IHL rules do not provide all the answers. Further, the limits dictated by ethical considerations may go beyond those found in existing IHL rules, notably with respect to anti-personnel systems.

The ICRC's focus is, as always, on protecting life and dignity of people affected by armed conflict and promoting respect for IHL. Against this background, it is clear to us that limits are needed on both the types of autonomous weapons and the types of situations in which they are used.

We call on States to address the following critical questions, among others:

• Is it legally and ethically acceptable to develop and use autonomous weapon systems that are designed to use force against persons?
• Is it legally and ethically acceptable to use autonomous weapon systems designed to use force against objects in areas where civilians and civilian objects are at risk?
• What limits should be set on the use of autonomous weapon systems to address their unpredictability? Should there be limits on their tasks, the duration (time-frame) and area (geographical scope) of their operation? Should there be human supervision with the ability to intervene and deactivate in real time?

The ICRC explains in more detail its views on the legal and ethical issues raised by autonomous weapon systems, and also by the use of artificial intelligence (AI) and machine learning in armed conflict, in a recently published report submitted to the 33rd International Conference of the Red Cross and Red Crescent, which will take place in Geneva in December.

After six years of discussions at the CCW, and as States begin to look ahead to the Sixth Review Conference of the CCW in 2021, the ICRC reiterates its call on CCW High Contracting Parties to act preventively before military technology developments overtake your deliberations.