Autonomous weapons: The ICRC recommends adopting new rules
Statement of the International Committee of the Red Cross delivered at the Convention on Certain Conventional Weapons (CCW) before the Group of Governmental Experts on Lethal Autonomous Weapons Systems - 3–13 August 2021, Geneva
The International Committee of the Red Cross (ICRC) welcomes the resumption of work by the Group of Governmental Experts (GGE) at this critical moment in multilateral deliberations on autonomous weapon systems, and with less than five months to go until the Review Conference of the Convention.
The ICRC appreciates the efforts of the chair of the GGE, Ambassador Marc Pecsteen de Buytswerve of Belgium, to solicit proposals for consensus recommendations of the Review Conference on the "normative and operational framework". It is the ICRC's view that an urgent and effective international response is needed to address the serious risks posed by autonomous weapon systems, as highlighted by many states and civil society organizations over the past decade.
These risks stem from the process by which autonomous weapon systems function. It is the ICRC's understanding that these weapons, after initial activation, select and apply force to targets without human intervention, in the sense that they are triggered by their environment based on a "target profile", which serves as a generalized approximation of a type of target.
The user of an autonomous weapon system does not choose the specific target, nor the precise time or place that force is applied. This process risks the loss of human control over the use of force and it is the source of the humanitarian, legal, and ethical concerns.
These concerns are significant when one considers that autonomy in the critical functions of selecting and applying force could be integrated into any weapon system.
The central challenge with autonomous weapon systems resides in the difficulty of anticipating and limiting their effects. From a humanitarian perspective, they risk harming those affected by armed conflict, both civilians and combatants hors de combat, and they increase the risk of conflict escalation. From a legal perspective, they challenge the ability of persons who must apply the rules of international humanitarian law (IHL) during the planning, decision and execution of attacks to comply with their obligations.
From an ethical perspective, this process of functioning risks effectively substituting human decisions about life and death with sensor, software and machine processes. This raises ethical concerns that are especially acute when autonomous weapon systems are used to target persons directly.
Today, autonomous weapon systems are highly constrained in their use; they are mostly used against certain types of military objects, for limited periods of time, in restricted areas where civilians are not present and with close human supervision.
However, current trends in the expanded development and use of autonomous weapon systems exacerbate core concerns dramatically. In particular, there is military interest in their use against a wider range of targets, over larger areas and for longer periods of time, in urban areas where civilians would be most at risk and with reduced human supervision and capacity for intervention and deactivation.
Worryingly, the use of artificial intelligence and machine learning software to control the critical functions of selecting and applying force is being increasingly explored, which would exacerbate the already difficult task that users have in anticipating and limiting the effects of an autonomous weapon system.
Against this background, the High Contracting Parties to the CCW have a responsibility and an opportunity to make progress in clarifying, considering and developing the normative and operational framework for autonomous weapon systems.
Given this opportunity, the ICRC offered recommendations to all states on 12 May 2021, which were also submitted to the chair of the GGE on 11 June 2021.
The ICRC recommends that states adopt new, legally binding rules to regulate autonomous weapon systems to ensure that sufficient human control and judgement is retained in the use of force. It is the ICRC's view that this will require prohibiting certain types of autonomous weapon systems and strictly regulating all others.
First, unpredictable autonomous weapon systems should be expressly ruled out, notably because of their indiscriminate effects. This would best be achieved with a prohibition on autonomous weapon systems that are designed or used in a manner such that their effects cannot be sufficiently understood, predicted and explained.
Secondly, the use of autonomous weapon systems to target human beings should be ruled out. This would best be achieved through a prohibition on autonomous weapon systems that are designed or used to apply force against persons directly as opposed to against objects.
Thirdly, the design and use of non-prohibited autonomous weapon systems should be regulated, including through a combination of limits on the types of target, such as constraining them to objects that are military objectives by nature; limits on the duration, geographical scope and scale of use, including to enable human judgement and control in relation to a specific attack; limits on situations of use, such as constraining them to situations where civilians or civilian objects are not present; and imposing a requirement for human–machine interaction, notably to ensure effective human supervision and timely intervention and deactivation.
It is the ICRC's understanding that these proposed prohibitions and restrictions are in line with current military practice in the use of autonomous weapon systems.
While these recommendations offer clear, principled and pragmatic guidance on where to draw the boundaries of what is acceptable in light of humanitarian, legal and ethical concerns, they are not ready-made treaty text and the finer details of these limits will require further elaboration by states.
It is encouraging, therefore, that there is an increasing convergence of views among states that certain autonomous weapon systems should be prohibited or otherwise excluded from development and use, and that others should be regulated or otherwise limited in their development and use.
These proposals reflect widely held views: a recognition of the need to ensure human control and judgement in the use of force; an acknowledgement that ensuring such control and judgement requires effective limits on the design and use of autonomous weapon systems; and an increasing confidence that such limits can be articulated at international level.
It is also encouraging that many states have shown a readiness to clarify how IHL already constrains autonomous weapon systems and some have proposed the further sharing of current military practice.
The ICRC is convinced that international limits on autonomous weapon systems must take the form of new, legally binding rules. New rules are required because of the seriousness of the risks, the necessity to clarify how existing IHL rules apply and the need to develop and strengthen the legal framework in line with ethical and rule of law issues and humanitarian considerations.
International law must continue to evolve in order to uphold and strengthen protections in the face of evolving military technology and practice. The High Contracting Parties to the CCW, which is a framework Convention anchored in IHL, acknowledge the "need to continue the codification and progressive development of the rules of international law applicable in armed conflict", as they have affirmed in the Convention's preamble.
Considering current military developments in how autonomous weapon systems are being used and deployed, the ICRC urges the High Contracting Parties to the CCW to take action now towards the adoption of new rules.
Building on the extensive and in-depth work of the CCW and this GGE over the past eight years, there is an opportunity to shape an international response that will effectively strengthen protections for those affected by armed conflict, uphold the legal obligations and moral responsibilities of persons conducting conflict and safeguard our shared humanity. It is an opportunity that the High Contracting Parties must seize and focus on in their work during this meeting.