Autonomous Weapon Systems Under International Law
Murat AKKUS
CONTENTS
Background ...............................................................................................1
Key Definitional Issues..............................................................................3
Research Question ....................................................................................6
Rationale ...................................................................................................7
Methodology .............................................................................................7
Chapter 1 - Autonomous Weapons.........................................................9
Introduction................................................................................................9
Definitions, Distinctions or Features of Autonomous Weapon Systems...9
Unmanned Weapon Systems (UWS).......................................................10
Autonomous Weapon Systems (AWS).................................................... 11
Meaningful Human Control (MHC)........................................................12
Critical Functions.....................................................................................13
Humans in the Loop, On the Loop, Out of the Loop...............................15
The Rules on the Conduct of Hostilities..................................................19
Substantive Obligations...........................................................................19
Standard of Precautions...........................................................................21
Bearers of Obligations.............................................................................23
Conclusion ...............................................................................................24
Chapter 2 - The Compliance of Autonomous Systems with ..............29
International Humanitarian Law..............................................................29
Introduction..............................................................................................29
Can AWS Be Deployed in Compliance with the Principle of ................32
Distinction?..............................................................................................32
Persons.....................................................................................................33
Active Combatants...................................................................................33
The General Position ...............................................................................33
The Legal Position on Uniforms and Adversarial Examples...................36
Civilians and Other Protected Persons ....................................................38
Civilians Not Protected from Direct Attack.............................................40
Persons Hors de Combat..........................................................................44
Objects.....................................................................................................46
xii Contents
‘Nature’ and ‘Location’............................................................................47
‘Purpose’ and ‘Use’: The Problem of ‘Dual-Use’ Objects ......................50
The ‘Definite Military Advantage in the Circumstances Ruling at the ..55
Time’........................................................................................................55
Civilian Objects and Specifically Protected Objects...............................56
Cultural Property .....................................................................................57
Objects Indispensable for Civilian Survival............................................59
Infrastructure That May Release Dangerous Forces................................59
Medical Capabilities................................................................................61
Enhancing Detection by Technical Means...............................................62
Enhancing Confidence in Technical Detection........................................63
Will AWS be Able to Sense Targeting ‘Doubt’?......................................63
Summary .................................................................................................67
Guaranteeing AWS Compliance with LOAC: Using Dynamic Diligence 68
Interaction Between Human and Machine ..............................................69
Periodic Assessment ................................................................................71
Dynamic Operational Limits ...................................................................72
Conclusion ...............................................................................................78
Chapter 3 – Can AWS be Deployed in Compliance with the ............81
Principle of Proportionality and with Adequate .....................................81
Precautions?.............................................................................................81
Introduction..............................................................................................81
Autonomous Attacks................................................................................82
How an Attack is Defined........................................................................86
Acts of Violence ......................................................................................89
Scale.........................................................................................................90
Severity....................................................................................................93
Against the Adversary..............................................................................94
Whether in Offence or in Defence ...........................................................95
Application to Delayed-Action Weapons ................................................96
Mines .......................................................................................................96
Cyberattacks ............................................................................................98
Cyber Hacking, Ruses and Human Accountability ...............................101
Contents xiii
Applying Current Law to AWS .............................................................106
The Evolution of Targeted Killing.........................................................106
Identifying the Attack ............................................................................109
Preparatory Obligations Relate to the Combined Actions of Software and
Humans............................................................................................ 116
Extended Operations.............................................................................. 117
Other Precautions in Extended Operations............................................121
Summary................................................................................................123
Potential Solutions.................................................................................123
Legal Use of AWS .................................................................................127
Conclusion .............................................................................................139
Chapter 4 - Prohibition of Indiscriminate Weapons and
Prohibition of Weapons Causing Unnecessary Suffering,
Superfluous Injury.........................................................................141
Introduction............................................................................................141
Is it Possible to Define an AWS in the Strictest Sense of the Term
‘Weapons’ for the Purposes of Article 36 Review? .........................144
What are the ‘Means of Warfare’ or the ‘Methods of Warfare’? ...........144
Defining Weapon, Means of Warfare and Methods of Warfare.............149
What is a Weapon?.................................................................................149
Implications of Weapons Autonomy for Legal Analysis.......................153
Superfluous Injury and Unnecessary Suffering .....................................155
Content of The Principle........................................................................155
Application to Autonomous Weapon Systems ......................................158
Prohibition of Weapons Which are Indiscriminate in Nature................164
Content of The Principle........................................................................164
Application to Autonomous Weapons Systems.....................................168
Protected Persons and Objects...............................................................172
Conclusion .............................................................................................174
Chapter 5 - Autonomous Weapon Systems and Legal Review of New
Weapons..........................................................................................177
Introduction ...........................................................................................177
The Legal Obligation to Conduct Legal Reviews of New Weapons ....177
Customary Law......................................................................................177
Treaty Law ............................................................................................178
Article 36 Scope of Application ...........................................................180
State Weapon Review Procedures – Article 36......................................185
Australia.................................................................................................187
New Zealand..........................................................................................188
The Netherlands.....................................................................................189
Norway ..................................................................................................190
Sweden...................................................................................................191
Switzerland ............................................................................................192
United Kingdom ....................................................................................192
United States..........................................................................................194
Similarities Between State Review Procedures.....................................195
Suggested State Review Practices ........................................................195
Conclusion .............................................................................................197
Chapter 6 - The Responsibility of States for Internationally Wrongful
Acts..................................................................................................199
Introduction............................................................................................199
State Responsibility ..............................................................................199
AWS and the Law of State Responsibility.............................................201
Unclear Attribution: Private Military Contractors.................................204
Contracting Phase ..................................................................................205
Recent Developments in the International Legal Framework ...............206
The Montreux Document and the U.N. Draft Convention on Private
Military and Security Contractors...................................................206
Including AWS within the ASR Framework.......................................... 211
AWS Conduct and State Responsibility ................................................ 211
State Organ Operators and Responsibility.............................................212
Preventing Violations as a Due Diligence Obligation...........................213
The Removal of Human Judgement ......................................................217
Classifying AWS as State Agents ..........................................................218
xiv Contents
Can a Stricter Liability Regime Overcome Potential Accountability
Gaps? ...............................................................................................222
Accountability in Domestic Courts .......................................................225
Conclusion .............................................................................................230
Chapter 7 - Individual Responsibility and Autonomous Weapon
Systems ...........................................................................................233
Introduction............................................................................................233
AWS Programmer Liability...................................................................233
Armed Conflict as a Threshold Requirement ........................................235
Principal and Accessorial Liability........................................................237
Ex Ante Aiding and Abetting: International Legal Theory....................239
The Application of the Effective Control Test within the Robotic
Context.............................................................................................241
The Requirement for Mens Rea ............................................................251
Summary of Programmer Responsibility ..............................................253
The Doctrine of Command Responsibility: A Poor Fit to Govern
Artificial Intelligence Systems........................................................254
AWS and the Irrelevance of Command Responsibility.........................262
Manufacturer Responsibilities for AWS................................................268
Manufacturers and International Law....................................................268
Criminal Responsibility of Manufacturer..............................................270
Civil Responsibility of Manufacturer....................................................271
Manufacturer Responsibility for AWS Design ......................................272
The Paradox of a Responsible Arms Maker ..........................................273
Manufacturer Responsibility for AWS Sale and Supply .......................273
Manufacturer Responsibility for AWS Use ...........................................274
Product Liability ....................................................................................278
No-Fault Liability ..................................................................................280
Summary................................................................................................281
Conclusion ............................................................................................282
Conclusions and Recommendations......................................................285
Bibliography .........................................................................................293
