CAMPAIGNER'S KIT - Campaign to Stop Killer Robots
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CAMPAIGNER’S KIT
CONTENTS
Campaigner’s Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Let’s Stop Killer Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
WHAT ARE THE ISSUES TO CONSIDER? ____________________________________________________ 5
Legal Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Global Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Human Control of Weapons Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Gender and Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Intersectionality and Racism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
HOW DO WE MAKE CHANGE? _____________________________________________________________ 33
Advocacy and Lobbying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Working with Parliamentarians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
How to Do a Scientist Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Engaging Military Personnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Building a National Campaign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8 Basics to Get Your Media Work Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Campaigning Online with Social Media. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
AUTHOR BIOS ____________________________________________________________________________ 84
W W W. S T O P K I L L E R R O B O T S . O R G
This kit was prepared by Erin Hunt of Mines Action Canada in March 2019 for the Campaign to Stop Killer Robots,
and updated in February 2020.
Design and layout by Jaś Lewicki
LET’S STOP
KILLER ROBOTS
The goal of the Campaign to Stop Killer Robots These states and China are heavily investing in
Mary Wareham has not changed since its inception. We are armed drones and other autonomous weapons
Campaign to Stop Killer Robots working to preemptively ban the development, systems with decreasing levels of human
production and use of fully autonomous weapons, control in their critical functions, prompting
also known as lethal autonomous weapons fears of widespread proliferation and arms
systems, autonomous weapons systems or killer races. A new treaty to stop killer robots
robots. The positive way of framing this goal is is urgently needed, before defense sector
that we seek to retain meaningful human control investments in artificial intelligence and related
More than a decade ago, roboticists and artificial intelligence experts over weapons systems and the use of force. technologies make these weapons a reality.
became the first to raise the alarm at the prospect of weapons systems A ban treaty is achievable, but time is running out. The CCW may include all the “major players”
that would select and attack targets without human intervention. States first discussed the challenges raised by killer but its consensus-based mode of decision-
Then, in October 2012, Human Rights Watch, the International robots at the Human Rights Council in Geneva in making means that a single state can successfully
May 2013. The matter was then taken up by the oppose proposals supported by the rest.
Committee for Robot Arms Control (ICRAC) and five other non- Convention on Conventional Weapons (CCW), also
governmental organizations co-founded the Campaign to Stop at the United Nations (UN) in Geneva, for further We’ve been there before. Past CCW failures to
consideration. Representatives from more than stem human suffering caused by antipersonnel
Killer Robots to provide a coordinated civil society response. 90 states have participated landmines and cluster
in eight CCW meetings on munitions resulted in external
lethal autonomous weapons “A ban treaty is achievable, diplomatic processes that
systems since 2014. delivered life-saving treaties
but time is running out.“ to ban the weapons. The lack
The CCW meetings of agreement among nuclear
have explored some of the fundamental legal, weapons states to disarm led other countries
operational, moral, technical, proliferation and other to create the 2017 Treaty on the Prohibition of
concerns raised by allowing machines to select and Nuclear Weapons via the UN General Assembly.
attack targets without further human intervention.
There is now widespread agreement about the Those humanitarian disarmament treaties
need to retain some form of human control over were all the result of genuine partnerships
future weapons systems and the use of force. between like-minded countries, UN agencies,
the International Committee of the Red
Yet states have made little progress towards Cross (ICRC), and dedicated coalitions of
achieving an outcome. Most of the participating nongovernmental organizations. These treaties
states have proposed moving to negotiate a are successfully reducing and helping to
new international treaty to prohibit or regulate prevent human suffering, even without the
lethal autonomous weapons systems yet signatures of all the major military powers.
these proposals have been explicitly rejected
by military powers such as Israel, Russia, Embarking on such a process can succeed, but
South Korea, UK, and United States. requires bold political leadership. Working out
the diplomatic pathway requires starting from
3
the bottom up, which is why the Campaign to
Stop Killer Robots has intensified its support
We must demonstrate that the public is onside
with the call to ban killer robots to demonstrate the
WHAT
ARE THE
over the past year to NGOs conducting political saliency of this cause. Creating pressure on
outreach in capitals around the world. We’re political leaders requires leveraging media interest,
looking for political leaders willing to help launch promoting our cause on social media, and utilizing
negotiations on a new treaty to ban killer robots. tools that can help spread the word about this
serious challenge and the need for a ban treaty.
ISSUES TO
To succeed, we must find parliamentary champions
willing to press the government to act and We know from past experience that governments
propose national laws and other measures to never take action without pressure from civil
ban fully autonomous weapons. Currently 30 society. Our arguments against killer robots
CONSIDER?
states have called for a ban on fully autonomous are clear and the case for a new ban treaty is
weapons, but we need more states on board. strong. We must build upon our efforts to date
and secure the necessary political leadership to
achieve our goal of a treaty banning killer robots.
“Our arguments against killer
robots are clear and the case
for a new ban treaty is strong.“
4
LEGAL
ARGUMENTS
Bonnie Docherty and Matthew Griechen INTERNATIONAL Fully autonomous weapons could not replicate
HUMANITARIAN LAW the human judgment necessary to assess the
Harvard Law School International Human Rights Clinic (IHRC) proportionality of a specific attack. Because
International humanitarian law (IHL), also known programmers cannot account in advance for the
as the laws of war, would govern the use of fully infinite number of scenarios that might arise on
autonomous weapons on the battlefield. Because the the battlefield, fully autonomous weapons would
weapons would operate without meaningful human encounter unforeseen and changing circumstances.
control, they would face particular difficulties in Unlike humans, however, these machines could
The legal case against fully autonomous weapons, or “killer robots,” complying with two fundamental rules of IHL. not apply human reason and experience when
balancing the relevant factors of this subjective test.
reinforces the moral, technological, and security arguments for banning First, customary international law and Article 48 of
this emerging technology.1 Fully autonomous weapons threaten to Additional Protocol I to the Geneva Conventions THE MARTENS CLAUSE
violate international humanitarian law and international human rights obliges warring parties to distinguish between
civilians and soldiers and between civilian objects States developing or using new technology must
law and would create a gap in accountability for unlawful acts. (such as homes or schools) and military objectives. consider the so-called Martens Clause, a provision
Weapons that cannot make such distinctions of international humanitarian law that links law
are considered “indiscriminate” and unlawful. and ethics. The Martens Clause, articulated in
many places, including Article 1(2) of Additional
Killer robots would encounter significant obstacles Protocol I, is a gap-filling provision. It declares
to complying with the rule of distinction. that in the absence of specific treaty law on a
Differentiating between civilians and soldiers, topic, people are still protected by “custom,” “the
particularly in an era in which combatants often principles of humanity,” and “the dictates of public
blend in with the local population, depends on conscience.” The clause creates a moral standard
more than recognizing a uniform. It also depends against which to judge fully autonomous weapons.
on understanding a person’s intentions through
such clues as tone of voice, facial expressions, or Fully autonomous weapons raise serious concerns
body language. Humans are better equipped to under principles of humanity. Humans are
understand such nuances than machines are. motivated to treat each other humanely because
they can feel compassion and empathy for the
Second, customary international law and Article experiences of other people. Fully autonomous
51(5)(b) of Additional Protocol I requires warring weapons, by contrast, would lack the emotional
parties to weigh the proportionality of an attack. capacity that underlies humane treatment.
This rule prohibits attacks in which the expected The principles of humanity also require respect
harm to civilians and civilian objects is excessive for the dignity of human life. As inanimate
in relation to the anticipated military advantage. machines, fully autonomous weapons cannot truly
Proportionality is not a mathematical equation. understand the value of a life and the significance
It depends on context, and the test is whether of its loss. They would determine whom to kill
a “reasonable military commander” would based on algorithms and would not consider
have found it lawful to launch the attack. the inherent worth of an individual victim.
7
The dictates of public conscience, which refer to notably empathy and judgment, necessary to make humans, they cannot experience suffering. Programmers and manufacturers would likely elude
shared moral guidelines, similarly argue against such determinations in unforeseen situations. liability under a civil suit. In some countries, such
fully autonomous weapons. In a December 2018 In most cases, humans also would escape criminal as the United States, weapons manufacturers are
survey of public opinion in 26 countries, more than Second, victims of human rights abuses have a right liability for the robot’s actions. The relationship immune from suit as long as they follow government
60 percent of people responded that they opposed to a remedy. As discussed more below, however, it between an operator and a fully autonomous specifications and do not deliberately mislead the
killer robots. In addition,
2
is not clear who could be weapon can be likened to that of a commander and military. In addition, proving a product is defective
leaders in disarmament held accountable if fully a subordinate because the requires overcoming
and human rights, peace “The principles of humanity also autonomous weapons robot and the subordinate significant evidentiary
and religion, science and require respect for the dignity violated international both act autonomously. “Holding a person liable for hurdles. Finally, civil suits
technology, and industry
of human life. As inanimate
human rights law by, Commanders are legally the unlawful acts of a fully are time-consuming and
have all condemned this for example, arbitrarily responsible for the expensive, especially for
technology, particularly machines, fully autonomous killing a civilian. actions of a subordinate autonomous weapon, however, victims living far from the
on moral grounds.
weapons cannot truly only when they knew would be challenging and in country that deployed
Finally, states have
understand the value of a life
Third, the principle or should have known most cases, nearly impossible.” the weapon at issue.
frequently appealed of human dignity of the subordinate’s
to conscience when and the significance of its loss.” underpins human criminal act and failed Thus, fully autonomous
calling for a ban on rights law. All human to prevent or punish it. While a commander weapons would not only face potentially
fully autonomous weapons or a requirement life has worth and deserves respect. As discussed who deployed a fully autonomous weapon with insurmountable barriers to complying with
of human control over the use of force. above, delegating life-and-death decisions to the clear intent to commit a crime might be international law, but would also allow commanders,
machines that cannot fully appreciate the value found guilty, it would be legally difficult—and operators, programmers and manufacturers to
INTERNATIONAL HUMAN of human life would undermine human dignity. unfair—to hold him or her accountable for the escape responsibility for violations that did occur.
RIGHTS LAW unforeseeable actions of an autonomous machine.
ACCOUNTABILITY
Given that fully autonomous weapons would likely
be used in law enforcement situations beyond the Both international humanitarian law and
battlefield, they should also be assessed under international human rights law require individual
international human rights law, which applies accountability for unlawful acts. Such personal
during times of peace as well as armed conflict. accountability helps deter future violations while
Fully autonomous weapons have the potential providing retribution for victims of past harm.
to violate three foundational human rights. Holding a person liable for the unlawful acts of
a fully autonomous weapon, however, would be ENDNOTES
First, under Article 6 of the International challenging and in most cases, nearly impossible.
1 For an overview of the problems of fully autonomous weapons and detailed responses to the views of critics, see Human
Covenant on Civil and Political Rights, all people
Rights Watch and Harvard Law School’s International Human Rights Clinic, Making the Case: The Dangers of Killer Robots
have the fundamental right to life, meaning A robot itself could not be held responsible
and the Need for a Preemptive Ban (December 2016), https://www.hrw.org/sites/default/files/report_pdf/arms1216_web.
they cannot be “arbitrarily deprived” of their under the law. Crimes involve both an act (such pdf. These organizations have also co-published stand-alone reports examining such issues as the Martens Clause, the human
lives. Killing is only lawful when it is necessary as causing death) and a mental state (such as rights implications of killer robots and the accountability gap. For a comprehensive list of these publications, see Human
to protect human life, constitutes a last resort, intent). While a fully autonomous weapon could Rights Watch, “Reviewing the Record” (2018), http://hrp.law.harvard.edu/wp-content/uploads/2018/08/Killer_Robots_
and is applied in a manner proportionate to the commit the act, as a machine, it would lack the Handout.pdf.
threat. The test is context specific, and killer mental state. Furthermore, fully autonomous 2 Campaign to Stop Killer Robots, “Global Poll Results Shows 61% Oppose Killer Robots,” January 2019, https://www.stop-
robots would not have the human qualities, weapons could not be punished because, unlike killerrobots.org/2019/01/global-poll-61-oppose-killer-robots/.
8 9
GLOBAL
SECURITY
There are many reasons to be concerned about 3. CONTINUOUS GLOBAL
Noel Sharkey the safety of civilians across the globe should BATTLEFIELD
autonomous weapons ever be developed. Let’s
International Committee for Robot Arms Control look at 10 of the strongest of these concerns Autonomous weapons systems could run on
that you can use when advocating for a pre- much less energy than existing military vehicles
emptive ban on autonomous weapons. and could easily be recharged with solar panels.
Weapons could be left behind - like landmines
- to patrol post-conflict zones and thus create
1. PROLIFERATION a continuous global battlefield. The result could
Autonomous weapons systems also known as fully autonomous have devastating psycho-social consequences.
weapons pose great dangers for international stability and global Without an international muzzle on the
development, testing, and production of
security. We are already seeing the beginning of an international autonomous weapons systems, we are likely to see 4. WARRING AUTONOMOUS
arms race among the superpowers. One of the most worrying mass proliferation of these weapons and counter WEAPONS SYSTEMS
developments is the development of swarm technologies. The weapons and on and on. Not all nations will have the WOULD INTERACT
ability to carry out weapons reviews of autonomous
idea is that a small number of military personnel could initiate a weapons systems required under international As more countries employ swarms of autonomous
large scale attack of swarms of tanks, ships or fighter planes. law. So it is likely that the standards required by weapons systems and autonomous counter
international humanitarian law (IHL) could slip. defences, these weapons as well as command
and control systems would inevitably interact.
When any mobile device controlled by software
2. LOWERED THRESHOLD programs interacts with a competing hostile
FOR ARMED CONFLICTS device controlled by unknown software, the result
of the interaction is scientifically impossible
Autonomous weapons systems could lead to more to predict. Thus, it would be impossible to
action short of warfare by minimising human calculate the impact on civilian populations.
military forces in conflict zones. This could enable
states to initiate the use of violent force without
the consultation procedures required to deploy 5. ACCELERATING THE
troops on the ground. Autonomous weapons PACE OF BATTLE
systems could seduce states into more armed
conflicts – at the expense of civilian populations. It is often said that the pace of battle is accelerating
to the point where human decision-making is
not fast enough. It is often said that the pace
“Autonomous weapons systems of battle is accelerating to the point where
could seduce states into more human decision-making is not fast enough.
New prototypes of aerial autonomous weapons
armed conflicts – at the expense systems are increasingly being tested at supersonic
of civilian populations.” and hypersonic speeds. This means even faster
autonomous response devices that in turn will
11
require ever-faster weapons. It is not hard to see 8. AUTOMATED OPPRESSION
that such a ‘pace race’ will eventually equate to
humans having little control over the battle-space. Autonomous weapons systems would be an
attractive tool for the oppression of populations and
the suppression of peaceful protest and political
“Humans need to be in control change. While soldiers can in principle refuse to turn
of weapon systems to counter their weapons on their own people, autnonmous
weapons systems would be programmed by persons
many of the potential dangers far away from confrontations and then could kill
with entirely computerised mercilessly on the basis of their coded instructions.
and autonomous weapons.”
9. NON-STATE ACTORS
6. ACCIDENTAL CONFLICT
We are currently witnessing an unprecedented
If the development and proliferation of autonomous diffusion of technology. The cost of robotics
weapons systems, particularly swarms, is allowed development is falling, with the required off-
to continue, supersonic or hypersonic (defence) the-shelf hardware now widely available. If
systems of one state could interact with equally autonomous weapons development is allowed
fast autonomous weapons systems from another to continue it will not be long before we
state. The speed of their unpredictable interaction see crude copies or grey market exports in
could trigger unintended armed conflicts the hands of non-state armed actors.
before humans had the opportunity to react.
10. CYBER VULNERABILITY
7. MILITARIZATION OF THE
CIVILIAN WORLD Humans need to be in control of weapon systems
to counter many of the potential dangers with
We are already seeing the use of new unmanned entirely computerised and autonomous weapons.
war technologies in civilian settings. Law The risks of software coding errors, malfunctions,
enforcement and border control agencies are degradation of communications, and especially
using unmanned systems for surveillance. Some enemy cyber-attacks, infiltrations into the
companies are even arming them with Tasers, industrial supply chain, jamming, and spoofing make
pepper sprays and other so-called ‘less than autonomous weapons systems inherently insecure.
lethal’ ammunition. With autonomous targeting
technology this could lead to violations of human
and civil rights by police and private security
forces with little possibility of accountability.
12 13
HUMAN CONTROL
OF WEAPONS
SYSTEMS This chapter shares guidelines that have
been designed to provide campaigners with
can be fragile. It requires attention and memory
resources and so it can easily be disrupted by stress
tools to assess whether proposed methods or being pressured into making very quick decisions.
of control are meaningful or not. It delivers
Noel Sharkey a plain English guide to lessons learned Automatic reasoning is essential to our normal
International Committee for Robot Arms Control from 30 years of scientific research on the daily functioning, but it has a number of liabilities
human supervisory control of machinery. when it comes to making important decisions such
as those required to determine
Part 1 is a primer on the the legitimacy of a target.
study of human reasoning. It “Automatic reasoning
briefly explains the types of jumps to conclusions.” Four of the known properties
Since 2014, high contracting parties to the Convention on Conventional biases that result in operators of automatic reasoning
making bad decisions and it explains the kind of illustrate why it creates problems for the
Weapons (CCW) have expressed interest and concern about the reasoning needed for meaningful human control? control of weapons. Automatic reasoning:
meaningful human control of weapons systems. Different states
use different terms from ‘appropriate levels of human control’ and Part 2 puts the primer on human reasoning to • neglects ambiguity and suppresses doubt.
work to show the types of human control that are Automatic reasoning jumps to conclusions.
‘person in the loop’ to the ‘wider loop’. There is also the notion of force unacceptable for making targeting decisions. An unambiguous answer pops up instantly
multiplication with one or two people operating a swarm of weapons without question. There is no search for
1. SHORT PRIMER ON alternative interpretations or uncertainty. If
systems. Yes, these are all forms of human control, but the important something looks like it might be a legitimate
HUMAN REASONING FOR THE
question is, what kind of human control is necessary to guarantee CONTROL OF WEAPONS target, in ambiguous circumstances, automatic
reasoning will be certain that it is legitimate.
that precautionary measures are taken to assess the significance of A well-established distinction in human psychology
potential targets, their necessity and appropriateness, as well as the divides human reasoning into two types: • infers and invents causes and intentions.
likely incidental and possible accidental effects of the attack? Automatic reasoning rapidly invents coherent
i. fast automatic processes that are needed to causal stories by linking fragments of available
carry out routine everyday tasks like riding a information. Events that include people are
bicycle, avoiding traffic or playing a sport. This is automatically attributed with intentions that
vital when we need to react quickly or carry out fit a causal story. For example, in the context
a task without engaging our conscious thought. of an armed conflict people loading rakes onto
a truck could initiate a causal story that they
ii. slower deliberative processes that are were loading rifles. This is called assimilation bias
needed for thoughtful reasoning. This is in the human supervisory control literature.
important for making important judgements
such as diplomatic, medical or judicial • is biased to believe and confirm.
decisions and, hopefully, even decisions Automatic reasoning favours uncritical
about getting married or divorced. acceptance of suggestions and maintains a
strong bias. If a computer suggests a target to
One drawback of deliberative reasoning is that it an operator, automatic reasoning alone would
15
make it highly likely to be accepted. This is or absolute. The levels are intended as thought commander should deliberatively assess necessity As the attack will take place unless a human
automation bias. Confirmation bias selects tools to help you to work out whether some and appropriateness and whether any of the intervenes, this undermines well-established
information that confirms a prior belief. new human control method stacks up. suggested alternatives are permissible objects of presumptions under international humanitarian
attack. Without sufficient time or in a distracting law that promote civilian protection.
• focuses on existing evidence and ignores absent environment, the illegitimacy of a target could be
evidence. A classification for levels of human overlooked and confirmation bias could take hold. The time pressure will result in operators neglecting
Automatic reasoning builds coherent explanatory control of weapons: ambiguity and suppressing doubt, inferring and
stories without consideration of evidence or A rank ordered list of targets is particularly inventing causes and intentions, being biased to
contextual information that might be missing. 1. a human deliberates about a target problematic as automation bias could create believe and confirm, focusing on existing evidence
What You See Is All There Is (WYSIATI). before initiating any attack a tendency to accept the top ranked target and ignoring absent but needed evidence.
It facilitates the feeling of coherence that unless sufficient time and attentional space
makes us confident to accept information as 2. program provides a list of targets and is given for deliberative reasoning. Level 5 control is unacceptable as it
true. For example, a man firing a rifle may be a human chooses which to attack describes weapons that are autonomous
deemed to be a hostile target with WYSIATI Level 3 is unacceptable. This type of control has in the critical functions of target selection
when a quick look around might reveal that 3. program selects target and a human been shown to create as automation bias in which and the application of violent force.
he is shooting a wolf hunting his goats. must approve before attack human operators come to trust computer generated
solutions as correct and disregard or don’t search for IN SUMMARY
It should be clear that each of these features 4. program selects target and a human contradictory information. Studies on automation
of automatic reasoning would lead to serious has restricted time to veto bias in the supervision of Tomahawk missiles found It should be clear from the above that there are
humanitarian errors. When people talk about that when the computer recommendations were many types of control that would not fulfil the
various types of human in the loop control systems 5. program selects target and initiates wrong, operators using Level 3 control had tended conditions of Level 1 control. You should be in a
or controlling a swarm, we need to look carefully attack without human involvement to treat them as correct. Level 1 operators were a position now to ask questions about any method of
to find out if they trap the operator in the error- little slower when things went well but performed control and find out how it fits in the Levels shown
prone properties of automatic reasoning. well when computer recommendations went wrong. in Table 1. The biases and problems with automatic
Level 1 control is the ideal. A
human commander reasoning described in Part 1 will help you to assign
2. LEVELS OF HUMAN (or operator) has full contextual and situational Level 4 is unacceptable because it does not the correct level. It might be between two different
CONTROL AND HOW awareness of the target area at the time of a specific promote target validation and a short time to levels or it might need an entirely different level.
THEY IMPACT ON HUMAN attack and is able to perceive and react to any veto and attack would reinforce automation bias Working in this way should assist in determining
DECISION-MAKING change or unanticipated situations that may have and leave no room for doubt or deliberation. risks to International Humanitarian Law.
arisen since planning the attack. There is active
Now that we have looked at some of the relevant cognitive participation in the attack and sufficient
properties of human reasoning, we can see what time for deliberation on the nature of the target,
that tells us about the control of weapons. In the its significance in terms of the necessity and
science world, different way to control machinery appropriateness, and likely incidental and possible
are discussed in term of levels. Level 1 would be accidental effects. There must also be a means for
the best and level 5 would be unacceptable. the rapid suspension or abortion of the attack.
In Table 1, the machinery levels have been Level 2 control could be acceptable if it is
adapted to describe levels of controlling weapons. shown to meet the requirement of deliberating
These should not be considered to be definitive on potential targets. The human operator or
16 17GENDER
AND BIAS
First of all, a note about gender. Gender who identify as men from being something else—
Ray Acheson doesn’t mean biological sex. It means the from acting outside the normative behaviour
socially constructed norms of how we are for men. It prevents gender equality or justice,
Women’s International League for Peace and Freedom supposed to act as women and men or reinforcing the binary between men and women
trans, non-binary, or queer identities. and negating the existence of other experiences
and identities. It prevents all of us as human beings
These norms can and do affect how we think about to explore strength, courage, and protection from
weapons, war, and violence. Throughout history, a nonviolent perspective. It makes disarmament
What does gender have to do with killer robots? we have seen that weapons symbolize power. The seem weak. It makes peace seem utopian. It
association of weapons with power comes from a makes protection without weapons seem absurd.
very particular—and very dominant—understanding
of masculinity. This is not to say that all men agree Looking at weapons through a gender lens is not
with or perpetuate this idea, but that this is widely just an academic exercise. It can help inform
considered the norm or standard for masculinity. disarmament and armament policy. To bring
us back to the question at hand—what does
This is a masculinity in which ideas like gender have to do with killer robots—we can
strength, courage, and protection are equated see that understanding the gendered context
with violence. It is a masculinity in which and implications of certain weapons helps us
the capacity and willingness to use weapons, understand the best way to prevent humanitarian
engage in combat, and kill other human beings harm. Autonomous weapons, also known as fully
is seen as essential to being “a real man”. autonomous weapons, may perpetuate negative
gender norms, or be used to commit acts of gender-
This type of violent masculinity harms everyone. based violence. These possibilities are useful for
It requires oppression of those deemed “weaker” demonstrating the need for meaningful human
on the basis of gender norms. It results in control over weapon systems and prohibiting
domestic violence. It results in violence against weapons that operate without such control.
women. It results in violence against gay and trans
people. It also results in violence against men. A GENDER ANALYSIS
OF TECHNOLOGY AND
Men mostly kill each other, inside and outside of AUTONOMOUS WEAPONS
conflict. A big part of this is about preserving or
protecting their masculinity—a masculinity that Autonomous weapons are being developed in the
makes male bodies more expendable. Women and context of the aforementioned norms of gender
children, obnoxiously lumped together as if they and power. Scholars of gender and technology
are the same thing, are more likely be deemed have long argued that gender relations are
“innocent civilians,” while men are more likely “materialized in technology”. That is, the meaning
be to be considered militants or combatants. and character (the norms) of masculinity and
femininity are “embedded” in machines. These
We are all suffering from the equation of violence scholars argue that technological products bear
and power with masculinity. It prevents those their creators mark. If technology is developed
19and utilized primarily by men operating within a they will be more precise. It is a typical argument autonomous weapons. Facial recognition software Then there is the culture of rape embedded in
framework of violent masculinity, their creations from the perspective of violent masculinity: those struggles to recognize people of colour; voice weapons themselves. One nickname given to a
will be instilled with that framework of thought, using the weapon can deploy violence without fear recognition struggles to respond to women’s voices drone by its operator, for example, is SkyRaper.5
knowledge, language, and interpretation. of facing physical danger themselves; and in turn or non-North American accents; photos of anyone This reflects the culture of domination that is a
argue that it will actually result in less violence. standing in a kitchen are labeled as women; people’s key component of violent masculinities. It also
Erin Hunt of Mines Action Canada has noted that bail is denied because a program decided that a reinforces the institutionalization of rape as a tool
“human biases are baked into the algorithms and Yet as we have seen with drones, this—at least, woman of colour was more likely to reoffend than of war. It helps the operators and developers of
the data we use to train a machine learning program the later argument—is far from the case. The tools a white woman.3 Imagine this kind of bias being the weapon own the use of rape for domination
often reflects our own patriarchal society with its and procedures used for determining targets for programmed into a weapon system designed to and to defeat a target, while simultaneously
class and race issues.” She argues, “One thing to “signature strikes”—attacks based on “producing target and fire upon targets without any meaningful participating in the normalization of rape as a larger
keep in mind is that only around 0.0004% of global packages of information that become icons for human control, without any human judgment to systemic issue.6 It also is an overt sexualization of
population has the skills and education needed to killable bodies on the basis of behavior analysis counteract that bias. It’s not a pretty picture. the nature of imperial violence: those operating
create [artificial intelligence] programing and most and a logic of preemption” 1—have resulted in weapons from far away deploy them unlawfully
of those people were born into pretty privileged hundreds of civilian casualties in drone strikes. The RAPE AND ROBOTS in other countries, penetrating their borders
circumstances. Similarly, a recent estimate done by same risks apply to fully autonomous weapons. If without their governments’ consent.7 Other
WIRED with Element AI found that only 12% of weapons without meaningful human control are Then there is the argument, raised by government weapons can be used the same way, of course.
leading machine learning researchers were women.” deployed on the battlefield or a policing situation, officials and others who try to argue in favour of But with the use of drones and the possibility
programmed to target and engage people on autonomous weapons, that one of their advantages of autonomous weapons, such practices seem
In this context, autonomous weapons, as tools the basis of software and sensors, the risks of is that they won’t rape. This is myth.4 Of course to have reached the level of official policy.
of violence and of war, will likely have specific mistaken identity or unlawful engagement run autonomous weapons can be programmed to
characteristics that may simultaneously high. It is not at all clear to tech workers, scientists, rape. If we’re thinking of them as machines to The imagery of rape and nonconsensual activities
reinforce and undermine hegemonic gender academics, or other experts that weaponized be used to kill people or destroy infrastructure, in this context is not an aberration. A culture of
norms. This in turn may have implications for robots will be able to comply with international we might not perceive this, but an autonomous sexual violence—and subsequent immunity—is part
the notion of men as expendable and vulnerable, humanitarian law or other rules of engagement.2 weapon could be programmed to inflict terror on a of the culture of dominance and invulnerability that
as predators and protectors, and pose population through rape. Sexual violence in conflict is part of the military’s purposeful development
serious challenges for breaking down gender is ordered by states and by armed groups alike of violent masculinities and a “warrior ethos”.8
essentialisms or achieving gender equality “Bias in terms of gender, race, using human soldiers. An autonomous weapon, if However, the idea that drones are invulnerable
or gender justice in a broader context. programmed to rape, would not hesitate to do so. does not necessarily imply that those who operate
socioeconomic status, ability, them are. In contrast, the supposed invulnerability
PROJECTING “POWER and sexual orientation can be It’s also important to consider the broader of drones is based on the dislocation of their
WITHOUT VULNERABILITY” programmed into machines, culture of rape in relation to weapons and war. operators from danger. The user is protected by
Rape and sexual violence are used as weapons distance from the subjects it is targeting with
If we look at how armed drones are used including autonomous weapons.” in conflict. The risk of this kind of violence is the drone. This separates the “warrior” from
and thought about now, we can see that the also heightened during and after conflict. War war, the body from the battlefield. This has
development of fully autonomous weapons present In addition to these concerns, there is also the destabilizes communities and exacerbates already important implications for violent masculinities.
similar risks. The argument for these weapons risk of bias in those software and sensors. If we existing gender inequalities and oppression of
is similar: drones and autonomous weapons are look at bias in programming algorithms, it’s easy women, queer folks, and others who do not
described as weapons that can limit casualties for to be concerned. Bias in terms of gender, race, conform to societies’ standards of gender norms.
the deploying force, and that can limit civilian socioeconomic status, ability, and sexual orientation
casualties in areas where they are used because can be programmed into machines, including
20 21AUTONOMOUS WEAPONS proficiency as a warrior skill. In terms of cyber shape, or biometric information. This reduces devalues male life—it suggests men are relatively
AND THE WARRIOR ETHOS security, soldiers are described as “cyberwarriors” by people to objects, undermining human dignity.13 more expendable than women. It increases the
their commander; technical prowess is elevated to a vulnerability of men, exacerbating other risks
Mechanizing warfare and protecting the militaristic skill. Meanwhile, “Profiles in sources like It also, as scholar Lorraine Bayard de Volo points adult civilian men face such as forced recruitment,
soldier from risk of bodily harm seems to be in Wired reinforce the connection between technical out, “invites and legitimates a masculine response.”14 arbitrary detention, and summary execution.16
contradiction to the ethos of violent masculinity. prowess and masculinity through featuring pictures Affected populations, viewing the perpetrators of
Engaging an “enemy” from a distance to which of the new ‘geek warriors’ in military gear, posing drone strikes as a predatory male, are incentivized More broadly, the reinforcement of gender
they cannot respond is like shooting someone next to the weapons which they pilot remotely, to adopt the masculine protector role in their norms through targeting men as militants works
in the back. It is the antithesis of methods of along with statistics about their kill ratios.” 11 communities, to fight back against the aggressor. against the establishment and sustainment of
warfare that celebrate bravery, courage, and a more equitable society. Framing men as the
sacrifice. “The attempt to eradicate all direct With autonomous weapons, perhaps the tech AUTONOMOUS GENDER-BASED militants, as the protectors of their communities
reciprocity in any exposure to hostile violence workers and software developers will soon be VIOLENCE AND REINFORCING willing to take up arms, in turn reinforces notions
transforms not only the material conduct of posing for photographs with the robots deployed VIOLENT MASCULINITIES of women as weak, as being in need of this
armed violence technically, tactically, and into battle or to police the streets. Regardless, the protection. This continues to enable women’s
psychically, but also the traditional principles of power displayed through detached, mechanized This in turn reinforces conceptions and practices exclusion from authoritative social and political
a military ethos officially based on bravery and a violence inherent in autonomous weapons, of violent masculinities, and can lead to gender- roles. It also reinforces the binary between
sense of sacrifice,” argues Grégoire Chamayou coupled with the arguments that these weapons based violence against men. In conflict, civilian women and men as weak and strong, as passive
in his text A Theory of the Drone. “Judged will not seek revenge, will not rape, and will men are often targeted—or counted in casualty and violent, and refuses to engage with other
by the yardstick of such classical categories, a reduce civilian casualties, do not undermine recordings—as militants only because they are men identities and experiences that do not conform
drone looks like the weapon of cowards.”9 violent masculinities, but reinforce it. The warrior of a certain age. While men are not necessarily to this binary. Reinforcing violent masculinities
ethos of violent masculinity—unemotional, targeted solely because they are men, taking sex also reproduces the power asymmetries and
WHAT DO AUTONOMOUS detached, serious, and rational—is protected. as a key signifier as identity and exacting harm gendered hierarchies that underpin many acts
WEAPONS LOOK LIKE, on that basis constitutes gender-based violence. of gender-based violence against women, queer-
IN THIS CONTEXT? Furthermore, while some may say that it is That is to say, if someone uses sex as a basis for identified people, or non-conforming men.
cowardly to send a machine in to kill rather than assessing whether or not a person is targeted, or
Arguably, they would complete the separation of men, drones and autonomous weapons alike if an attack is allowed (are only men present?), or The damage doesn’t end there. Marking certain
body from battlefield. “One of the troubles with “project a predatory masculinity, a powerful and in determining the impact of an attack later (i.e. populations as threats simply because they are
unmanned aerial vehicles is literally the peril of abusive machine that emasculates targeted men” during casualty recording), then they are using the men of a certain age in a certain location or
becoming ‘unmanned’ in every sense of the term,” (emphasis added).12 As with the rape culture sex of that person not as the motivation for the exhibiting behaviour deemed by algorithms to be
argues Chamayou. Mary Manjikian suggests already reinforced and perpetuated by drones, attack but as a proxy for identifying militants, or suspicious has implications for the normalization
that “media portrayals of the new ‘technogeek autonomous weapons would arguably exacerbate “acceptable targets”. This is gender-based violence. and abstraction of violence. As Thomas Gregory
warrior’ have noted that the men who command the process of dehumanization in warfare that This erodes the protection that civilians should explores, it ignores the people that are affected—
systems like Israel’s Iron Dome mobile anti- is essential to combat. An autonomous weapon, be afforded in conflict and violates many human their bodies and their embodied experiences. He
rocket interception system are not stereotypically using algorithms and software to determine rights, including the right to life and due process.15 asks what happens to the bodies of those who are
male leaders.”10 But rather than accept this and engage targets, also goes even further in targeted by remote warfare technologies. “What
“emasculation” of warriors, the military and its “emasculating” or dehumanizing the “enemy” It also has broader implications in the reinforcement do their experiences tell us about the limitations of
supporters are simply changing the goal posts. than any previous weapon technology. A weapon of gender norms, including violent masculinity. language for thinking about the pain and suffering
operating without meaningful human control Assuming all military-age men to be potential or caused in war? What does it mean when violence
Some media reports, based on the language of will rely on characteristics of objects to sense a actual militants or combatants entrenches the overshoots the more elementary goal of taking a life,
military officials, have come to laud technical target, including the objects’ infrared emissions, idea that men are violent and thus targetable. This dedicating itself to destroying the body as body?”17
22 23While this may be the result of any use of force, an understanding that the operation of weapons ENDNOTES
with any weapon or technology, autonomous without meaningful human control, weapons 1 Lauren Wilcox, “Emodying algorithmic war: Gender, race, and the posthuman in drone warfare,” Security Dialogue, 7 Septem-
weapons, in unique ways, risk undermining programmed to target and kill based on pre- ber 2016, https://journals.sagepub.com/doi/full/10.1177/0967010616657947.
human dignity; committing gender-based programmed algorithms of who is considered to 2 See for example Bonnie Docherty, Heed the Call: A Moral and Legal Imperative to Ban Killer Robots, Human Rights Watch, 21
violence; reinforcing violent masculinities; further pose a threat, used without consent in foreign lands August 2018, https://www.hrw.org/report/2018/08/21/heed-call/moral-and-legal-imperative-ban-killer-robots.
exacerbating cycles of or in the streets of local
3 See for example Bonnie Docherty, Heed the Call: A Moral and Legal Imperative to Ban Killer Robots, Human Rights Watch, 21
violence and conflict and cities, will result in civilian August 2018, https://www.hrw.org/report/2018/08/21/heed-call/moral-and-legal-imperative-ban-killer-robots.
oppression of women “In a context where weapons casualties, psychological
4 See for example Bonnie Docherty, Heed the Call: A Moral and Legal Imperative to Ban Killer Robots, Human Rights Watch, 21
and queer folks. The are treated as tools of power, harm, and destruction of
August 2018, https://www.hrw.org/report/2018/08/21/heed-call/moral-and-legal-imperative-ban-killer-robots.
way that sensors and civilian infrastructure. That
software will be used to violence, and subordination this in turn will result in a 5 Jeremy Scahill and Glenn Greenwald, “The NSA’s secret role in the U.S. assassination program,” The Intercept, 10 February
2014, https://theintercept.com/2014/02/10/the-nsas-secret-role.
disembody targets before of others, increasing the violent masculine response
physically disembodying 6 Erin Corbett, “On Nicknaming Predators,” The Feminist Wire, 22 June 2015, http://www.thefeministwire.com/2015/06/
remoteness and abstraction of from affected communities, on-nicknaming-predators.
the person with worse is reinforcing gender
significant. It points to violence is not the answer.” inequalities and oppressions. 7 Jasbir K. Puar and Amit Rai, “Monster, Terrorist, Fag: The War on Terrorism and the Production of Docile Patriots,” Social Text
an increasing remoteness 20, no. 3 (2002).
and abstraction of violence, an execution of Such understandings should have significant 8 See for example Cynthia Enloe, Bananas, Beaches, and Bases (Berkeley: University of California Press, 1990); Penny Strange,
human beings by machines that, as autonomy implications for our thinking about and approach It’ll Make a Man of You: Feminist View of the Arms Race (Nottingham: Five Leaves Publications, 1983); Franck Barrett, “The
and the use of algorithms are increased in the to the development of autonomous weapons. Organizational Construction of Hegemonic Masculinity: The Case of the US Navy,” Gender, Work and Organization 3, no. 3
(1996); and Maya Eichler, “Miltarized Masculinities in International Relations,” Brown Journal of World Affairs, Volume XXI,
development and operation of weapons, is likely Campaigners can think about how this kind
Issue I (Fall/Winter 2014).
to lead to increasing civilian casualties and of analysis and argumentation could help tech
also to further erosion of the sense of value of workers and policy experts see the need for 9 Grégoire Chamayou, A Theory of the Drone, New York: The New Press, 2014, p. 88.
human life when it pertains to “the other”. meaningful human control over weapon systems. 10 Mary Manjikian, “Becoming Unmanned: The gendering of lethal autonomous warfare technology,” International Feminist
In a context where weapons are treated as tools Journal of Politics 16 (1): 52–53.
The gendered culture of violent masculinities of power, violence, and subordination of others, 11 Mary Manjikian, op. cit., p. 53.
that surrounds the development of autonomous increasing the remoteness and abstraction of 12 Lorraine Bayard de Volo, “Unmanned? Gender Recalibrations and the Rise of Drone Warfare,” Politics & Gender, 12 (2016):
weapons, likely to be embedded within the violence is not the answer. Dealing with violence 65.
technology and its use, will create new challenges and conflict as a social institution, rather than 13 Killing by machine: Key issues for understanding meaningful human control, Article 36, April 2015, http://www.article36.org/
for preventing violence, protecting civilians, a technical challenge to be “solved” with new wp-content/uploads/2013/06/KILLING_BY_MACHINE_6.4.15.pdf.
and breaking down gender essentialisms or weapons technology, is imperative. Understanding 14 Lorraine Bayard de Volo, op. cit.
discrimination. Understanding how autonomous the gender dimensions of both violence and 15 Ray Acheson and Richard Moyes, Sex and drone strikes: gender and identity in targeting and casualty analysis, Women’s Inter-
weapons are likely to be perceived in a gendered technology could help campaigners engage with national League for Peace and Freedom and Article 36, October 2014, http://www.reachingcriticalwill.org/images/documents/
way by their developers, operators, and their new audiences and contextualize our work against Publications/sex-and-drone-strikes.pdf.
victims is crucial to developing policies that can weapons in a broader context of gender justice. 16 R. Charli Carptenter, “Recognizing Gender-Based Violence Against Civilian Men and Boys in Conflict Situations,” Security
help break the cycle of violence. This could include Dialogue vol. 37, no. 1, March 2006.
17 Thomas Gregory, “Drones, Targeted Killings, and the Limitations of International Law,” International Political Sociology 2015,
Vol. 9, No. 3, p. 207.
24 25INTERSECTIONALITY
AND RACISM
Taking one identity such as religious affiliation, When it comes to artificial intelligence (A.I.)
Hayley Ramsay-Jones socio-economic status, age, gender identity, or there is an increasing body of evidence that shows
sexual orientation, and looking at them separately that A.I. is not neutral and that racism operates
Soka Gakkai International is also a useful undertaking because it allows us at every level of the design process, production,
to examine how the discrimination of specific implementation, distribution and regulation.
identities can manifest differently. When doing Through the commercial application of big-data we
so, it is important to keep an intersectional are being sorted into categories and stereotypes.
approach in mind as individuals can experience This categorization often works against people of
What is intersectionality? And why is it important when we are discussing multiple and overlapping points of oppression. colour when applying for mortgages, insurance,
killer robots and racism? With historical and theoretical roots in Black credit, jobs, as well as decisions on bail, recidivism,
Acknowledging the need for inclusion and custodial sentencing, predictive policing and so on.
feminism and women of colour activism, intersectionality is a concept that visibility of marginalized groups has become
acknowledges all forms of oppression such as ableism, classism, misogyny, increasingly important to activists, scholars and
social movements around the world, across a “When we apply biased A.I. to
and racism; and examines how these oppressions operate in combination.1 variety of social justice areas. An intersectional killer robots we can see how
approach highlights that all struggles for freedom
from oppression are interlinked and enables us to
long-standing inherent biases
identify the challenges that a lack of heterogeneity pose an ethical and human
poses to the legitimacy, accountability and rights threat, where some
solidarity present in our movements.
groups of people will be vastly
Highlighting the need for social movements more vulnerable than others.”
to proactively address systemic racism within
their organizations, the importance of inclusion, An example of this is the 2016 study by ProPublica,
visibility and ownership, is essential in order to which looked at predictive recidivism and analysed
break cycles of violence. Focusing on the systemic the scores of 7,000 people over two years. The
nature of racism, how racism would be reinforced study revealed software biased against African-
and perpetuated by killer robots and the potential Americans, who were given a 45% higher risk
threat that they will pose to people of colour2: reoffending score than white offenders of the
intersectionally is a key element of this work. same age, gender and criminal record.4
RACISM AND ARTIFICIAL When we apply biased A.I. to killer robots we
INTELLIGENCE can see how long-standing inherent biases
pose an ethical and human rights threat, where
“To dismantle long-standing racism, it is important some groups of people will be vastly more
to identify and understand the colonial and historic vulnerable than others. In this regard, killer
structures and systems that are responsible for robots would not only act to further entrench
shaping how current governments and institutions already existing inequalities but could exacerbate
view and target specific communities and peoples.”3 them and lead to deadly consequences.
27You can also read
