Philosopher and computer scientist Peter Asaro ’94 wants world leaders and ordinary
citizens to consider the dangers of programming war machines to decide who lives or
Story by KATE ARTHUR
When he was in fourth grade, Peter Asaro ’94 got the assignment of making a Valentine’s
Day mailbox. But unlike most 9-year-olds, he didn’t dig around his mother’s closet
for a shoebox. Instead, he cannibalized circuit boards, a clock-radio speaker and
a remote-controlled tank to build a robotic mailbox that visited his classmates’ desks.
The story is a harbinger of what was to become a defining interest in Asaro’s life:
how technology and people interact and influence one another.
Asaro is a philosopher of science, technology and media. He is also a talented practitioner
of technology, doing innovative research in the areas of virtual reality, human-computer
interaction, artificial intelligence, machine learning and robot vision at the National
Center for Supercomputer Applications and the Beckman Institute for Advanced Science
and Technology. For Wolfram Research, he was involved in the design of the natural
language interface for Wolfram|Alpha, which is also used by Apple’s Siri and Microsoft’s
Bing to answer math queries.
His multiple talents made him a good fit at Illinois Wesleyan, where he double majored
in philosophy and computer science.
Asaro traces his interests in those disciplines to a question raised by one of his
IWU philosophy professors more than 20 years ago: Can machines think?
“I’ve never really got off that question,” he says. “It started me on this path.”
He again combined computer science and philosophy at the University of Illinois at
Urbana–Champaign, earning master’s degrees in both before completing his Ph.D. in
the history, philosophy and sociology of science.
Asaro is now a visiting fellow at Princeton University’s Center for Information Technology
and continues his role as an assistant professor at the New School in New York City,
where he served as director of the School of Media Studies’ graduate program. Widely
published in international peer-reviewed journals and edited volumes, he is at work
on a book that looks at how advanced robotics and social and ethical issues intersect.
As a filmmaker, he wrote and directed a feature-length documentary, Love Machine, examining the surprisingly complex relationships between technology and the human
experience of love.
Asaro is now filming a sort of sequel to Love Machine, a documentary about autonomous “killer robots” designed and authorized to target
and kill on their own without human intervention. Its title: War Machine.
He knows it will take more than a film documentary to get policymakers and the general
public to take the subject seriously. Asaro also realizes the common perception of
“killer robots” derives from fiction like the Terminator films — something that could happen but is still a long way off.
But if you simply replace the image of a machine that looks like Arnold Schwarzenegger
with one that resembles a drone, the reality of lethal autonomous robots becomes more
feasible. “In fact, there are already combat drones capable of autonomous flight and
the autonomous targeting of weapons is in the testing and development stage,” says
Asaro, “so it’s better to carefully consider the implications now, while we still
have a choice.”
In 2009, he co-founded an NGO committed to the peaceful use of robotics and the regulation
of robotic weapons, the International Committee for Robot Arms Control. He is also
a leading expert for the Campaign to Stop Killer Robots, which launched publicly in
London two summers ago.
As an influential thinker on the legal and ethical implications of lethal autonomous
weapons, he has been interviewed by major media outlets and has presented his analyses
at two United Nations conferences in Geneva. He also travels around the world to encourage
political leaders and average citizens to consider and publicly elaborate their nations’
policy on fully autonomous weapons.
This spring, Asaro spoke with IWU Magazine by phone from his Brooklyn, N.Y., apartment.
When could we see killer robots on the battlefield?
Part of that depends on how you define it. What we have been working with as a definition
at the United Nations is autonomous targeting and firing of weapons. If the system
automatically picks targets on its own and has the authority to fire a weapon without
human supervision, that is an autonomous weapon. Under that description certain anti-missile
defense systems have that capability, though they are not really targeting people
— or they are not supposed to be targeting people; they make mistakes sometimes.
There is also some next-generation drone technology that has fully automated capabilities.
The U.S. has been developing the X-47B, which is an aircraft carrier-based drone that
carries bombs and missiles, and can take off and fly autonomously. That system has
been flight-tested and will be ready in five to 10 years. Our real concern is if there’s
some algorithm determining what to target, instead of a human.
You have a responsibility as a military commander to make a decision on what to target
and that is a complicated decision. If you are attacking an ammunition depot next
to a school you have to make a calculation — what’s the potential impact of killing
students in school versus the military value of that ammunition depot. A robot doesn’t
really understand the situation. It’ll just say this pattern matches that pattern
so it’s a target.
We’re a long way from developing systems that can do that responsibly and accurately.
Even if we could, there’s a more fundamental moral question about whether we really
want to delegate the authority to actively kill people to machines.
Which nations are the closest to developing these weapons?
The U.S. has been at the forefront for a long time and has certainly spent the most
money on it. China, especially in drone technology, has kept up. Others in the forefront
of drone technology are Israel and U.S. allies throughout NATO.
How is morality compromised when machines replace human soldiers?
You have a large moral burden to confirm that the killing that you’re doing is morally
and legally justified, and I don’t think machines can do that. They’re just not capable
of moral reasoning. They can plot a calculation and they may do what we want a human
to do in a situation, but that doesn’t make them moral actors.
Machines aren’t really accountable. We can say they didn’t work the way they were
supposed to work … but you’re not going to try a robot for a crime. We’re not going
to say it’s a moral failing of a robot because it didn’t do what it was supposed to
do. Maybe it was a technical mistake or programming error, or maybe it should never
have been deployed.
You have discussed the threat of these autonomous weapons as an important human rights
issue. How does it rise to that level?
Because it really changes our concept of human dignity. If we think about slavery
or torture, those things are bad because the person who is enslaved or tortured suffers
a bad consequence but it’s also bad for everybody. The fact that slavery exists, or
torture exists, harms everyone as a human because it diminishes what it means to be
There’s a real question about whether allowing machines to take human lives independent
of human control and supervision has the ability to diminish what it means to be human
or the value of a human life.
If we think about the use of robotic police forces to suppress peaceful uprisings
or just for policing, do we really want machines roaming around using lethal force
on civilian populations? That very clearly becomes a human rights question. We allow
police to use lethal force for self-defense when they’re doing their jobs, but robots
can’t be killed. With that logic, the robot shouldn’t be allowed to harm people, maybe
to restrain them, but not [to use] lethal force. The use of lethal force in the military
falls under a different set of laws, but human rights still exist in war, so it is
But is it possible robots could be superior to humans in moral judgment or ability
to follow the law?
There are a variety of tasks that can be turned into calculations that computers and
robots might perform better than humans, but moral and legal decisions are not reducible
to calculations. When you decide to be a virtuous person, you make a certain moral
choice or choose one value over another. Through reasoning you ask, who do I want
to be? That’s part of your effort to construct an identity. Robots don’t do that.
They just follow the program that’s been given to them. They don’t have the ability
to step back and think: Is this the robot I want to be or do I want to find another
What about the argument that robots could save human lives by being deployed in combat?
We already design systems that can be remote-controlled, like deactivating roadside
bombs. But there are still people who are responsible and have meaningful control
over the weapons. We don’t really need to automate that in order to get safety for
The groups you work with have encouraged the United Nations to take a stand on this
issue. How successful have you been?
We’re still at the advisory level, trying to bring diplomats from 120 countries up
to speed on how this technology works and what the issues are, and we’re hoping they’ll
move forward toward a treaty. Their next official meeting will be in November and
we hope at that point they will consider whether to move these meetings to a treaty
level of negotiation. What we will focus on is the concept of meaningful human control
and a requirement that any weapon system that’s developed should have some form of
meaningful human control.
Do you share the concern raised by scientists such as Stephen Hawking that computers
with artificial intelligence could soon be robust enough to pose a threat to humanity?
The fact that computers can do calculations and process certain types of data much
better than us is already the case. But what does it mean to say that they are smarter
than us? Google can look up information much faster than we can but it doesn’t know
how to cook an egg.
Knowledge is a very practical thing. We will be able to build really smart, capable
machines. Whether they become self-aware, that’s difficult to define — and if we can’t
define it, I don’t know how we’re going to engineer it. And it seems so complicated;
I don’t think we’re going to do it by accident.
What I think would be dangerous would be for us to delegate the responsibility for
various human activities to machines which are not really capable of performing them.
In those cases they will make mistakes, or we will simply move the goalposts and change
Artificial intelligence is creeping into our lives but we’re accepting it quickly.
Are we being naïve when it comes to our relationship with devices like our smartphones
and self-driving cars?
Smartphones are interesting in how fast they were accepted and how broadly. It took
about a decade for cell phones to be accepted but only three to four years for smartphones.
They track you and create all this data about you. As you install apps, you’re giving
them permission to know who all your contacts are and read your texts. People have
been very willing to trade a lot of privacy for a relatively small amount of functionality
Self-driving cars also raise a lot of interesting questions. Whatever you program
has some kind of consequence. You should always try to minimize harm, but there may
be some situations where it’s not clear what the minimal harm is. Is it better to
run over an old person than a young person? Are you trying to protect the occupants
more than the people outside the vehicle?
I think the algorithms that are going to be implemented in the first generation are
going to be mostly based on physics, and it’ll try to avoid obstacles. If it can’t,
it’s not going to be processing an obstruction as a person or deer or light pole.
Was there a specific point, earlier in your life, where you had an epiphany about
potential moral problems with some of the computer technology you were busy researching?
The first epiphany was in a seminar I took as a philosophy major with [IWU Philosophy
Professor] Larry Colter. It was called “Can Machines Think?” That put me on the path
of wanting to learn more. I took all the computer science classes that were offered,
and I couldn’t get enough of it. I took an artificial intelligence class with [IWU
Computer Science Professor] Susan Anderson-Freed.
I got a great education at Wesleyan, and I don’t think I’d be doing what I’m doing
if I hadn’t done that. I really kept that pattern going as I went to grad school,
trying to mix together computer science and philosophy.
A lot of my work since graduate school has been thinking about this question of social
values and ethics and how that relates to technology. As we build technology, we build
values into them, and that question was always in the background. It’s not so much
the explicit danger, but the fact that we should recognize every technology we use
has these values built into them.
If all your work could answer one question, what would it be?
How we can improve society with better technology. We’re at a place where we have
an enormous amount of technological innovation taking place, but it’s sort of disconnected
from a lot of these more humanist and liberal arts sensibilities of what we really
want technology to do.
We’ve gotten really good at answering questions about how to get technology to do
some specific capability but we haven’t gotten to the point as a society of understanding
what it is we want technologies to do for us. That’s part of what I’m trying to do.
I can see it in the killer robots. Sure we can design these robots to kill people
very efficiently but is this really what we want to do? Or do we want to build technologies
that protect civilians and ensure that human rights and human dignity are respected?
Isn’t that a better goal to start with?
Of all your roles — computer scientist, advocate, author, filmmaker — which do you
personally find the most rewarding?
Teaching is always the most rewarding. Getting to work with the students and seeing
them develop and what they produce is always quite amazing.
I have students who are now working on arms control at the United Nations for their
home countries, who are screening their independent films at major film festivals
and who are developing the next generation of media technologies. It is always exciting
to think that you helped them get to a place where they can make a difference.