In an unavoidable collision involving a robotic driverless car, who should die? That’s the ethical question being pondered by automobile companies as they develop the new generation of cars.
Stanford University researchers are helping the industry to devise a new ethical code for life-and-death scenarios.
According to Autonews, Dieter Zetsche, the CEO of Daimler AG, asked at a conference: “if an accident is really unavoidable, when the only choice is a collision with a small car or a large truck, driving into a ditch or into a wall, or to risk sideswiping the mother with a stroller or the 80-year-old grandparent. These open questions are industry issues, and we have to solve them in a joint effort.”
Google’s own self driving car gives cyclists extra space if it spots them in the lane, which theoretically puts the inhabitants of the car at greater risk of collision, but does it anyway. This is an ethical choice.
“Whenever you get on the road, you’re making a trade-off between mobility and safety,” Noah Goodall, a researcher at the University of Virginia.
“Driving always involves risk for various parties. And anytime you distribute risk among those parties, there’s an ethical decision there.”
Google is constantly making decisions based around information and safety risks, asking the following questions in a constant loop.
1. How much information would be gained by making this maneuver?
2. What’s the probability that something bad will happen?
3. How bad would that something be? In other words, what’s the “risk magnitude”?
In an example published in Google’s patent, says Autonews, “getting hit by the truck that’s blocking the self-driving car’s view has a risk magnitude of 5,000. Getting into a head-on crash with another car would be four times worse -- the risk magnitude is 20,000. And hitting a pedestrian is deemed 25 times worse, with a risk magnitude of 100,000.
“Google was merely using these numbers for the purpose of demonstrating how its algorithm works. However, it’s easy to imagine a hierarchy in which pedestrians, cyclists, animals, cars and inanimate objects are explicitly protected differently.”
We recently reported how Google has released a new video showing how its self-driving car is being taught to cope with common road situations such as encounters with cyclists. We’d far rather share the road with a machine that’s this courteous and patient than a lot of human drivers.
We’ve all been there. You need to turn across the traffic, but you’re not quite sure where, so you’re a bit hesitant, perhaps signalling too early and then changing your mind before finally finding the right spot.
Do this in a car and other drivers just tut a little. Do it on a bike and some bozo will be on the horn instantly and shouting at you when he gets past because you’ve delayed him by three-tenths of a nanosecond.
But not if the car’s being controlled by Google’s self-driving system. As you can see in this video, the computer that steers Google’s car can recognise a cyclist and knows to hold back when it sees a hand signal, and even to wait if the rider behaves hesitantly.
Add new comment
69 comments
I think you're clutching at straws here
This scenario is no different to when you have driving lessons and you're taught to drive slowly past parked cars - in case a child jumps out. If you don't give a very wide berth and you're travelling at a speed where you can't react to something jumping out then you've fucked up.
If you can't give a wide berth because the road isn't wide enough, you go very slow.
Again, no need to be making decisions about who to kill.
Yes, but no matter how slowly you drive past a line of parked cars, there always comes a point where there exists the potential for a child to run out before you can react.
At 40mph, that distance might be 40 metres away, at 10mph mph that point might only be 10 metres away. But even if you crawl along at 10mph every time you pass a parked car - a child might run out when you are only 5 metres away and you still don't have time to react.
As I said... At some point it is simply the other person's fault. That will be tragic in some cases, but no more or less tragic than times that person is killed by a human driver, and overall the improved safety will outweigh the handful of times that someone dies who would have died anywy.
There's always the "suicidal third party" problem; when someone jumps in front of you, giving no time to avoid. There's nothing you can really do about this with human or computer controlled cars, although computer controlled cars would in most cases fair better due to better reaction times and better vehicle control.
Lets take your example where someone jumps in front of you "a few metres ahead".
A human driver would certainly hit the cyclist at 30mph. Human reaction time means they wouldn't manage to press the brake pedal before the collision.
A driverless car would react in a fraction of a second and would immediately engage the brakes. The car would still probably collide with the cyclist, but at a slower speed, perhaps 20mph, which makes the crash an order of magnitude more survivable for the cyclist. Win!
Completely fair points, thank you and well said.
That last paragraph though, could you please outline a scenario where you need to choose between killing a cyclist, a grandmother, or the vehicle occupants where you haven't royally fucked up? Just one scenario. If you can do this, I'll edit my post to include the word "almost certainly". I just can't fathom how you can ever be in that situation.
That's a rather large assumption, would one of these cars crawl along the road at 15mph if there are people on the pavement? I doubt it, if these cars crawled around slowly then no-one would want one. And there is often no room for giving 'a wide berth' on Britain's roads.
If there's no room for giving a wide berth, of course they'll drive appropriately slow; as a human driver *should* be doing now. This is only really an issue in cities with lots of pedestrians near roads, where speed limits are low anyway. I don't see the issue here, it's not as though they have pavements running next to the motorway. If I'm driving down a country lane and I see a pedestrian, I slow down and drive wide if possible. I don't carry on at 60mph, as many people would.
There will be times where these cars go slower than humans would in the same scenario. But to claim that overall journey times will be slower as a result is beyond stupid. And in those instances where they are slower? That's brilliant, because nobody is fucking dead. There are so many areas where autonomy could completely blow human ability out of the water:
-How often do you sit at a red light when there's no traffic around? Autonomy would mean you don't even need traffic lights.
-How often do you slow down and stop to go over crossroads, roundabouts etc? Autonomy would give the power to map where the traffic is heading, set an appropriate speed, and cross straight over that roundabout / crossroads without need to slow down and check for traffic
-The "2 second" rule wouldn't be needed. You could fit twice as many autonomous vehicles on the same journey.
-Speed limits could in theory increase in many areas where it's appropriate, and it would be far easier to manage different speed limits based on what time of day it is.
There are just so many benefits. To suggest that nobody would want one because they'll drive as slow as they *should* drive when near pedestrians is short-sighted and ignorant.
I've looked closely at this, even autonomous cars would need to leave over a 1 second gap purely because of differences in braking distances, even the same model of the same car can have hugely different braking distances because of wear and tear and the amount of driving that day.
See how distances vary:
The Power to Stop - Car Comparison - Feature Article - Page 3
Now, because drivers already tailgate, you won't see a particularly noticeable difference in the gaps between cars from those you see now. Note that most of the stopping distance of cars is in the braking distance - not the reaction time.
I am not sure that twice as many cars is a recommendation.
Driverless vehicles will kill many times less people every year than human-driven vehicles. Everything else is irrelevant.
Automatically driven cars are a game changer not because they will be owned privately but because they will potentially replace commercial vehicles.
This is a good point. We can expect vociferous campaigns against them from cabbies in particular.
That'll be entertaining at least. Joint protests from Uber and black cab-drivers?
What if you use it for both, getting to work during the week, and for the pleasure of using it on weekends? Does it then morph magically back and forth between toy and non-toy status?
What a ludicrous argument to make, indeed giving off a distinct troll smell.
@ NiallMcA Not. Obviously. A car can indeed be a toy, but not if it's getting you to work/Tesco/holiday. It's the use of something solely for the pleasure of using it that confers 'toy' status. Not that we don't all enjoy playing occasionally...
If you want public money to be spent on a pursuit which is mostly (and with some credibility) seen as a hobby for the moneyed middle classes, public image matters. says sidesaddle
I would have thought, and indeed hope, that most posters and readers of road.cc don't have this attitude.
Cycling throughout northern Europe is seen as a form of everyday transport, not a sport/leisure pursuit. If we are to deal with the problems of mass motorisation (noxious and greenhouse gas emissions, noise pollution, visual intrusion, danger to all other road users, degradation of the local environment, aggregate "external costs" - for those into conventional cost-benefit analysis, health problems for those going by motorised transport, loss of local community etc.) then we need to support cycling and walking properly. That's everything from law enforcement and deterrent sentencing through the right kind of highway engineering and, if necessary, vehicle engineering which reduces the potential of drivers to endanger others - like cyclists.
I do know that for many cycling is a bit weird and done by a strange minority. But that doesn't mean that we should go along with this kind of prejudice - in fact, we should oppose it.
Let's face it an autonomous car is unlikely to be sold on performance, so that just leaves comfort, style and safety. One problem that I foresee is that whilst you can sell a car on it's safety rating, most buyers would consider the safety of the 'driver' and passengers as having priority. Therefore if manufacturers are not tied down to a set of algorithms (and of course they will cheat, just like they do with MPG figures) then market forces are going to favour the cars that prioritise the passengers. Just wait until the accident stats start coming out, official or otherwise and see how quickly they affect people's perception of which brands are the safest.
Don't get me wrong, I can seriously contemplate a time when I own or use a driverless car, I just think that there is a mountain to climb in terms of legislation, bringing the legal codes up to date and defining the boundaries of product liability.
I think there will be a collaboration to agree a universal algorithm. Not to do so would leave each company open to be sued by one party or other every time there is an accident.
Also in an inevitable collision it would be important to ensure that each vehicle knows exactly what the other one is going to do. You don't want to get a situation where both vehicles decide to take out the cyclists to avoid crashing with one another and everybody ends up being killed.
There is already a universal protocol, called the Highway Code (plus various case laws). For all the bile the legal system actually solves these ethical problems pretty well (albeit it is sometimes spotty is in enforcing its own rules). Follow that and you have no lawsuit.
It's entirely ethical for a company to design their product to avoid lawsuits, on the basis of trusting that the law is doing its job and that, therefore, it its product does not generate lawsuits it must not be misbehaving.
The implementation is up to each individual OEM, personally I'd rather have variety. If OEM #1's implemenation doesn't cover some edge case, I'd rather only one car on the road failed rather than all of them.
You missed the single most important unique selling point of a driverless car; the one thing that is a game changer for the owner - and that is practicality.
* Out on your bike and suffer an unrepairable mechanical problem? Get out your iPhone and press the "Come get me" button on your Car App.
* Want to ride a non circular route? Drive to the start, get your bike out, and then send your car to the end to wait for you.
* Going to the pub? Get your car to pick you up when you're drunk.
* Most families with two cars will likely be able to manage with only one, saving a huge amount of money.
There are so many new scenarios that are simply not possible without an autonomous car.
Exactly. Yet people still pick on tiny details and fringe cases whilst missing the point. Just look at some of the things people are banging on about here:
-What happens when a cyclist jumps out from behind a skip whilst a grandmother is on the pavement immediately near by
-The thought that these cars may (or may not) go really slow around pedestrians
-Three paragraphs on why the 2-second rule might not necessarily change
People just can't see the wood for the trees.
And we haven't even touched on things like you'll be able to do things whilst you get driven around such as working/eating/communicating etc.
But...that's how it is already!
Or did you miss out the word 'safely'?
I still wonder though - how does this claim fit with the argument that the human driver will always be ready to over-ride the system if something unexpected happens?
* Nowhere to park you car nearby? Send your car back home to park!!!!
Errrr
They should do it as configuration options for the owner when they buy the car.
Erroneous assumptions that the car will be able to determine a persons age, determine there is a baby in the pushchair, be able to recognise non-moving things as people etc etc.
It's more like: there's a thing roughly person sized and it's moving so try not to hit it.
Ps if you watch the video (which isn't new), at 0:36 you can see road-workers in the bottom right video, but on the main screen - the cars interpretation, the car does not seem to recognise the workers as people, it doesn't seem to distinguish them as being different from the traffic cones until quite late and even then the determination doesn't stick.
Do people have no inkling of human nature? The moment driverless cars hit real streets people are going to play with them. Hapless passengers in the thing will be subjected to constant brake tests and wild swerving as its Asimovian predilections are tested to the limit. Despite the average road.cc users paranoia virtually all road users treat each other with at least some respect and consideration. How much respect does anyone have for a computer? So we'll summarily pull out in front of it, box it in and often leave it in stationary electronic turmoil as it realises that ANY move it makes is to some extent risky. I can also foresee situations where it will be extremely loathe to overtake cyclists, not improving our public image one little bit.
Oh dear, that famous "public image" again.
If cars are less likely to be driven too close to cyclists, that's a good thing.
If you want public money to be spent on a pursuit which is mostly (and with some credibility) seen as a hobby for the moneyed middle classes, public image matters.
So you don't see it as a way for kids to get around then? Or for those who choose not to/can't afford a car?
Choosing not to own a car is in no way similar to not being able to afford one. A few years ago I was in the latter class, towing a trailer on my Ammoco and touting some of Aldi's finest luggage (front basket highly commended, panniers less so). The very moment I could afford a car, I did. I'll guarantee it cost vastly less than your bike. I'm afraid you've largely just underlined the point I made above.
Pages