The Mathematics of Murder: Should a Robot Sacrifice Your Life to Save Two?

[TechMan]

Very interesting read.  Should an autonomous car choose it slams into?

http://www.popsci.com/blog-network/zero-moment/mathematics-murder-should-robot-sacrifice-your-life-save-two

[RevDisk]

Very interesting read.  Should an autonomous car choose it slams into?

http://www.popsci.com/blog-network/zero-moment/mathematics-murder-should-robot-sacrifice-your-life-save-two

And what is the legal liability for hardware hackers who then elevate the priority of their own safety over others? What if I reprogram my robotic car to weigh its choices in favor of saving my own life? Is that manslaughter?

Example. Suppose I drive a normal 4 door sedan. As per hypothetical scenario, I could save my life by ramming say, a Smart Car with four nuns and two kittens (obviously killing everyone in the car and turning the vehicle into confetti) with little more than a dented bumper to my car, or drive off a cliff solely killing myself. If I hack the EEPROMs to give myself the highest priority weighting, and it ends up killing said nuns and kittens instead of committing a noble sacrifice, should I be held legally responsible for essentially not committing suicide? Criminally? It's not a minor question. Sure, you could argue I'm being a selfish and horrible person. That's not the question here.

Are we to enshrine the Greater Good into our laws, at the expense of personal choice? Will people be allowed the choice to choose self-preservation, in advanced, without knowing the potential implications thereof?

[Scout26]

And what is the legal liability for hardware hackers who then elevate the priority of their own safety over others?  What if I reprogram my robotic car to weigh its choices in favor of saving my own life? Is that manslaughter?

Example. Suppose I drive a normal 4 door sedan. As per hypothetical scenario, I could save my life by ramming say, a Smart Car with four nuns and two kittens (obviously killing everyone in the car and turning the vehicle into confetti) with little more than a dented bumper to my car, or drive off a cliff solely killing myself. If I hack the EEPROMs to give myself the highest priority weighting, and it ends up killing said nuns and kittens instead of committing a noble sacrifice, should I be held legally responsible for essentially not committing suicide? Criminally? It's not a minor question. Sure, you could argue I'm being a selfish and horrible person. That's not the question here.

Are we to enshrine the Greater Good into our laws, at the expense of personal choice? Will people be allowed the choice to choose self-preservation, in advanced, without knowing the potential implications thereof?

And what if said nuns are on their way to talk the mother of the next Mao/Stalin/Hitler out of aborting her baby?  It is on such small things as this, that history turns.   Supposed the British had better snipers in WWI, perhaps a certain Bavarian Corporal, who was a company runner, would have been just one more name on the causality lists.  

[MillCreek]

I am trying to remember if this was addressed in the Three Laws of Robotics.

[Pb]

If I'm going to buy a robot car, it darn well better choose to save MY life first...

[RevDisk]

I am trying to remember if this was addressed in the Three Laws of Robotics.

I, Robot was a collection of short stories showing that the Three Laws of Robotics were deeply flawed.

If I'm going to buy a robot car, it darn well better choose to save MY life first...

Should you be liable for the consequences of that choice? Punished for being selfish? See the 4 nuns in a smart car scenario.

[AJ Dual]

The good news is that "who to risk or who to kill" decisions in properly designed autonomous car systems will be like the issue of hand-wringing over aircraft fatalities vs. traffic fatalities.

It's human nature to argue and debate about this, but the "trolley conundrum" scenarios in an all-autonomous car system or automated freeway will be a very small percentage of accidents within of a very small percentage of accidents as compared to today's manual driving figures. 

Traffic engineering on freeways and large roadways already has TONS of "as a last resort, kill the one driver/car rather than let it threaten more people"-tradeoffs implicit in it's design, and we accept them because they're mundane and we don't "see" them. And despite what people may think, our modern cars crush and crumple zones, and the way they're constructed are NOT 100% geared towards the survivability of the passengers, but also done with some thought of the safety of others as well.

We as individuals and as a society make choices like this all the time already. Side-effects and adverse effects of medications/drugs all sorts of things.

[birdman]

I think the dominant issue with autonomous cars (bearing in mind, I consulted on a few pushes in that direction) is going to be liability.

Currently, a driver is held responsible, and as such, has insurance.  Let's say your liability portion is on average $500 per year.  Well, assuming 100 million licensed drivers, that's $50 billion.  Given that the profit margins aren't all that huge, let's assume the insurance companies pay out probably 50% of that in a given year.

The issue is when its no-longer an individual responsibility.  When an autonomous car crashes and kills someone, who gets the bill/lawsuit.  If its the individual, then all is well.  But it seems like a good market for trial attorneys.  If its the mfg, then their insurance isn't really insurance, its cost of doing business, to the tune of potentially billions right off on the bottom line, a cost factor that I don't believe had been adequately folded in.

And WHEN (not if) an error is discovered, the punitive damages are going to be insane.

[TommyGunn]

Early versions had the problem all worked out:

[RevDisk]

I think the dominant issue with autonomous cars (bearing in mind, I consulted on a few pushes in that direction) is going to be liability.

Ayep.

Autonomous cars will not catch on until the liability issue is solved. If it is solved. Only "solution" I can see is government intervention, either by passing laws to limit liability, or somehow completely change how auto liability is handled. Only free market solution is private roads with their own contracts and liability waivers, which I don't see competing with the existing road infrastructure. Capital investment necessary is huge, and profit margins are too slim.

The only quasi solution is to have high but not full automation. Human driver exists solely as an override, and accepts all the liability if they fail to override a glitch. Basically, turn drivers into commercial airline pilots.   

 

[MechAg94]

If the only two choices left are a cliff or running over a car full of nuns, the car wasn't very smart to begin with.

I would expect an automated driving system to be able to look ahead far enough or slow to those recommended speeds through blind curves.  I should also be able to tell when/if the tires begin to lose traction.

[TechMan]

My question is how in the heck did you fit 4 nuns and 2 kittens into a smart car?

[fifth_column]

My question is how in the heck did you fit 4 nuns and 2 kittens into a smart car?

Sounds like somebody has a big weekend planned!

[Viking]

My question is how in the heck did you fit 4 nuns and 2 kittens into a smart car?

Plot twist: the nuns and the kittens were already dead, and somebody had stuffed them into a huge blender and then filled the car with the nun-and-kitten goop.

[lee n. field]

I, Robot was a collection of short stories showing that the Three Laws of Robotics were deeply flawed.

The rules for self driving cars better  not be "to serve and obey and protect men from harm".

[brimic]

Plot twist: the nuns and the kittens were already dead, and somebody had stuffed them into a huge blender and then filled the car with the nun-and-kitten goop.

What a neat way to dispose of bodies- place in robot car, program in long trip.

[lupinus]

These *expletive deleted*ers can't even build a ignition switch properly. I'm not sure I want their thinking car thing.

[brimic]

These *expletive deleted* can't even build a ignition switch properly. I'm not sure I want their thinking car thing.

They'll run on a microsoft operating system.

[TechMan]

They'll run on a microsoft operating system.

That's it, BSOD at 70MPH down I75.

[Sergeant Bob]

The nuns are toast.

[HankB]

The rules for self driving cars better  not be "to serve and obey and protect guard men from harm".

FIFY. 

As soon as I saw the I, Robot comment, I thought of Jack Williamson's Humanoid story . . . but you beat me to posting the reference.

Personally, while we might see autonomous vehicles in controlled situations (airport baggage trains or courtesy shuttles?) I think it will be a LONG time before self-driving cars are on the public highways, due to liability issues. Let's say that robot cars are twice as safe as manually driven cars - that would still result in a LOT of injury collisions. If .gov limits or eliminates legal liability for crashes for the manufacturer or seller, I see a response that the powers-that-be won't be happy with at all.

[MechAg94]

Right now, we have cruise control and the newest thing is look ahead radar with auto brake activation.  At least I thought that was an option in some vehicles.  I have reverse sensors that help me avoid stuff.

I can see having some system that looks at the road stripes or reflectors and is able to auto-steer straight down the road.  In combination with the lookahead radar/auto brake activation and cruise control, you would be able to auto-drive straight down "some" roads without running off the road or rear ending someone.  I assume you would need some sort of redundant watchdog system (screaming passenger?) to alarm if one of them isn't controlling right. 

I see quite a few limitations once you get beyond those basic functions.  At that point, you are getting into decision making and lane changes and speed selection that would go way beyond basic control functions.  A lot of winding roads and back roads you would just not be able to do any of this.  What would the system do in the event of a blow out or hitting an ice spot or a water puddle.

Many chemical and industrial plants are very highly automated.  Most of them are still not unmanned.  I think we are a long way from automated cars in common use. 

[roo_ster]

Sounds like somebody has a big weekend planned!

Probably too much but overscheduling is a force of habit.