Will driverless technology someday make human drivers obsolete? Would you be willing to trust your safety to a mere algorithm? What if you knew that the algorithm might decide to sacrifice your life to save the lives of others? These are just some of the questions we discuss in this week’s episode—Driverless Cars at the Moral Crossroads.

We human beings are terrible drivers. Think about it—we text and drive, we drink and drive, we drive while half asleep. Part of me would gladly take a computer over a human behind the wheel almost any day of the week.

On the other hand, I love driving stick shifts. Every car I’ve ever owned except my first death trap of broken down Ford Pinto—remember those?—has been a manual transmission. I love that feeling of being in control. And another part of me wouldn’t want to give that awesome feeling up and put a computer in charge.

But although I think of myself as a good driver, and love being behind the wheel rather than in the passenger seat, other drivers simply suck. 94% of all accidents are caused by human error. And it’s not just a few bad apples—even though 15% of drivers cause 85% of all accidents. The real problem is that the average human driver is no cup of tea. The average human driver will cause 3 or 4 accidents and have 7 or 8 near misses for every actual accident over the course of their driving careers.

The thing is, though, that at least people care. Computers don’t. They don’t care about you. They don’t care about me. They don’t even care about themselves! Fortunately, we just need to program them to driveas if人类对他们很重要。但这并没有让事情变得更容易，真的。因为现在我们必须问，哪些人类?他们自己的乘客呢?其他无人驾驶汽车的乘客?行人?骑自行车的人吗?

Of course, in the abstract, every life matters. But the question is whether they should all matter equally to a self-driving car. I don’t know about you but when I’m behind the wheel, my instinct for self-preservation kicks in. And when somebody I care about is in the passenger seat, I do my best to keep them safe.

To see where I’m going with this, imagine that you and your loved ones are passengers in a driverless car. A pedestrian suddenly jumps into its path. The car calculates that it can perform either a maneuver that would harm the pedestrian or one that would harm you. Which should it perform?

工程师们会告诉你，他们会尽最大努力确保这种情况从一开始就很少出现。汽车会不断地扫描道路，寻找危险的迹象，拥有比人类更好的传感器和反应时间。也许这句话有一定道理，但“很少”仍然不等于“从不”。这意味着我们不得不决定，自动驾驶汽车有时是否应该愿意牺牲乘客来拯救行人。这是一个必须面对的问题。

How we answer that question will depend on the what ethical theory we decide to program into our cars. If it’s programed like a utilitarian, it will decide what to do by calculating the greatest good for the greatest number. This means that in some circumstances the car might decide to sacrifice its passenger to save the lives of others.

That may sound fine in theory, but I’m not sure it would work in practice. Ask yourself whether you personally would trust your life to a utilitarian car, programmed to treat you as one human among others, with no special concern for your survival?

有人会问:“但我们每次坐出租车或优步不都已经这样做了吗?”不是真的。不管人类司机对你有什么感觉，他们仍然有自我保护的本能。如果你一开始就知道出租车司机没有这种本能，我敢打赌你会有点不愿意上车。

Look, I’m not suggesting that driverless cars be programmed to be suicidal or to treat their passengers like disposable cargo. But I am suggesting that in the absence of the human instinct for self-preservation—which you cannot possibly program out of a well-functioning human, but might not want to program into a self-driving car—it's very much a non-trivial matter to decide exactly what driverless cars should be programmed to prioritize.

It’s tempting to think that maybe people should be able to choose the moral theory of any driverless car they are passengers in—sort like having the option to upgrade to an awesome sound system or get heated seats. You pay a little extra if you want the car to be a bit partial to you and yours and pay even more if you want it to drive like a ruthless getaway car.

Though that may sound good at first, it would obviously lead to moral chaos on the road. What we need to reach consensus about the moral theory we program our cars to obey, before we turn the roadways over to them. No doubt that will be a hotly debated question for years to come.

One thing is certain, whatever moral theory we, as a society, eventually choose for autonomous vehicles, car manufacturers will no doubt do their best to make passengers feel special and catered to. Until they read the fine print… “warning, in case of a dire emergency, this car may decide, on moral grounds, to do harm to you, its passenger…”