I want to see mainstream autonomous vehicles (AVs), but I remain bearish about mainstream high-level autonomy.
My reasons for bearishness are not related to the technology that powers self-driving cars, or demand for AVs. Instead, it’s the systemic risk that wide-scale AV deployments create. What will change when we have numerous fully autonomous cars on the road?
Most of the commentary about AVs doesn’t consider second-order effects in their deployment so I’d like to start a discussion on that. (In this post I take high-level autonomy as a technological eventuality and assume a political climate that supports that.)
In this post I write about unintended consequences from Level 4 or 5 autonomy.
- Level 4: High Automation. Automated, but geographically restricted, dynamic driving even if a human driver fails to respond.
- Level 5: Full Automation. Unconditional, full-timer performance of all dynamic driving needs.
Many of the arguments for AVs are centered around safety and optimization. Those arguments are hard to disagree with if you are comfortable with change (again setting the tech and political issues aside). First, let’s consider safety. About 3,000 people die per month in the US from traffic accidents (a broader metric than just automotive deaths, but representative here). That 3,000 is a lot, and a number that the AV safety argument draws on, though the fatalities are lower per capita than most other countries. Still, how did we get to the number 3,000 in the US?
If we look at how automotive deaths have trended over the past 100 years, driving has become much safer, especially as measured by deaths per billion vehicle miles traveled (VMT). Chief causes of improvement: better roads, regulations, Ralph Nader-driven safety measures to the vehicles themselves, and fewer vehicle mismatches like fast cars and slow horses sharing the same road.
Those safety and infrastructure measures are not present equally around the world. In some places, traffic deaths are caused more by bad roads and lack of nighttime lighting than anything else. The cars and drivers can only do so much.
As a thought experiment, instead of decreasing, what would it take to increase traffic deaths per capita? With traditional cars, forcing automotive deaths to increase would be nearly impossible. It would take a strange coordinated, long-term purposeful political and corporate attack to eliminate safety regulations like speed limits, seat belts, air bags, car crumple zones, driver intoxication limits, driving age limits, budgets for road repair and clearing, signage, lighting, traffic monitoring, rapid post-crash response, and more. It would in theory be possible to lower automotive safety through those top-down means. But it would not be possible at scale to force or trick individual drivers into killing each other.
Individual, non-scaled shocks to automobile precursors like horses were also common. Many early transportation accidents were caused when horses became spooked, either by other horses or from encountering passing trains, cars, or the occasional marching band.
It’s interesting to look at the safety of horse-drawn buggies today. The nature of the accidents has changed. Today, the anachronous but legal drivers of buggies mostly suffer from accidents when hit by speeding cars and trucks. Many of those crashes today are caused by mismatched vehicle size.
Again, while horses and horse-pulled vehicles were and are less safe than automobiles today, since every horse-driven vehicle required a sentient being to control a sentient being, limits on systemic risk also held. No one can cause horses to create accidents at scale.
The safety argument claims that with AVs the 3,000 monthly traffic deaths of today could be reduced to close to zero. Where humans lose focus, become distracted, get stressed, make mistakes, don’t notice an oncoming accident, fall asleep, drive intoxicated, refuse to buckle their seat belts, drive without maintaining safety features on their cars, AVs would never do that. AVs represent a potential improvement on human driving. Why would anyone argue against that improvement and for the status quo of 3,000 monthly deaths in the US? Who would argue for status quo of 100,000 monthly deaths globally? But what second-order risks are there in this argument?
When it comes to the other argument in support of AVs — optimization — the arguments also avoid second-order effects. The reason for this seems to be similar to why we seldom hear of unintended effects in other types of new business innovation. That is, the businesses that innovate are rewarded for their innovation and presumed ability to scale to a large market. They are not rewarded for considering whether their innovations increase risk elsewhere.
The optimization arguments center on what would be possible with fully autonomous vehicles. And I like to imagine the potential. Self-driving cars could coordinate to distribute traffic, travel much closer together than human-driven cars, free people from the drudgery of unhealthy and boring driving for more productive work, collect and share environmental data, moderate demand through price changes, carry the very young and very old safely, allow people to sleep rather than manage their commute, reduce needed fuel, and on and on.
Over the past decades, systemic shocks to optimization of traditional driving have operated differently than they probably would for AVs. These previous shocks were in the form of oil embargoes, regulation, and company-level shocks affecting commodity prices (sheet metal etc) and import/export duties.
Chaos at scale
AVs and people will interact differently in practice than they do in theory, both in individual scenarios and at scale. Individually, people break rules, cross the street outside of the crosswalk, bend down to tie a shoe in the middle of the street, drop something, look at their phones, get angry. They are sober or intoxicated, trying to preserve themselves… or trying to destroy themselves. Importantly, they behave differently depending on what vehicles they encounter.
When I lived in New York City I’d occasionally see people slap the side of a limo as its driver attempted to turn sharply on streets designed for smaller cars. Why did they slap the limo and not other cars? The turn was too wide, it’s carrying someone “rich,” it’s fun, it’s a way to express themselves… It’s something that was not uncommon in NYC, while it might be unheard of elsewhere.
They also slapped the limo because of the distance between the back of the car (the slap) and the driver. It’s not easy for a limo driver to hop out of his car and berate someone or chase them (though I have seen that too).
AVs will create different reactions as well, in the individual case like the example above, though they may decline over time as people get used to seeing AVs.
As attractive as the benefits of AVs, let’s consider how they open up different risks than traditional vehicles. Here I’m talking about the opportunity for disturbing improved vehicle safety and optimization, at scale.
Connected vehicles have had their performance hacked, controls hacked, and have been infected with ransomware. The Car Hacking Village is one of the most interesting things for me at a security conference. GPS navigation has been hacked. Collecting information (audio, video, location, and rider details) is possible while not disturbing the AVs movement. When AVs are mainstream and coordinating with each other, the potential for wide-ranging disruption grows along with the potential optimization gains. As shown from percolation theory, only a small percent of cars need to be hacked for major traffic disruptions. It’s something that becomes possible with system complexity and possibilities to scale the changes.
The trade-off looks like this.
Current state of traffic deaths: You will suffer from 3,000 monthly traffic deaths in the US and higher per capita rates in many other places. You may be able to decrease these rates further through better safety features. There is still a lot of room to improve road infrastructure around the world. Improved safety features and lower level autonomy will make further improvements. The rate of change will be slow. But as long as humans drive you will probably have traffic deaths as a major cause of death.
Potential future mainstream AV state: You have the potential to dramatically decrease traffic deaths. But once in a while deaths and chaos will shoot upward as an AV fleet is hacked. Unpredictable human behavior will continue to disturb theoretical safety and optimization. Other hacks, like privacy loss at scale, will not impact transportation but will affect those transported. While going from 3,000 monthly deaths to perhaps 30 is amazing, those 30 will be felt 100 times worse as it will be the fault of a company, not a fellow human.
Considerations
- Human behavior will change as a result of mainstream self-driving cars. This may be a short-term change felt across part of a generation, but will be driving history to deal with.
- There is systematic risk from tampering with AVs at scale, to both the safety and optimization gains. In traditional driving, some uncertainty saves lives.
- Do you choose stable high death rates or potential low death rates that may spike up without warning?
- There will be different and more extreme reactions to the theoretical fewer accidents and deaths caused by machines and corporations, as compared to the larger number caused by individuals.
- As AVs become cheaper per mile than traditional cars (a common claim), people will consume more car travel, reducing the fuel and health benefits of AVs. This effect is already seen with ride share today.
- There is systemic risk of losing access to certain ways of doing things (traditional human driving) and depending on a larger more complex and complicated system.
- Perhaps a car being used for only two hours a day is not bad. After all, I only use my office for eight hours a day. I only use my laptop for a around that much a day. I only use my cooking utensils for an hour a day. I only use my shower for 10 minutes a day. Maybe cars are cheap as is and that economic argument is invalid.
- Are there industries and goods that, well, “depend” on automotive deaths? For example, organ donations, road cleanup, towing, and more.
- Agreed that autonomous vehicles open up a lot of new possibilities.