**Update 2019: I wrote this a long time ago, I'm not sure about the feasibility of a safe autonomous car that depends on a remote operator. The general principle of labor ratio will probably still hold true though. **
The self driving car is just around the corner if you ask Elon Musk. He promises a "Summon" feature that will let your Tesla drive autonomously and pick you up anywhere, and most people envision a car that can make every driving decision by itself. The reality is that humans will still be in the loop, and we are just making the first baby steps away from a 1:1 ratio of operators to vehicles.
One human behind the wheel of a car or truck can handle virtually any driving situation. When people imagine a self-driving car, most think that every possible decision or corner case must be handled in the software before a car can come and pick you up without a driver inside. When Tesla announces that this feature is coming within 2 years, it sounds infeasible.
However, there is a way to move gradually towards full autonomy without having bullet-proof driving software. Let's refer to it as remote autonomy. There will be a human behind the wheel, they will just be in a call center miles away, directing a fleet of autonomous vehicles through any uncertain moments.
As you probably guessed, your Tesla Auto-Summon is going to cost money, probably a recurring subscription or per-mile operation fee. If you are lucky, it will cost less than the Uber you took to the airport.
But you won't have call center workers actually driving your car via video game controllers. The first generation of self-driving cars will be able to handle the easy stuff themselves. Autonomous cars will self-drive down long highways and country roads, but the remote workers will be ready to handle any exceptional situations.
What happens if your car gets stuck in some bad weather along the way? Or a clever pedestrian just stands in front of it and won't budge? The remote operator will take appropriate action, such as directing the car to pull over at the next exit, or calling someone to take a look at the car in person. If there's an accident, you will want someone on-hand to talk to emergency responders.
I think one of the easiest ways to see where we are heading is to consider another case: waste management. A decade ago, it was common for the garbage truck to come every week with 4 or 5 workers to pick up the bins left in the street. We had five operators for one vehicle. Today, in many neighborhoods, garbage trucks have just a single driver and a robotic arm that picks up and dumps standardized garbage cans from the curb. We've already made the first step of reducing the number of human operators from many to one per vehicle.
The next step will be remote autonomy. With a basic autonomous system, a garbage truck could move down the street house-by-house and line itself up next to every can. Back at the office, a worker would just confirm each selection, and perhaps make adjustments if anything was missed. In the first iteration, one person might only be able to manage 2 or 3 trucks remotely. As technology improves, that would increase, but only to a point. With the first generation, a city might be able to save 50% of its labor costs by hiring one driver for every two trucks. With the second generation, it could save even more by moving to one driver for every ten trucks. Eventually, it will be too difficult to solve the final corner cases in the software compared to hiring a small team who will manage a city's entire fleet of trucks.
A similar team of remote workers will manage Tesla's Auto-Summon feature as well as Uber's fleet of self-driving cars.