A simple yes / no question would improve ratings in the on demand economy

When we are asked to rate the work of people we encounter in the on demand economy – our Uber drivers, Handy cleaners or DoorDash deliverers – those of us with a soul can face a tough choice.

When faced with the ubiquitous 1–5 star rating system, most people give five stars. So many do this, in fact, that Uber will delist drivers in some cities if their average falls below 4.7. This’s stunning. It means that a driver who carries 10 passengers and earns 5 stars from nine and 1 star from one could be de-listed.

Uber’s argument, in reality, is that they’re adjusting for the fact everyone gives five stars reflexively. So to not get five stars, the service must have been terrible.

So here’s the conundrum: if you believe in the value of ratings, and in marking on a curve, you risk your more thoughtful rating approach putting someone who isn’t that bad out of a job. So many people, since we’re not living in a Randian fever dream, just give 4–5 to anything that someone shouldn’t lose their job for. Full marks, then, often means “this wasn’t great, but I don’t want to punish you with unemployment.”

Instead, we should just answer a simple yes / no question: Do you want to be served by this person again?

So, does this avoid the pressure to give good scores under the fear of putting someone out of a job? Yes and no, I think.

No, in that saying you wouldn’t want to be served by someone again won’t help them. At the margin that may cause them to lose their job. So people who don’t want to cost anyone their job face the same problem.

But on the other hand, this radically reduces the risk that your rating causes the wrong person to lose their job. Very few people, I think, would never want their rating to put someone out of a job (eg. In the case of a dangerous driver or larcenous cleaner) It eliminates the scenario where you try to grade on the curve and your perfectly fine driver gets deactivated while a worse driver who had a compulsive five giver stays. By simplifying the question, we reduce the need to worry about the actions of other answerers, which should improve the quality of the ratings.

I love bringing people together to solve difficult problems and implement great solutions - across software, investing, and Effective Altruism.

I love bringing people together to solve difficult problems and implement great solutions - across software, investing, and Effective Altruism.