When I buy something I have expectations about it. Those expectations are built by marketing, by my past experience, by what I’ve read on the product page, and by reviews. It doesn’t matter if the product is a bar of soap or a taxi ride: When I finally use the product there are three possibilities. It can meet my expectations, exceed them, or fail.

Clayton Christensen taught a framework for looking at products as things we hire to do a particular job. For example, you might hire a raincoat to stay dry, a cup of coffee to focus, or a drive-thru milkshake to feed yourself on the way to work. By this logic, product reviews are employee performance reviews. If the raincoat has no sleeves then it’s not very good at its job, unless its job includes keeping your arms free.

We rarely get to interview the products we buy before we hire them. Instead, we rely on reviews from past customers to inform our purchasing decisions. We don’t trust marketers the way we trust other people who bought the same thing before us. Enterprise companies will ask for references before buying your software; I check Amazon reviews before buying soap.

For consumer products, star-based reviews are ubiquitous: Amazon listings, Ebay sellers, Uber drivers, Airbnb rentals, Google Play apps, etc. etc. It’s a way to summarize multiple written reviews into a single, comparable metric. In principle this lets you quickly decide between a long list of options. In practice it’s a much weaker signal.

The reason why is that no one knows what stars mean. What is a four-star product? What is a two-star product? How different are they? Nobody knows! Not even the companies who host these reviews.

In the customer experience industry, post-call or post-chat surveys use Likert scales to collect and accumulate feedback. When you’re asked to review your experience with an agent, the prompt will say “on a scale of one to 5, where one means X and five means Y…” because without well-defined extremes, it’s very hard to interpolate.

Even then, CX consultants know people are unlikely to take post-call surveys unless they’re pissed off or overjoyed. I’ve read lots of reviews that start with versions of “I don’t normally leave reviews, but this time I had to…” A proper replacement for star-based reviews would increase the likelihood you bother to leave one.

On Amazon/Uber/etc, 1 star and 5 stars are not defined, nor is how many counts as “good enough.” There are clear drawbacks for both buyers and sellers. In competitive markets, 5 stars has become the baseline for good-enough service: fewer stars can jeopardize a seller’s future on the platform. Knowing this fact changes reviewer behavior. I’m reluctant to rate service that’s just okay because I don’t know the impact my review has. I’ll rate something when it’s awesome, when I have a problem with it, and in cases when I know how the feedback is used. This skews ratings towards the extremes.

If you work in tech, it should embarrass you a little that we haven’t implemented better rating systems, even for collections of products that are too heterogenous for Likert scales. It’s not hard to think of alternatives. The demo below builds on the ideas I opened with to reduce the friction in reviewing products by making it clear what your rating means.

Instead of stars, this rating system uses a range slider. The slider’s default value is the middle, which we’ll interpret to mean the product you’re reviewing met your expectations. If that’s the case, click Submit and you’re done. If the product exceeded or failed to meet your expectations, drag the slider towards the left (it failed) or towards the right (it exceeded) to the degree that it did and click Submit. It’s that simple.

Keep in mind that it’s a demo. This hasn’t undergone accessibility testing, and replacing star-based ratings will mean asking everyone to unlearn whatever heuristics they choose stars with today. Still, no matter what new system becomes the standard, we’re overdue for a replacement.

Now, imagine you’re asked to review your last purchase:

The product has not been reviewed yet. Submit your feedback to say how your experience compared to your expectations of the product.