When I buy something I have a set of expectations about what it will do for me. Those expectations are built from experience, by marketing campaigns, by what I can read on the label or product page, and by reviews. It doesn’t matter whether I’m buying a bar of soap or a taxi ride. After purchase there are three possibilities: the product can meet my expectations, surpass them, or fail.

Clayton Christensen asked business students to try looking at products as things we hire to do a particular job. For example, you might hire a raincoat to stay dry, a cup of coffee to focus, or a drive-thru milkshake to feed yourself on your commute. One consequence of this framing is that product reviews start to look like employee performance reviews. If the raincoat has no sleeves then it’s not very good at its job, unless one of its responsibilities is to keep your arms free.

We rarely get to interview the products we buy before we hire them. Instead, we rely on reviews from past customers to inform our purchasing decision. We don’t trust professional marketers the way we trust other people who bought the same thing before us. Enterprise companies might ask for references before buying your software; I check the Amazon reviews before buying soap.

Star-based reviews are ubiquitous in the world of consumer products: Amazon listings, Ebay sellers, Uber drivers, Airbnb hosts, Google Play apps, etc. It’s the standard way to summarize multiple written reviews into a single comparable metric. In principle this lets you quickly choose from a long list of options. In practice it’s a much weaker signal.

The reason why is that no one knows what stars mean. What is a four-star product? What is a two-star product? How different are they? Nobody knows! Not even the companies showing them to you.

The customer experience teams I work with have taught me people are most likely to take post-call surveys when they are pissed off or overjoyed. I’ve read many product reviews that start with versions of “I don’t normally leave reviews, but this time I had to…” A proper replacement for star-based reviews should increase the likelihood you leave one at all.

Post-call and post-chat surveys use Likert scales to collect and aggregate customer feedback. When you’re asked to review your experience talking to an agent, the prompt will say “on a scale of one to five where one means X and five means Y,” because without well-defined extremes it’s hard to interpret the results.

CSAT and NPS scores depend on this. In the question above, X and Y are most often “very unsatisfied” and “very satisfied.” The number of ratings above 3 are divided by the total to calculate CSAT. If a company asks how likely you are to recommend them to a friend or colleague, that’s NPS. That scale will be 0 to 10 where 0 is “extremely unlikely” and 10 is “extremely likely.” The percentage of ratings below 7 is subtracted from the percentage of ratings above 8. Companies care deeply about this number and use it to proxy customer sentiment.

But when you review products, 1 and 5 stars are not defined. Neither is how many stars means “good enough.” Even if you assume the definitions above, this system has drawbacks for both buyers and sellers. In many markets, 5 stars has become the baseline for decent service: fewer stars can jeopardize a seller’s future on the platform. Knowing this fact changes reviewers’ behavior. I’m hesitant to rate something I think is just okay because I don’t know the impact my review has. I’ll rate something if it’s awesome or if I have a problem with it. This skews ratings towards the extremes.

If you work in tech, it should embarrass you a little that we haven’t implemented better rating systems, even for collections of products that are too heterogenous for a Likert scale. It’s not hard to think of potential alternatives. The demo below uses the idea I opened with - that there are three ways a product can relate to your expectations - to reduce the friction of leaving reviews by making it clear what your rating means.

It does that by swapping stars for a range slider. The slider’s default value is its center, and we’ll interpret that to mean your expectations were met. If that’s the case, rating the product is as easy as clicking Submit. If the product exceeded or failed to live up to your expectations, drag the slider towards the right (performed better than expected) or towards the left (performed worse than expected) to the degree that it did and click Submit. Averaging these ratings over time is a better way to track how customers feel.

That said, this is not intended to be a real replacement. Whatever comes next will have to meet accessibility standards, and replacing star-based ratings means asking everyone to unlearn whatever heuristics they’re using today. That will take time.

For now, imagine I asked you to review your last purchase:

The product has not been reviewed yet. Submit your feedback to say how your experience compared to your expectations of the product.