Should investors even bother looking at product reviews when 99.2% of consumer brands score above average?

Many investors in consumer goods look at product reviews during some period of the due diligence process.  If they see that a certain brand has an average of 4 out of 5 stars they may take it as an indication of the strength of a brand, but as it turns out, this approach could be misguided.  According to our analysis of over 28 million product reviews, 80.9% of consumer goods companies have an average of 4 out of 5 stars or higher across their product lines and 99.2% of companies have an average of 3 stars or higher, meaning those 4 stars may not be such a good sign after all.

We used our machine learning platform Helio to pull a sample of product reviews for more than 11,000 different consumer goods brands and analyzed the reviews of those brands.  We rolled product reviews up to a company level and filtered out any brands that had less than 100 reviews.  The average product review across these brands was 4.26 out of 5.00.  These results are in line with previous findings.  A large-scale analysis of 80 million Amazon reviews found that the average review across products with at least 5 reviews was 4.16.

Averages are one thing, but what about distribution on a category level?  Are some consumer products just reviewed higher than others?  To help answer that question, we grouped all of a company’s product reviews by primary category and then plotted the distribution.  On the chart below, we focused on the 8 product categories that we thought were the most interesting.  Each dot represents a company that is part of the corresponding category and the average review for the overall category is shown in orange.

Distribution and Average Product Reviews for Different Brand Categories

Screen-Shot-2017-09-20-at-11.35.39-AM.png

Peeling back the layers and looking at the average review across categories still doesn’t reveal much variation.  In fact, the difference between the category with the lowest average review (household durables) and highest average review (non-alcoholic beverages) is only 0.19 stars.

One straightforward approach for investors looking for a profitable brand would seem to be to just filter out any brands that score in the bottom ⅕ or so of product reviews and pay special attention to brands in the top ⅕, regardless of the nominal value of the overall reviews.  As it turns out, this approach could also be misguided.  We took our revenue estimates of brands in the top 20% of product reviews and compared them to brands in the bottom 20% below.

Median Revenue Estimates for Top and Bottom 20% of Average Product Reviews Screen Shot 2017-09-20 at 11.40.12 AM

Not only are brands in the top 20% not clear winners when it comes to revenue, but in several categories, they actually lag behind brands in the bottom 20%.  As can be seen, picking a good brand is not as easy as just picking brands with high ratings.  Recent research from the International Journal of Advertising shows that customers can actually be skeptical if products have reviews that are too high, worrying that some of the reviews are fake.  If a product has a few negative reviews, a customer is more likely to believe the positive reviews.  The same research found that the probability of a purchase increases with ratings from 4.2-4.5 stars, but decreases as ratings go higher than 4.5.

The lack of clarity around product reviews and revenue is not just limited to current revenue.   We plotted projected revenue estimates by average product reviews for the 11,000+ products we analyzed and found that there was no obvious correlation between the two, as seen below.

Projected Year over Year Revenue Growth and Average Product Review Screen Shot 2017-09-20 at 11.42.25 AM

To answer the question posed in the title of this piece, for the 99.2% of consumer companies that have above average product reviews, it probably isn’t worth an investor’s time to even look at those reviews.  For the 0.8% of companies that score below average, investors would do well to ask follow up questions, but these are rare exceptions to the rule.  More than just noise, product reviews can actually cloud investor judgement if they are taken at their face value.  An investor may give too much weight to the non-meaningful difference between say a 4.2 and 4.5 and lose track of other metrics like brand, leadership, geography, and distribution, that are much more predictive of company success.

 


The views and opinions expressed in this post are not necessarily those of CircleUp or its affiliates.  Data from Helio is for illustrative purposes only.  It is not meant as an indicator of future company performance or as a measure of suitability for investment.

%d bloggers like this: