Why star ratings are more accurate than percentage ratings

A few years ago when we were restructuring the way our hostel reviews were displayed we considered changing the travellers’ hostel rating to a percentage out of 100 rather than a score from one to five.

This seemed like a good idea at first as we would be able to more accurately differentiate between two closely rated hostels. However after consideration we decided against it as we found that it gives inaccurate ratings compared with the rating out of five that we have continued to use.

In theory this would mean that star ratings would break down to percentage ratings as follows:

Well that was the idea, but in practice the average person tends to give an abnormally high score when rating on a scale of one to 100 compared with a star rating from one to five, which would mean that the majority of scores would be between 70% and 100%.

If a traveller is to give a true objective rating they should use the mid-point as an average and score above or below that point if they think the hostel they are rating is above or below average. In other words when working with a rating from one to five, they would start at 2½ stars and score higher or lower depending on the standard of the hostel and working with a percentage score you should start at 50%.

With a rating from one to five it works more or less like this (although the ratings are skewed slightly higher than I would like because of a limitation in the software that prevents you from giving a zero score).

However with a percentage score the ratings are through the roof – just look at the ratings on hostel booking sites like Hostelworld. On these sites the average isn’t 50% (as it should be), but instead it is somewhere in the high 70s or low 80s. If you are not familiar with these ratings you would expect all the hostels to be brilliant, but clearly there are many hostels there that deserve a much lower rating.

I think the problem stems from a lot of travellers wanting to be nice. They stayed at the hostel and found it dirty with limited facilities, but the staff were friendly so they want to be nice in return and give them a good score.

This trend toward giving nice (rather than accurate) ratings stems partly from websites like eBay where users rate each other. On these sites well over 95% of ratings are extremely positive, mainly because users are scared of a negative rating in return. This trend should not carry over into instances where the assessor can’t be assessed but it does.

I think the trend towards unfeasibly high percentage ratings also comes down to perception and the lack of established precedent.

Percentage-based rating systems are a relatively recent phenomenon and have only come into play in the past few years as hostel booking sites like Hostelworld have become more popular.  The ratings on these websites are implausibly high because there is no historic precedent set by professional reviewers for what percentage rating should be applied to different quality standards. With no precedent to follow, travellers writing the reviews just give a high rating because they want to be nice as a favour to the friendly staff at the hostel or they follow the lead set by other travellers who have rated the same hostel.

In contrast, star ratings have been around for years. They were around long before anyone had heard of the internet. The five-star rating system has been used for hotels and for reviews in newspapers and magazines for everything from music, electrical equipment, restaurants and wine to the latest blockbuster film.

Professional reviewers in the traditional media have high standards and don’t just randomly give out four or five star ratings. A high rating is something to be earned. A positive review doesn’t have the same impact when all the reviews are glowing with praise, but it has real meaning when put into context with other less favourable reviews. Ratings are a great way to compare products or services, but only when all the ratings are given objective consideration based on their merit with five stars awarded only to the truly exceptional and half a star to the really dreadful, with honest and objective ratings for everything that falls in between.

In addition to the ratings and reviews that you, the traveller, can write on hostelcritic.com; we have our own star rating system. Our staff are trained to follow an objective set of standards that they apply when giving our own star rating. We have rated over 2000 hostels in over 15 countries and have yet to award an overall score of five stars. We realise that hostels are constantly improving and 20 years from now we may have several five star hostels, and when we do give a five-star rating the hostel will have well and truly earned it.

Many hotels proudly advertise as one or two stars and many chefs consider being awarded a single Michelin star as a highlight of their career, so giving one or two stars is certainly not mean; it is just honest, accurate and truthful. And honesty is something that should be reflected in all reviews and ratings.

Most travellers write hostel reviews because they want to help other travellers. So don’t be afraid to give one star if the place is crap – you’ll be doing other travellers a favour. By giving poor hostels the rating they warrant, the really good hostels with four stars or higher will stand on their own and get the recognition that they have earned.

6 Responses to “Why star ratings are more accurate than percentage ratings”

  1. That’s a really interesting way of looking at it – must be a lot of psychology behind this!

    I think as you say a lot of hotel sites favour % as it looks better with a higher score!

  2. I think people’s instincts about percentage rating come from school grades. In school, 50% is failing, 75% is mediocre and only above 90% is fantastic.

  3. I didn’t even consider school grades, but it makes a lot of sense and I’m sure that it is part of the reason why people give such high percentage ratings.

  4. Ah come on!!!! Why are you people still using the “Crap” word?!?!?! Why?!?!?!?!?! :(

Trackbacks/Pingbacks

  1. A Look at Hostels in the USA - Page 3 - Hostel Management Forums - 19 February, 2009

    […] are more evenly spaced along the scale rather than all grouped at one end. I have written a blog post that goes into more detail about this. __________________ BUG: the Backpackers’ Ultimate […]

  2. Hostel manager sends rude emails to his former guests | BUG Blog - 12 June, 2009

    […] Why star ratings are more accurate than percentage ratings […]

Leave a Reply