Qualmark skews its star ratings too high

Qualmark is New Zealand’s main star rating agency which is the ‘official’ star rating used by Tourism New Zealand. It was established in partnership with Tourism New Zealand and as such it has an interest in promoting tourism rather than providing travellers with impartial advice.

Furthermore, like many other – so called, official – ratings agencies, they are paid by the accommodation establishments for the rating. This cosy partnership between accommodation providers and the organisation that is supposed to be rating them just leads to higher ratings than the hotels and hostels deserve.

As an independent travel guide BUG (and hostelcritic.com) never accepts payment for a rating. Having any sort of direct business relationship with a hostel that we rate would jeopardise our independence. Our aim is to provide independent consumer advice, which means that we review and rate a hostel for the benefit of the traveller – not the hostel.

Clearly when an organisation like Qualmark is being paid by the accommodation it is supposed to be rating and is partnered with an organisation involved in promoting tourism, then the ratings are likely to be skewed towards a higher star rating. After all, they look after their customers – the hotels and hostels they rate – rather than travellers like yourself. Looking after their customers means giving a star rating that hotels and hostels will be proud to display.

I am not saying that they are corrupt in any way. Qualmark (and other ratings agencies such as AAA Tourism in Australia) follow a strict set of guidelines to come to their rating and I am sure that Qualmark’s assessors take their job seriously. However the algorithm used to calculate those ratings produce higher ratings than a truly independent ratings assessor such as hostelcritic.com (or Michelin if you’re talking about fancy restaurants). After all they look after their customer, the hotels and hostels that pay for the ratings; while hostelcritic.com looks after our customers, the travellers who use our website and buy the BUG guidebooks.

You would expect that the ratings would be spread between a ½ star and five stars with the majority of ratings near the middle (2, 2½ and 3 stars). That is the case with hostelcritic.com’s ratings, but Qualmark’s ratings are through the roof. Their backpackers’ hostel ratings do not rate a single hostel less than three stars and even awful hostels like Wedgwood House YHA in Picton and Barnacles Seaside Inn YHA in Paraparaumu get at least three stars while those same hostels barely manage to score two stars with their hostelcritic.com ratings.

New Zealand also has BBH‘s backpacker perception ratings. This is a more subjective rating system than Qualmark’s system, but it is generally more trusted than Qualmark for giving an indication of a hostel’s quality. However it is isn’t a complete rating system as it only rates hostels that are part of the BBH hostel network, thereby eliminating most YHA hostels from their ratings.

Refer to the following table for an idea of the disparity between Qualmark, hostelcritic.com and BBH ratings:

Hostel Qualmark rating BBH rating hostelcritic.com rating
Accents on the Park (Nelson) 5 stars 80% 4 stars
Anndion Lodge (Wanganui) 5 stars 4 stars
Base Backpackers (Wellington) 4 stars plus 3½ stars
Criterion Art Deco Backpackers (Napier) 4 stars plus 3 stars
Downtown Backpackers (Wellington) 4 stars 66% 2 stars
Kiwipaka Rotorua (Rotorua) 4 stars plus 2½ stars
Moana Lodge (Plimmerton, Wellington) 5 stars 89% 4 stars
YHA Auckland International (Auckland) 5 stars 3½ stars
YHA Franz Josef (Franz Josef Glacier) 5 stars 3½ stars
YHA Mount Cook (Mount Cook) 5 stars 4 stars

While the hostels may love having a high Qualmark star rating, it is a disservice to travellers having all the ratings bunched up at the top end of the scale. For instance, in Wellington there is only one star separating Downtown Backpackers (Qualmark awards them 4 stars, hostelcritic.com awards them 2 stars) and Moana Lodge (Qualmark awards them 5 stars and hostelcritic.com awards them 4 stars). Qualmark’s rating of these two hostels would have you believe that they both offer a similar standard of accommodation but in reality these two hostels couldn’t be more different. Moana Lodge is a lovely place that has worked hard to earn its high rating, on the other hand Downtown Backpackers is a big old place in a run-down building that most travellers tolerate for its central location. Qualmark’s rating only separates these two hostels by one star.

Compare this with other star rating agencies such as the English and Welsh Tourist Boards, which assess star ratings but are more accurate with lower ratings awarded to simpler or poorer quality hotels and hostels. This is better for the traveller as it gives a better indication of what a place is really like as opposed to Qualmark’s system, which basically involves saying that all the hostels are excellent when they all clearly are not.

Fortunately travellers have the alternative of BBH and hostelcritic’s own independently assessed star ratings.

2 Responses to “Qualmark skews its star ratings too high”

  1. You must be joking. All star ratings are based on within the country they assess. Qualmark is based on traveller research and what they look when staying in backpackers. Most important is Health & safety and cleaniness and also covers business practices, facilities,legal requirements and evacuation procedures. There is no other system like it in the world and also Qualmark is the envy of many countries.
    A valued Qualmark licence holder. not a one night guest survey

  2. Hi TimnThank you for your assessment of our hostel. I thought I would mention that we did achieve 94% in our BBH ratings for 2010 but after a lot of thought decided not to renew our Qualmark licence despite achieving 5 stars as it didn’t seem to make a difference. In our own marketing research, we found fewer than 1% of our visitors mentioned it as an important factor in their decision to come and stay with us. We were left thinking that the essence of what they were assessing was not necessarily what we were striving to achieve. When they give equal or more emphasis to the quality of an authentic customer experience (which includes cleanliness surely) we will re-think our position. I’m sure eventually there will be change but for now Qualmark doesn’t, to our way of thinking, reflect what so many absolutely wonderful backpackers offer throughout New Zealand. The many, smaller, personal, boutique, high quality places are the envy of the world but they couldn’t possibly meet the requirements of Qualmark for bars, air-conditioning, multiple facilities etc etc Many may well score 1 or 2 stars by Qualmark’s system. However these same hostels are the backbone of this popular accommodation sector in New Zealand (with or without Qualmarking) and are supported by guests of ALL ages. That’s my twopence worth anyway.nHappy travellingn

Leave a Reply