Farhad Manjoo has posted a good article in Salon on why online review sites for restaurants, hotels, books, and cafes disappoint. On a recent visit to a cafe in Sausalito, California, he found the food so-so, even though the reviews promised the best breakfast in town.
There’s nothing new about this. A lot of people who have booked hotels based on glowing reviews on Tripadvisor, have found their holidays or weekends completely ruined by unfriendly hoteliers, noise, dirty rooms and worse.
I’ve been to restaurants highly rated on Iens, a Dutch restaurant site, only to be terribly disappointed. I realized that it’s easy for friends and family of the hotel or restaurant owner to post glowing reviews. The best way to judge a restaurant is to see over a period of several months what people wrote. I still go to Iens because there’s nothing else. And so it is with Tripadvisor and Yelp. There’s nothing else – yet.
Another problem with mass review sites like Tripadvisor is that for the boutique hotels and B&Bs which will never receive the numbers of visitors that a Marriott does, you will get few or no reviews. This is logical since the hotel or B&B has only perhaps 5 rooms. So if one person had a great (or awful) experience out of the 100 people who went to that B&B this year, is that representative of the experience? Mass market review sites work only for mass market locations (chain hotels, chain restaurants, chain stores).
The Salon article also mentions that most people don’t bother posting reviews unless they had a terrible experience or a fantastic one. So reviews are skewed towards the extremes.
Who is the reviewer – that’s what counts
I think at the heart of every review is the person who is writing it. The Salon article briefly mentions this, but does not give it the importance it deserves. Today there is no way to sort reviews based upon whether the person writing it shares the same tastes as you. There’s no way to find out if the person writing a review about a hotel or restaurant in Paris is someone who travels once every five years or someone who loves food, travels and eats out frequently, and shares the same passion for Banon cheese as you.
On these mass review sites, all opinions are given the same weight. We know that’s not how we do things in the real world. I would assign far more importance to advice given to me by Andy Abramson about food and wine than to a review written by someone who rarely eats out and cares little about wine.
There are two ways to get reliable reviews:
(1) a website open only to a select crowd, an exclusive club, of like-minded people who trust each others’ opinions; or
(2) taste matching on mass market review sites like Tripadvisor.
Some sites like Weekendhotel.nl, which is run by Willem Vos of Amsterdam, already present only a selected number of hotels and B&Bs (note: there are plans to add reviews here too). Other people address this problem by running exclusive mailing lists (e.g. Suzanne’s Files).
But until someone does (1) and (2), I’m relying on opinions of friends who are in the know, and not on a Tripadvisor reviewer whose idea of a perfect weekend is getting drunk in Amsterdam and stumbling into an Easyjet flight back to London.
Here’s what a friend of mine (frequent traveler, wine connoiseur, fellow foodie has to say):
I have found that the review sites that have user generated content (and that includes Zagat) are far less valuable to me than those written by a single source. The “jury†concept for food, wine, hotels, etc. does not work because there is no consistency between the various people writing reviews and the total body of reviews. People reviewing restaurants were all not there at the same time, eating the same food, or under the same conditions. On the people/review side they are not all at the same level of experience. In the end the user generated content sites are still binary to me. If all the reviews say “GOOD†or “BAD†I know to consider the place or avoid it. But if I am considering it, I then do real research for a reviewer who is 1) local and 2) whose opinion I like and trust.
I’m curious to hear what other people have to say about this.
13 June 2007 at 15:15
Hi Esme
If you send me a private e-mail I have an invite for you regarding this subject.
13 June 2007 at 23:14
the wsj just did a long article on tripadvisor which comes to similar conclusions that you do:
http://online.wsj.com/article/SB118065569116920710-search.html?KEYWORDS=tripadvisor&COLLECTION=wsjie/6month
It seems like there are three ways to “qualify” ratings and reviews:
1. People I know
2. People like me (as determined by collaborative filtering
3. People like me (as determined by their self description
Each have pros and cons, and each relies on a large underlying corpus of reviews to be able to cull from
14 June 2007 at 15:19
“But until someone does (1) and (2), I’m relying on opinions of friends who are in the know…”
Indeed, friends in the know should be an option, and if you had a site that showed where your friends had been it would make it that much easier. That is why we started TripConnect.com, which is based on discovering who in your network has been to a location then tapping in to that knowledge to get advice from someone who really understands your tastes. Beyond your immediate circle of friends you can form “invite only” groups or join groups based on activities and interests.
15 June 2007 at 16:08
As of today, someone just did #1- and it’s TripAdvisor:
http://news.com.com/8301-10784_3-9729700-7.html
20 June 2007 at 14:00
You should check out VibeAgent.com – they do both 1 & 2. They’re still in private beta, but if you ask they’ll probably give you an invite – they gave me one!
23 June 2007 at 22:07
Esme,
Great seeing you if only briefly at Supernova.
I’m working on this very problem to a small degree with a project of mine. The idea is to build a series of guide by business people – for business people. Guides more in the Zagats model than the long form reviews ala nytimes or Yelp.
I’ve also just recently learned about wcities which appear to sell the same set of reviews and listings to a very large swath of the online travel sites (which is why, for example, most of them do not seem to list Sarabeth’s Upper West Side location – though it is one of the best places in NYC for breakfast, nor do they list the also great Fairways on the upper west side whose 2nd floor cafe is fantastic).
So my thesis is that while the single voice guide (usually limited to either a small set of cities or a very specific topic) is great – that a possible alternative is a guide which has a very specific purpose and focus.
Yelp or Tripadvisor are trying to address everyone (as is to a large degree Zagats).
In contrast guides such as Lonely Planet or Rough Guides have a very specific (and in those cases nearly the same) audiance in mind – so can be usually richer and more reliable for that audience (but also not very useful if you are not traveling on a budget).
We should have the guides site up later this summer – I’ll make sure to send you a link. The basic concept is something akin to a wiki – but with some editorial layers and likely easier to use tools than most wikis.
The issue of “friends of the chef/owner” etc is one which plagues nearly all online review and discussion forums. In online food discussion (I was active on Chowhounds for many years, then 3 years ago on http://lthforum.com a site friends in Chicago set up as an alternative to chowhounds). On Chowhounds there was a rule of banning of “shills”. LTHForum took a related approach but also had space for the professionals to contribute – just in spaces that were clearly indicated for that purpose.
I think there is a value and role for engaging with the restaurants (& hotels etc) directly. For one, they are in the best position to verify information such as hours of operations, specials, current menues, capabilities & capacities etc.
But the editorial content – the reviews & discussions should be isolated from that involvement.
I also personally find – for me at least – that 100’s of different reviews (ala Yelp etc) are a bit exhausing and overwhelming to go through – and I find myself tiring of reading about people’s dates (on Yelp in particular). Plus I almost always get the sense that my interests and those of most of the reviewers differ pretty radically.
An important but difficult topic.
Shannon
29 October 2009 at 05:59
Hi Esme
How are you doing?
Sorry to bother, but due to my permalink changr this track back gives a 404 (-: