When searching online for a new gadget to buy or a movie to rent, many people pay close attention to the number of stars awarded by customer-reviewers on popular websites. But new research confirms what some may already suspect: those ratings can easily be swayed by a small group of highly active users.
Vassilis Kostakos, an assistant professor at the University of Madeira in Portugal and an adjunct assistant professor at Carnegie Mellon University (CMU), says that rating systems can tap into the “wisdom of the crowd” to offer useful insights, but they can also paint a distorted picture of a product if a small number of users do most of the voting. “It turns out people have very different voting patterns,” he says, varying both among individuals and among communities of users.
Full article at Technology Review (thanks, ReliegiousMarie)
Worth noting that crowds are most successful when experts are contributing (ie. stock market, eBay) as opposed to general public contributing (eg. product reviews by non-experts).
I know how giving stars for a product can only produce an average score out of everyone who does it, which is why I always read comments rather than take the stars at face value.
LC x
I only post a review if I’m very disappointed or very, very happy. So indeed. Very distorted!
yeah that makes sence actually, makes alot of sence, i used to go by other peoples ratings, but now i think it’s ideal to go by the info and trailer.
My boss was telling me she’d left a comment on a hotel reviews website, and was reading some of the other comments, some of which she thought were unfairly negative because the hotel was very cheaply priced. I pointed that some people think you should review regardless of price because then people can make up their own mind whetner what you describe is good value or not. And that review sites like that are therefore inherently unreliable unless people state whether they are taking price into account or not.
She said that that was wrong, she thought you should take price into account, which sort of missed (or made) my point…
You certainly can’t trust it for picking lottery numbers. If you took the average of all their predictions, you’d end up with “25″ for all six numbers…
Was it not Terry Pratchett who said “The Average Intelligence of the Mob is the I.Q. of the thickest person in the Mob, divided then by the number of people in the Mob” ?
oh drat. That explains why all there appliances keep breaking that i’ve bought from argos dependent on reviews.
David W: funny you should say that, stock market experts have been used in experiments to show that individual “experts” often do worse than a crowd. It’s in the Wisdom of Crowds book, and they came up in Quirkology as well, check it out, it’s amazing.
Re Crowd Sauce: No, you wouldn’t – only if people really picked lottery numbers at random. Still, they e.g. use birth dates a lot, so numbers >31 are picked less.
However, your point holds – wisdom of crowds is worthless when it is not about knowledge or creativity, but you try to beat a random event like a lottery.
I think if you also look at those who are compelled to write a review / rate something – as Joseph said above.
Let’s say you go on a good-but-not-amazing holiday, or you buy an alarm clock, for example, that does it’s job, doesn’t break, and doesn’t display some unexpected superpower, in my opinion, you’re not going to be as likely to write a review on it. You don’t get many toaster reviews saying ‘fantastic – it toasted my bread!’; they’re probably more likely to be about the toaster breaking.
Apparently crowd wisdom isn’t always necessarily a good thing
http://www.telegraph.co.uk/technology/news/6227357/Web-bot-project-makes-prophecy-of-2012-apocalypse.html
Can you trust the wisdom of authority?
Very true. I dont buy things based on reviews. They’re so skewed :/
@ Lady Claire – i do what you do
I always read the comments rather than look at the star rating because of this…. although i dont really buy much online anyway.
That would explain the popularity of the Three Wolf Moon T-Shirt on Amazon!
http://bit.ly/TtThI
I wouldn’t trust a crowd to successfully identify an emergency that’s for sure!!
Thanks Robert Cialdini
I never check the stars, I see them, but don’t see them as something I should get from. It’s quite often you yourself who will have to judge anyway. Especially gadgets or movies ofcourse (why would you want to read the stars for those?). Professional reviews of e.g. movies, books and such are most often not my kind either .. it is a matter of taste. Those reviews are pretty subjective (what else can it be ..).
I have never thought about it very much, but it’s so true, that the reviews mainly represent a small number of people -the very happy ones and the very unhappy ones. I have to admit that I have booked a rather expensive holiday based on good reviews, and indeed it was a lovely place (Four seasons Maldives).
Also, if I would be a hotel manager, I would probably get friends and family to write good reviews for me?
Anyway, wishing everyone a nice Sunday night
Derren why do you mess with us so?
First you predict the lottery live but show us the numbers after they were picked.
Then you explain how it was done with the wisdom of crowds.
Then only seconds after you say that you might have just rigged the lottery and nothing that just happened matters.
THEN you don’t show us footage of you and some kids on a bus from a year ago of the kids picking the winning numbers.
then you acknoledge the fact that it exists on your blog but show no video or do anything about it.
And now you tell us that Wisdom of Crowds might not even work that well????
WHAT DO YOU WANT US TO BELEIVE?! I am so confused *sigh* Well either way you got me hooked to solving this mystery.
@Joel – Yeh, I’m all confused too. I was just coming to terms with relaxing into trying to be a better collaborator; working hard not to get frustrated when those I work with can’t make decisions, endlessly discuss instead of do and everyone tries way too hard not to step on anyone’s toes – and whomever takes the reigns (which I tend to do after much rolling of eyeballs) is therefore an evil dictator. And now you’ve whipped the rug from underneath my feet and my newly-found non-judgemental trial period grinds to a halt! You evil, evil peeps.
Hmmm,
Well I tend to use my own instincts about most things.
Technology etc.. I’ll look at various sources to check between specifications and the various ‘abilities’ of each differing product. What most of the terms used mean. Look at what I feel I need the item to do, check budget (if any, haha) and then take an ‘educated intuitive’ decision.
Some things though, you just need to go with your own head and heart.
If you can learn to stop doubting and second guessing yourself, you may find that you already ‘know’ a lot more than you initially thought you did.
Pax, amor et concordia.
x
[...] post: Can You trust Crowd Wisdom? « Derren Brown Blog By admin | category: lottery numbers | tags: are-picked, crowd, crowd-sauce, lottery [...]
[[[[[[[[[ Kevo Thomson says:
September 27, 2009 at 6:14 pm
That would explain the popularity of the Three Wolf Moon T-Shirt on Amazon!
http://bit.ly/TtThI ]]]]]]]]]
LOL. I just went to check out that link and the reviews for that shirt are some of the funniest things I’ve ever read.
Here in Germany, we had general elections this week-end.
The University of Munich made a very interesting experiment the months before: They set up an ‘election stock market’, where participants could buy/sell ‘Party’ stocks, which were paid for with real money after the elections were over. For example, a party that got 30% in the elections was paid for with 30 Euros. Trading was possible until friday evening before the election.
Interestingly, this stock market predicted the real election result astonishingly accurate. Better than most public survey institutes. The wisdom of the crowds can obviously yield very good results, if real interest (in this case: money) is behind the motivation.
The so-called Wisdom of Crowds has some basis in statistics, in that one can achieve a better estimate of some quantity by combining a large number of INDEPENDENT estimates. For example, if lots of people estimate the weight of a cow, the average is likely to be a good estimate with relatively small error. This only works for two reasons: the individual estimates have no systematic bias, and the errors are uncorrelated. As soon as people start to influence each other’s judgements, however, these assumptions no longer hold: huge errors can then result. ‘Groupthink’ is an example of this. Another example is stock market bubbles and crashes. Even betting exchanges can exhibit biases – I’ve made a good amount of money over the years by exploiting the ‘stupidity of crowds’.
@ Pahoxx & Kevo Thompson:
LMFAO… Thank you!
Funny as hell those reviews.
Although, I have to admit… I do now feel oddly compelled to buy one. Hmmm.
Pax, amor et concordia!
x
Question:
You know when Derren pick the crowds lottery numbers and averages them. Does he first arrange them in consecutive order first? Because if you don’t as the number of people you are asked tends to infinity the number picked will tend to 25. Anyone remember what he did?
@ captainff
Re: http://www.telegraph.co.uk/technology/news/6227357/Web-bot-project-makes-prophecy-of-2012-apocalypse.html
That’s just not fair.
It’d better be wrong because that’s not much of a prezzie really is it?
Pax, amor et concordia!
x
I don’t think there’s such a thing as the ‘wisdom of crowds’, there’s only mimesis (copying/imitiation). Like birds taking off and flying in a flock all animals display mimesis, which is the almost instantaneous copying of a ‘model’, in the case of a flock the model is the bird next to you. We copy each others desires; we imitate them; we covet them. Its the same thing as saying there’s no such thing as orignial thought or opinion (even this), its all a copy of someone else, even if that person is imaginary/fictional. There appears to be no ‘wisdom’ in it for human beings since mimesis historically leads not only to mild ‘groupthink’ and ‘distortion’, mimetic behaviour in crowds, especially in times of crisis, inevitably leads to violence (usually involving a scapegoat) – no wisdom there!