Ask Not For Whom The Bell Tolls, It Tolls For Community Tasting Notes
Grades are getting better. Are our nation’s school kids getting smarter? Maybe. Are grades getting higher? No doubt. Kids are coming out of high school with a 4.3 GPA, a meaningless number when paired up with the GPA of a kid coming out of a school with unweighted classes. The number means nothing out of context. But, when paired with a student’s rank in class and school strength, a picture begins to come through.
Same thing is happening to wine tasting notes. Scoring a wine once was an objective process. The wine was given points for clarity, taste, finish, nose, etc… The points were added up and a stamp was slapped on it. This wine is an 86, so let it be written. Just like grades, scores began to creep up. Instead of an 85 being a fine score, it was a quasi-failure. Sure, you got an 85… but you didn’t get a 90. So many dollars are at stake that low scores can’t be risked or tolerated. So, what happened? Scores slid up.
16,029 tasting notes are logged in Vinfolio’s Vincellar as of Feb 12. Of those 16,029 notes, the scores broke down as such:
Score inflation ahoy! Let’s see that in chart form.
With the unwashed masses slapping scores on every bottle rolling off their Vinotemp fridges or Safeway* shelves (N/A in Maryland), scoring is a relative measure that is sliding the bell curve to the left.
Everyone’s bell curve peak is different. I like mine to be around 85. 85 is a good ol’ wine. Over 90 and it moved me. Below 80 and I probably won’t go out of my way to drink again. Others, like Jon the DC blogger, like to use the whole range available. A 77 from him might mean that he liked the wine just fine. Extrapolate this problem out to the thousands of winos who are banging tasting notes into websites and one quickly loses any sort of order. The problem is worse on the high end (shown above). People are handing out 90 points to wines that, according to the attached note, they wouldn’t ever drink again.
Everyone knows Parker’s scale. Most Burgundy lovers know Burghound’s scale. But nobody knows the scale that calicabd00d used to rate his 2007 Monkey Bay Sauvignon Blanc.
The solution: show the median score from a given reviewer next to the score he gave. If I stamp a 93 on a wine, and my median score is an 84, then a reader of that note knows that I really liked the wine. Likewise, a 75 isn’t so damaging to a wine’s ego if that reviewer regularly clocks scores in the min-seventies. A statistics bonanza follows, with a singe review now being able to carry
- the note (psh, who even reads these anymore?)
- the score
- the reviewer’s median score
- the difference (score-median)
- the standard deviation from this reviewer’s mean score
- NERDS!
Factor these things into price per bottle, and we’ll need to get time slices on one of those super computers that IBM makes for the Pentagon. All this just to distill a wine into a meaningful two-digit rating.
February 13th, 2009 at 4:51 am
Wasn’t that my idea?
I’m on board, though. And I do try to give some indication in the note (to be read or not) what my score really means.
February 13th, 2009 at 5:29 am
I think you may be missing something here…
Basically, once people have spent money on something, they want to justify their spending, and feel like smart consumers. So, this has a two-fold impact to scoring.
1) If a wine is bad, they are less likely to post. They don’t want to sound like a dumbass who spent $80 on a wine that tastes like it was $8.
2) For wine that is actually good for the price, they want to post to show how savvy they are. “Look, I found this really yummy wine, and I want you to know about it”
This phenomenon has been discovered by many online retailers (like Amazon.com)as well, which is one of the reasons they allow you to go rate products, both good and bad. People are just more likely to be positive. Obviously, some folks go the other direction (Jon), but most don’t. And, Amazon also allows reviews because they know it comforts people into making a purchase without knowing that much… “Hey, if 99 people rated it 5 stars, it’s got to be good!!”.
February 13th, 2009 at 6:11 am
@Jon “Wasn’t that my idea?”
May have been. If so, consider it stolen!
@Jamie “People are just more likely to be positive.”
I’ve always believed the opposite. While there are plenty of positive reviews on Amazon, people will usually only go out of their way to bitch about something.
With regards to wine, people really enjoy slamming an expensive wine. It’s something like street cred to say that Opus One sucks.
“For wine that is actually good for the price, they want to post to show how savvy they are. “Look, I found this really yummy wine, and I want you to know about it””
Though this is certainly true too.
February 13th, 2009 at 7:43 am
Very well written post.
I also agree with Jamie. I experienced this phenomenon when I was a consumer electronics reviewer. People would go on and on about their receiver or speakers and I would listen to them and I could tell that they were basically looking for me to justify their purchase. People tend to lose all objectivity when it comes to something they are passionate about – especially if they spend alot of money on it. Bottom line is you have to take all of this stuff with a grain of salt and use it as a guideline, not as an absolute. If 9 out of 10 reviewers gave a wine sub 70 scores than the wine more than likely blows. But if the ratings are all over the place then who knows? It could be great, it could be plonk.
Having said all that, I also agree with Gary in that people (bloggers in particular) love to slam expensive wine (although, I too think Opus One is overpriced and not that great).
So…whom to trust? How about yourself? I say trust your own palete and use the professional critics as a guide to steer you towards flavor profiles that most appeal to you.