Log In / Join Now

In Defense of Blind Tastings

Photo by: Greg Gorman

Posted: May 26, 2006 12:49pm ET

There’s a thread on another Web site discussing the pros and cons of blind tasting.

A reader sent me a clip from that dialogue citing one comment that blind tasting is “vastly overrated.”

Really?

Maybe the person meant "vastly underappreciated."

Blind tasting has been the wine industry’s standard for evaluating wines for a long time. Perhaps it’s only critics -- many of whom don’t taste blind -- who disagree about this practice.

While blind tasting is hardly perfect, it is the most objective way -- and fairest to producers -- to analyze wine. It's the way we at Wine Spectator review all new releases, and except where otherwise noted, all wines are reviewed blind.

Had this person said that blind tasting isn’t the only way to evaluate wines, and that it has its imperfections, that would have been different.

The discussion also raises some questions about how different reviewers analyze their wines, how much time they spend with them, and whether critics change their scores after the wines are revealed.

I’m happy to talk about my tastings and will from time to time.

Here are some of my practices:

I know the varietal, vintage and appellation of most wines.

Wines are opened about an hour before tasting and one pour is splashed into the sink, to let the wine aerate.

I use a warm-up wine, which was previously reviewed and rated, as a reference point.

I typically taste eight to 10 wines to get my bearings (and I taste with one of my colleagues; they’re there to learn and share their views). I write notes and score the wines, with the bags still on. Once I feel comfortable with my ratings (and that can vary depending on the varietal), I begin to unveil some of the wines.

When I find a wine that’s outstanding, or sensational, or tight, or something I want to reconsider, I leave it bagged. On a good day I might retaste a dozen or more wines, still blind, before knowing their identity.

There are times -- the 2001 Peter Michael Les Pavots springs to mind -- when a wine is so astounding that I taste it six or seven times before finding out its identity. Heck, most of that bottle was gone after all of us tasted it time and again.

I often retaste (blind again) a wine that gets a 98-point rating. Why not? Wines like that don’t come along that often.

The notion that I only spend 15 seconds or three minutes on each wine is also pure conjecture. It depends on the wine. Bad wines are easy to score, and the notes are short. Great wines deserve a more comprehensive description.

On good days (yesterday was a bad Merlot day), I’ll take some of the wines home and drink them with dinner. It gives me another chance to think about what I liked about a wine.

Other times, such as with a dense, backward Petite Sirah, I’ll let it sit, still bagged, overnight and retaste it the next day. I don’t change the score, but might tweak the note to reflect a broader drink window.

There’s nothing wrong, either, with a critic adding a comment in the note after it’s been rated and unveiled.

As I’ve said, I’ve come to appreciate blind tasting because it often renders surprises and forces you to use your sensory skills rather than your ability to read a label.

I’m sure you have some thoughts on this subject.

David Niederauer
May 27, 2006 11:45am ET
Not only is blind tasting a great way to evaluate wines it is also interesting and fun. And comparing tastes contemporariously with peers is the best learning experience one can have.

I would ask you Mr. L if you give wines with unusually low scores the same chance to confirm your score as you do to confirm the high scoring wines.

How many glasses should we get out of one bottle? What do you think the minimum size pour should be?
Gene Keenan
san francisco —  May 27, 2006 1:28pm ET
Hey James,

Thanks for making so transparent your process for judging wines. I think the points being made on "the other board" are as follows:

Anyone who thinks blind tasters are always honest and live or die by the blind evaluation is living in fantasy land. I.e., the adjustments occur frequently and regularly when the taster finds he has blown a wine evaluation and suddenly decides it needs to be rethought...

...Do you really think tasters have successfully evaluated blind all the time massive, tight, big young wines? Yet, oddly, we see very few scores that read "2000 Lafite: 78."...

This last point is interesting in that I have tasted young barolos and they can be difficult to taste. Do you know if JS does blind tastings of Barolos?

Thanks,

Gene K
Maryann Worobiec
Napa, CA —  May 27, 2006 1:48pm ET
I know you've written about it before, but I think it's worth mentioning again that you use a single tasting glass for all varietals. It's the Les Taster "Les Impitoyables" (French for "the merciless" or "the unforgiving") glass.
James Laube
Napa, CA —  May 27, 2006 1:51pm ET
Gene,I find the adjustments argument interesting, since we don't change scores after the bottle is idenfitied, and apparently they are aware of others who do or have done so.Are wine reviews occasionally blown? Yes. But no one (except my staff) knows exactly what the wine I've tasted tastes like (that is it might be different than another bottle). Sadly there is enough bottle variation to merit retastings. I try to deal with this by indicating tasted twice, with consistent notes, or this note was the best of three bottles tasted.All of our WS tastings are done blind, JS's too.The Lafite example is almost perfect, because it is typically very tight and not very flashy. Experienced tasters should be able to evaluate the level of concentration.One more thing. My ratings are for the wine right now. Some others rate wines on where they expect they will be at their peak. I think guessing where a wine might peak is iffy, too.JL
James Laube
Napa, CA —  May 27, 2006 1:58pm ET
Hi David,I try to have every single wine tasted by at least two of our staffers; me and MaryAnn Bovio or Tim Fish or Dan Sogg.Wines that appear to be good samples but get ordinary scores are not routinely retasted, unless we suspect it has been a victim of poor storage or a bad cork.At the end of the day we're aiming for wines we can recommend. So if a wine shows poorly I'm not likely to retaste it.How many glasses out of a bottle? I would imagine 10 or 12. I'll give it a try tonight!JL
Gene Keenan
san francisco —  May 27, 2006 3:19pm ET
James, thank you for the quick and thoughtful response.

gk
Arshavir Kouladjian
Los Angeles, California —  May 27, 2006 5:29pm ET
First of all, I want to Praise you Laube, James on your evaluation of 2003 Cabs (Mid Palate and Stem Finish) and refering to 1998, 2000 is exactly what I did at Cabernet Classic in Oct, 2005, same notes. Only you should take it one step further by 1998 -> BURNT taste wine, 2000 -> Tobacco filled year and state that 2003 has all the fruit and great taste but only Mid Palate skip and dry finish. Also noting that TIME can only tell whether 2003 will fill in the gap in the middle and secondary flavors fulfill the finish. Other than that, an ABSOLUTE PRAISE on genuine notes. ----> On the topic of Blind Tasting, I wanted to add that When you goto Dinner with group (especially fine) there isn't BLIND tasting. Everyone sees the label and that raises the rating immediately before they even try the wine!!!!!!! Some critics understand that and have taken full consideration of the potential of Psychology. I am not knocking your method of rating because it has proven to be outstanding, The label and reputation of the winery has lifted peoples spirits more than rating points. (Although your blind ratings build label name and reputation. hehehhe lots of irony.)
Jason Carey
willow, ny usa —  May 28, 2006 11:10am ET
I think some of the underlying conflict has to do more with the idea of tasting out of the context of wine against food, more than tasting blind,,, I think that people tend to rate non food friendly wines higher when tasted this way. Theese blockbusters tend to be grate by the taste, but after more than one glass or with food they are too much
Eileen Stamp
USA —  May 28, 2006 3:17pm ET
I agree in total with the use of blind tasting to evaluate wines. Excess information, producer, etc. is irrevelant when attempting to objectively determine a wine's quality. However, I would argue that even such clues as vintage, varietal, and appellation are also counterproductive to fair assessment. If you taste blind, but have a set of parameters with which to frame the wine, then the tendency is to look for qualities or characteristics which squarely place that wine with its indicated terroir, grape variety and year. I think this may perhaps exclude outstanding wines which may be atypical from being scored as outstanding. For instance, if white Burgundies from 2003 are being tasted and scored, are they scored with the heat of 2003 as a frame of reference? In other words, would a ripe 2003 Chablis Vaudesir with plenty of vibrant tropical character (I'm thinking Christian Moreau here) score as well as it did if it were blindly tasted against a more austere 2004, which may be a little more true to the character of Chablis? Thanks for listening.
Robert Gott
Doral/Florida —  May 29, 2006 12:53pm ET
James, Thank you so much for giving details on your tasting method. To me blind tasting is the most effective way to rate wines objectively. One question I have is that if you are tasting several wines in a relatively short period of time how do you clear your palate? I wouldn't think water would do much after many wines and any food would affect the taste of the wine.
Kelly Sickler
Syracuse, NY —  May 29, 2006 1:05pm ET
Blind tastings are key. We all give more credit when we see the label, and think- "this must be good", even if it is not. I also have experienced the metamorphesis that wine undergoes with simple food pairings. I would LOVE to see a simple food suggestion with some of the Winespectator Selection wines in every issue. Especially wines that were very highly scored- to make the most of them. I recall having a few wines in my time that tasted blah, because I made poor food pairing choices. A good wine can become great with the right match, as easily as a great wine can be ruined with the wrong one.Thank you.Jonas
James Laube
Napa, CA —  May 29, 2006 1:33pm ET
Robert,I don't do anything special between wines, except pause for a moment or two. If I'm tasting a really great, rich and extracted wine, I'll make sure to remember the next wine is following a tough act. What's more, keeping the wines in bag allows me to go back through them after an hour or so, and that typically reinforces my initial impressions.After reviewing more than 100,000 wines for WS, it's easy (most of the time) to evaluate a wine and I can usually tell whether it's in the strike zone (outstanding) or lacking something that might make it outstanding.
James Laube
Napa, CA —  May 29, 2006 1:35pm ET
Kelly, I'll pass your suggestion along. We do address food and wine pairings from time to time and with our Food X issue last September there were some food/wine/cheese/fish matchups recommended.
Willim Tisherman
Katonah, NY —  May 29, 2006 3:56pm ET
Everything you say in support of blind tasting is fine and true. The problem is that blind tasting with the purpose of SCORING is a travesty. No matter how honest and earnest you are, the review is inextricably tied to tasting THAT wine, among THOSE samples, in THAT ORDER on THAT day (and WITHOUT FOOD, but that's another story). The result is a number (yes, and a tasting note), but unless you and all the other WS editors are capable of replicating your scores and notes with precision, the review you offer will always and only be representative of one opinoin of one person on one day. The number becomes a tattoo -- immutable. This indelibility makes the whole notion of scoring wines a false pursuit of truth. The real truth is that even the same wine somes up differently to the same person wehn opened in a different context. It is truly sad that Wine Spectator is so caught up in the power trip of rating wines that they seem completely unwilling to accept this.
Mark J Frost
Mercer Island, WA. —  May 29, 2006 6:37pm ET
My experience of a hundred plus blind tastings over the years shows a far more honest rating and evaluation of the wines. Inevitably, once the taster knows the wine, their bias and previous perceptions cloud their judgement.I particularly enjoy the francophiles rate California wines high and their French couterparts low. The after the fact excuses would make a politician blush.Will they change? No, but blind tastings keep us all honest.
Anthony Clapcich
May 30, 2006 8:06am ET
Willim, scoring wines is perhaps like trying to score a piece of art-- it is very subjective and contextual. However, the Wine Spectator is not on a "power trip", nor does the editorial staff ever profess that wines exists only for the "science" of scoring. I do believe that wine scores can have negative effects on the way producers make, market, and sell their wines, but scoring can also help the average consumer wade through the ocean of wine that is out there. What the folks at WS, Robert Parker, Tanzer, et al. do provide is a rough guide. These people have dedicated their lives to tasting wine. After tasting hundreds of thousands of mediocre wines, they are perhaps in the best position to point out a gem or two. Each human being's palate provides a unique interpretation of a particular wine, and therefore, a wine expert's tasting notes are almost irrelevant. Instead, what we benefit from is an expert's collective experience. It is the consumer's job to then taste a particular wine and see if their tastes/palate align with the expert's. I don't always agree with the editors of WS, but they do a decent job...they are more like an old fashioned compass than a satellite computer driven GPS system!
James Laube
Napa, CA —  May 30, 2006 12:55pm ET
Anthony, I wish everyone appreciated the role of the critic as well as you do. Use the ratings as a guide but trust your own taste. When I go into a wine shop, faced with thousands of wines (many of which are new to me) I will often use the shelf talkers as a guide. Better to have had someone recommend a wine that take a blank stab at a foreign wine.

Too many people are slaves to ratings. Are they the most insecure in trusting their tastebuds?
David Niederauer
May 30, 2006 4:56pm ET


One of the three biggest lies said by oenophiles is, "I don't pay any attention to how many points (insert wine reviewer's name) gives a wine".

"Poured yesterday at 10:12am, after 285 days in the bottle which has been stored at 54'F at 77'H, has been open for 17.45 minutes in a 68' indoor room with no windows, poured into a Framstad Bordeaux #62 glass, with the previous wine tasted 2.33 mintus before and it was a 1990 Ch. Schmoltz... this wine tasted like... crap!". Is that what we want?

Blind tasting is a much better way for an oenophile to evaluate different wines than reading a magazine. And one HUGE point IMO is hardly even mentioned here. It has to do with the most important reason we drink any wine. I describe it as "FUN". And really I believe that a blind tasting with wine peers... well, it just doesn't get any funner!
Tim Sylvester
Santa Monica, CA —  May 30, 2006 7:22pm ET
James--Couple of related questions, do you get palate fatigue after a certain number of tastes? How about when drinking multiple wines at a dinner party or a BBQ, for that matter? Do you "reserve" your palate for special tastings (for pleasure, not for WS) or for special bottles?Thanks, TFS
Willim Tisherman
Katonah, NY —  May 30, 2006 9:46pm ET
Please don't mistake my comments for condemning blind tasting. Blind tasting is certainly FUN, and it is always the most certain way to ensure tasters' objectivity. But the reality I emphasized - that blind tasting is inescapably situational aand impossible to replicate - is lost in the sauce when ratings from single critics' blind tastings become essentially permanent, immutable and recycled like gospel.

The argument that points are "rough guides" is absurd; puffs, stars or letters would be true rough guides. The 100-point-scale implies precision; and no matter how precise WS editors are when they grade a wine, there is no way they can replicated those point grades, so why use the scale to begin with? I am willing to accept the notion that WS editors individually are not on a power trip, but the magazine no doubt derives tremendous clout from their ratings. The numbers are warping the wine market; they have outlived theri utility. I firmly believe we would all be enjoying a more rounded and mature wine experience if articles and shelves alike were not polluted with numbers, which grow more an more numbing and meaningless with every vintage.

Personally, I find it shocking that any wine lover -- let alone a professional wine critic --would consult a shelf-taker (with or without a rating) as a guide over a person. If you're in a wine store and can't ask a person about a wine, then why bother to go into that store at all? Let's put some more value in the human element, yes? Good retailers know their stuff, and are more capable than most wine writers I know at communicating about wines... without having to rely on numbers.
James Laube
Napa, CA —  May 31, 2006 12:44pm ET
Tim, I rarely get palate fatigue because I don't try to overdo it and taste too many wines. On a typical tasting day I'll either taste 15 to 20 wines in the morning and or a similar number in the afternoon. Reds in the morning; whites in the p.m.On other days I'll taste 15 to 20 wines and call it a day.
James Buckley
ny —  May 31, 2006 1:59pm ET
wine reviews are just single moments in time,we as a people have long been raised on numerical ratings..whether good or bad.i've had kids in my grammar school class who got nothing but 95 to 100% on every exam.i got mostly 80 to 85%.i'm doing as well or better than most of them;so much for numerical ratings..gees i was just thinking,maybe they didn't age that well.
Jamie Kutch
San Francisco, CA —  May 31, 2006 4:29pm ET
Blind tasting, "vastly overrated."? I think not!

I have been humbled so many times tastings blind. For me, blind tasting means placing a bottle of never before tasted Pinot Noir up against the likes of Marcassin, Kistler, & Williams Selyem and having a perfectly level playing field. It means no influence from someone who is a friend or an enemy. It means all past vintages and tastes someone has had are now meaningless. Blind tasting doesn't care about past favorites, industry movers, deep pockets, reputations, vineyard sources, cost, etc. It gives the same opportunity to all. If a winery has been in a family for 100 years or it's a winemakers first vintage, let the best taste win. When we taste blind we remove a very powerful sense, the sense of sight. Suddenly everything looks the same. Here's the topper... Besides sight we remove something even bigger. Something so large often even our subconscious can't be trusted. We remove emotion. Emotion shouldn't be apart of critical tasting in the first place. So what are that, what are we left with? Our sense of taste. Now isn't that what we are trying to determine in the first place?

Jamie Kutch
Winemaker
David Williams
Carlsbad, CA —  June 1, 2006 12:22pm ET
I am confused about one aspect of your blind tastings--Producer specific articles. You just came out with the ratings for all seven of the Kosta Brown offerings. When you were presented the seven bottles did you not know who they were from or why you were tasting these seven? Did you only know they were Pinot from Sonoma? Did you not suspect the producer?
James Laube
Napa, CA —  June 1, 2006 12:33pm ET
David, the Kosta Browne tastings (like all new releases) are blind. These seven wines were tasted in flights of Pinots from California. Did I suspect the wines were KB? I recall thinking that one of the wines might be the Kanzler (Tim Fish is my witness), but in general I remember being astounded by the quality of the bagged wines (and some other producers that were tasted that day as well) and did suspect it might be the same producer. I try not to guess who the producer might be and just focus on the wine in front of me. We do know the vintage date, varietal and appellation.
James Laube
Napa, CA —  June 1, 2006 12:36pm ET
David, the Tasting Highlights are intended to pull together multiple wines made by the same producer to provide a focused overview of a given winery.
David Williams
Carlsbad, CA —  June 1, 2006 11:55pm ET
James, thank you for your response and clarification. I got a couple of bottles of the 2003 Sonoma Coast--which I'm holding. I then got on their mailing list and was only offered 2 bottles each of Sonoma Coast and the Russian River. Took the offer. Looks like I made out.

Would you like to comment? Want to join or start a discussion?

Become a WineSpectator.com member and you can!
To protect the quality of our conversations, only members may submit comments. Member benefits include access to more than 315,000 reviews in our Wine Ratings Search; a first look at ratings in our Insider, Advance and Tasting Highlights; Value Wines; the Personal Wine List/My Cellar tool, hundreds of wine-friendly recipes and more.

WineRatings+ app: Download now for 365,000+ ratings.