Recently, several prominent wine writers argued on Twitter in a contentious back-and-forth with me and others that blind tasting was bad. It's tasting without context, they said. I am not setting up a straw man here. Here are some of their actual tweets:
"Why should wine routinely be tasted blind, devoid of context or perspective? Why deprive those who would judge it of that information?" contended Bruce Schoenfeld, who writes a wine column for Travel + Leisure magazine.
"I question whether blind tasting … can uncover the most compelling and virtuous wines," read another comment from Jon Bonné, wine editor of the San Francisco Chronicle.
Virtuous? Oh my, is that the top of a slippery slope.
I learned the value of blind tasting early in my wine-drinking life. In the 1970s I lived in Miami and participated in a wine group that included some serious collectors and an important retailer. From time to time we each brought a mystery bottle from our cellars and poured it into a decanter out of sight of the others.
Although we tasted blind, several members knew that my wine was usually from California, and would dismiss it (even when it was a BV Private Reserve; I did not cheap out with this crowd). One day, the host suggested we secretly switch decanters. My Robert Mondavi Cabernet Sauvignon Reserve 1968, one of California's early classics, went into his decanter, his Châteaux Margaux into mine. Thinking his wine had to be one of his Bordeaux first-growths, they all swooned over it, and dismissed "my" Château Margaux. When the identities were revealed, the condescenders of California decided the Mondavi was lacking and the Margaux was fantastic.
This story echoes in my mind whenever anyone questions the necessity of blind tasting in making judgments about wine. But I wanted to understand what this new way of thinking really was about, so, since we live in the same town, I bought Bonné a cup of coffee and asked him to explain his views.
"I do a better job as a critic when I seek out newsworthy wines," Bonné said, "the ones that make smart decisions about their cultural values, farming values and esthetic values." Bonné cited Skinner winery's vivid Grenache from the Sierra Foothills as a wine that might be overlooked in a blind tasting. And Cabernet Sauvignons such as those from Stony Hill or Corison can get swamped if surrounded by more opulent styles.
A valid point; it's long been known that bigger, more muscular wines can overshadow the charms of more delicate samples when tasted side-by-side (blind or not). Experienced tasters know that. So when I come to a wine that seems relatively insipid, I don't just blow it off. I take a moment to let my palate refresh and go back to it a few minutes later. With patience, the good ones show what they have, if they have it.
However, I take issue with the notion that one can't discover worthy wines in blind tastings. I've lost count of the wines that showed something special in a blind tasting, which I followed up to learn their stories. That's where the context comes in. Is it a new producer, or a familiar producer doing something different? The character of the wine leads to the context, not the other way around.
Another theme is that an art critic, a film critic or a music critic can know the context before they review the actual experience. Should a wine critic be any different? I worry that biases can easily play into a fair evaluation. Symphony orchestras might still be missing the talents of women, Asians, Latinos and African Americans if blind auditions (now standard) had not proved they sounded just as good when everyone tries out behind a screen.
My stance is simple. If we know—or even suspect—the identity of the wine we're tasting, it's too easy to find exactly what we expect, not what's really there. Think of those collectors who believed California wine to be inferior. And I'm not alone in this conclusion. There have been many scientific studies that demonstrate the power of expectation to influence judgment.
Today, however, the position is flipped. Bonné is writing a book about the "new California," wines that play against prevailing styles of the recent past. But if you are convinced that off-the-beaten-path wineries are more interesting than the mainstream, isn't that what you are bound to find if you don't taste blind?
And you might be right, but to prove it I am convinced you must take away the cues. Find the evidence when tasting blind with competing wines. Either the wine in the glass really reflects its "virtue" or it doesn't. Then you can make an editorial judgment of which wines to highlight for readers.
In our Twitter conversation, the travel writer argued that we can't judge a wine fairly if we don't know what the winemaker was aiming for. But as far as I am concerned, if we cannot sense the winemaker's intent in the glass, then you're arguing that it doesn't matter how it really tastes.
I do not always taste blind. Experiencing wines at lunch or dinner, at trade tastings, in winery cellars or in interviews with winemakers provides context. Tasting the delicate structure and finesse of Adelsheim Pinot Noirs with winemaker Dave Paige, studying the vineyard sources and what they bring to the wine, sensitized me to those characteristics when I encounter his wines, or similar styles, in my blind tastings. Having tasted Washington's Red Mountain reds on other occasions, I can adjust to their tannic grip when I encounter it.
Although reviews you see in the New Releases section of Wine Spectator were tasted blind, the tasting note is not without context. Once the score and characteristics are determined and the identity revealed, I can add a phrase that compares this vintage to previous ones, or include a key piece of germaine information.
Mostly, however, I let the note stand on its own, and pursue the context for where it belongs: in stories about the region, the winery, the winemaker or the wine type. Then the context reflects what's in the glass, rather than the mind of the taster.
Readers value critics who can lead them to wines they want to drink. We offer our opinions, and if you agree with the assessment when you try the wine yourself, you might try another recommendation. If this happens often enough it builds trust. The only way I can build trust is to taste honestly and critically, and to do that I need to eliminate as much bias as I can. That means tasting blind when reviewing.
To further illustrate how subtle cues and beliefs can affect our judgments, I offer the following story. It is 100 percent true. I have the column I wrote at the time to prove it.
The late restaurateur Vic Bergeron, the original Trader Vic, was a showman of the first magnitude. One day in the early 1980s he invited every wine writer in Northern California to come to a tasting. He had made a discovery that would revolutionize wine aging. Storing wine under a pyramid, he told us, accelerates the aging process and makes any wine better in months instead of years. To prove it he offered us examples of the same wine aged under the pyramid and in his regular cellar.
A line of waiters appeared each holding two bottles of the same wine, identical except that half the bottles had gold seals on them. The waiters poured with great fanfare. The writers almost unanimously preferred the wine with the gold seal. No prizes if you can guess which one had been aged under the pyramid.
This was pretense, not blind tasting. First Vic set up an expectation that there would be a difference between the two bottles. Then he accentuated it by changing the appearance of one of the bottles. The subliminal meaning of a gold seal isn't hard to grasp.
On the way back to the office I bought two bottles of the same wine and affixed a gold seal to one of them. I set up my own tasting, collaring colleagues at random to taste them and tell me which one they preferred. It was almost unanimous. They found the gold seal bottle clearly better.
The lesson to this parable is about expectations. Even when we aren't aware of it, anything that can suggest one wine is better than another in a tasting will bias our judgment at some level, "virtuous" winemakers included. That's what matters.
Sandy Fitzgerald
Whitesville, KY — March 5, 2013 3:40pm ETGreat Post, I truly believe that blind tasting is the way to go to unjudgementally evaluate a wine. My only problem with the scoring methodology is when one has, for example, a Loring Pinot next to a feminine more silky pinot in a line up. Often, the majority of the people hate the one and love the other or vice versa. IMO blind tastings are more revealing when all the wines in the blind flight are stylistic similar.
Harvey Steiman
San Francisco — March 5, 2013 4:38pm ETBruce Schoenfeld
Boulder, Colorado — March 5, 2013 6:22pm ETI never said that tasting blind was bad. For scoring wines, it's pretty much necessary.
What I questioned was the value of thinking about wines in a competitive context. Different wines serve different purposes, and we're drawn to them for different reasons. We don't always want the "best" wine -- even if we could pin down what such a thing meant. We want wines that stimulate us or comfort us or make us nostalgic, perhaps, depending on our mood. Wines that challenge us with one meal, but maybe a wine that's right in our strike zone with the next.
My point was that tasting blind robs a writer of the chance to think about wines in this way, which happens to be the way that most people interact with wine. That's all.
What Jon's point was, I don't know. But I do believe I started the discussion.
In the further interest of accuracy, I'm Wine + Spirits editor of T+L, but I don't write a wine column for the magazine.
Harvey Steiman
San Francisco — March 5, 2013 6:47pm ETIn didn't come off that way in your Feb 28 twitter posts in which you refer to "the faux science ...of blind tasting," that blind tastings "strip wine of context that make it far more interesting," and "I just don't understand why it's fairer to judge something in a vacuum than with the context that makes it interesting."
John B Vlahos
Cupertino, California — March 5, 2013 7:06pm ETHoward G Goldberg
New York, N.Y. — March 5, 2013 8:24pm ETa boring, white-walled room. No additional stimulus to boost scores."
Bruce Schoenfeld
Boulder, Colorado — March 5, 2013 9:32pm ETMy discussion was about the utility of thinking about wine that way as opposed to opening a bottle (with the label in full view), drinking it with a bite of food, and listening to what it has to say.
That's the danger of quoting Tweets. Go back and look at the full trail and you'll see what I mean.
Harvey Steiman
San Francisco — March 5, 2013 10:02pm ETBut to be clear, as Howard underlines, the wine in the glass comes first. Then I put it in context. In turn, the experience with context helps to fine-tune what characteristics I might seek in future blind tasting. And only wines that I blind taste become reviews.
Harvey Steiman
San Francisco — March 5, 2013 10:10pm ETPacific Rim Winemakers
Portland OR — March 6, 2013 3:59pm ETNicolas
And I wonder how you got Jon to let you buy a coffee, he never lets me buy....
Harvey Steiman
San Francisco — March 6, 2013 4:11pm ETLouis Shenk
Metairie, LA — March 6, 2013 5:06pm ETMost wine drinkers don't necessarily drink and judge wine the same way. How the wine pairs with food, or what mood the drinker is in - does he want a big wine, or a subtle wine that does't call too much attention to itself? I'm always surprized when a wine I liked at a restaurant doesn't impress me at home, in a different context. But it shouldn't surprize me, should it?
Greg Rush
Somewhere between Napa/Sonoma/Tuscany — March 6, 2013 5:38pm ETStewart Lancaster
beaver, pa — March 8, 2013 2:44pm ETJordan Harris
Leesburg, Virginia — March 9, 2013 10:22am ETYou mention above your experience with a Mondavi vs. Margaux. This would not have shown to be truly blind somewhere like Wine Spectator either since you would have known it was at least Bordeaux vs. Napa (or would have been thought of as California back then). Today these biases still take place with regions like Virginia, New Mexico, and Colorado or even old world appellations like Minervois, Bergerac, Lebanon, and so on. These regions are know before being tasted in most "blind" situations which is going to create a clear bias from the start.
It would be great to see the editors get shuffled around every once in a while to have to take on a different beat in order to take personal preference out of the equation for regional differences. It would also be cool to see a few issues where maybe the wines were rated by variety by editors instead or region so you might taste aaa Cabernet from Napa, St. Julien, Virginia and Stellenbosch all together to see true qualitative difference from place to place instead of having regional bias. The problem there is that there are then varietal biases. Are perfectly made Gamay or Viognier not as spectacular to some as top notch Syrah or Chardonnay. It certainly doesn't show in scores from "blind" tasting because variety is known and creates a bias.
Obviously this all creates a logistic nightmare and will never be done, but it is something to think about. What is the difference between the accepted "blind" tasting methods and what Bruce and Jon mention. The "blind" technique takes producer and precise site out of the equation so a wine is accessed taking regional bias or variety bias into context. The none blind takes possible production techniques, and terroir into context. A wine may be judged differently because it comes from To Kalon or Mazi-Chambertin or had extended maceration, rather then because it is from Napa or Burgundy vs some thought to be lesser area. Are either really blind and void of bias, absolutely not.
I re-iterate that double blind and checks and balances would create a fantastic service to readers and wine drinkers. However, current neither the current accepted "blind" tastings or non blind do this. Which is better of the current potential methods, I don't know.
Harvey Steiman
San Francisco — March 9, 2013 1:03pm ETSo in our regular tastings we focus on eliminating the biggest biases, involving producers and specific sites. Within any category, you know we did not favor any particular winery. (In my Margaux vs. Cab example, the bias being addressed was whether the wine I brought was being judged correctly, even in a double blind tasting.)
Also, in my tastings I never know when a ringer (a wine from another region or one previous reviewed) might show up unannounced to keep me honest. This happens on a regular basis. Also, when the whole roster of senior tasters gets together, we usually taste a flight of wines blind and compare notes. And our process for selecting the Wine of the Year is also done double-blind, Pinot Noirs, Barolos and Cabernets on the same table unidentified by producer, grape or region.
Don Rauba
Schaumburg, IL — March 9, 2013 1:21pm ETIt strikes me that the only cause to defend "(single) blind" tastings is to make sure various legendary appellations are never outscored, which is the cynical way of saying it. From another angle, incorporating more in terms of double blind would help keep everything on a level playing field. Is bitterness really a detractor for all wines in all regions? I have reason to doubt it. Maybe that's why I find the year-end top 100 list more interesting because it seems to accomplish a kind of overarching comparison in a way that individual scores don't always succeed in doing.
Harvey Steiman
San Francisco — March 9, 2013 1:41pm ET1. The foundation of the Top 100 is how the score, the price and the character of the wine combine to make it stand out among others. Tasting blind, we know that we found the quality and character without knowing who made it or what the price was when we tasted it.
2. Even within "legendary" appellations, there are duds. If a narrow appellation would identify the wine, we taste the wine under the broader appellation to eliminate that bias. Also, some great wines in the New World carry the same broad appellations that cheap wines do. Grange, for example, is labeled South Australia, same as many $10 wines.
Thomas Matthews
New York — March 11, 2013 8:50am ETDeciding what kinds of information about a wine to disclose, and what kinds to hide, is a complex decision that does not have a perfect answer. There are costs and benefits no matter where on the scale you decide to draw the line.
Wine Spectator believes, as Harvey states, that the most powerful, and pernicious, impact towards bias comes from two key pieces of information: producer and price. We feel it is essential to fairness and objectivity not to reveal these facts about any wine in our tastings.
It is possible, as Jordan argues, that regional bias can also influence judgment. However, we believe that an important aspect of a wine's quality is its ability to reflect, and convey, its origin and vintage (and, for New World regions, its varietal identity). After nearly three decades of experiment and adjustment, we believe that the costs associated with possible regional bias are lower than the benefits associated with testing a wine's character against these external realities, which might be summed up as its "terroir."
So our goal is to eliminate, or at least reduce as far as possible, regional bias. How? As Harvey states: by using ringers (both from within and beyond the region), to establish consistency; by tasting wines from other regions, both blind and non-blind, to establish context; by tasting with other editors, to calibrate our palates. It may not be a perfect solution, but we think it delivers the best chance for a true reading of every wine's character.
Thomas Matthews
Executive editor
Jon Bonne
San Francisco, CA — March 11, 2013 6:34pm ETOne thing re: Grenache. While I'm sure I noted the Skinner as a standout (and perhaps a subtle wine to try and interpret when tasted amid flashier Grenaches) I think the Foothills wine I might have been more concerned about tasting blind was the La Clarine Mourvedre. A fine (if controversial) wine, but one that's hard to place in context when considered blind.
Kyle Schlachter
Colorado — March 12, 2013 12:23am ETI agree that knowing producer and price introduce bias. However, you understate the impact of regional bias; it is real and it has been reported many times in scientific literature. it is perhaps the most important piece of information that influences consumers (yes, WS editors have more experience that "regular people"). But even professionals expect to an Cote-Rotie to be "better" than a Colorado syrah. Perhaps old world v. new world would be enough information instead of knowing Rhone v. Barossa v. CA syrah. Even within each of those regions there are a variety of styles, so does region really add more than it hinders (the wine that is...)? As for the ringers, are the scores of ringers used in the publication? If a VA cab were in a CA cab flight would the score from that tasting make it into the database or would the VA cab's score come from the VA cab flight? BTW, great blog post.
Don Rauba
Schaumburg, IL — March 14, 2013 1:19pm ETTom & Harvey, thank you both for providing such an open forum for discussion. While many of us may harbor misconceptions, we appreciate your patience to discuss/explain/engage in intellectual debates. And we subscribe because we believe there is plenty of merit to your ratings (relatively unbiased). I wouldn't pay to subscribe to a non-blind taster.
I appreciate the issues of complexity (what Harvey described as palate adjusting) that double blind tasting would require. I acknowledge Jim Laube's points about costs as well. But I still maintain that something more could be done to mix up the comparisons a bit more in order to remove the possibility of defending or favoring what we all know are prized terroirs/regions.
For example, why not comingle Washington, Argentinian, Chilean, California (Meritage) and Bordeaux reds e.g. to highlight the differences and delve directly into the regional distinctions tasted/observed (rather than "expected")? Do we really get that benefit when the samples are so neatly segregated? What about Oregon, New Zealand, and Burgundian Pinot Noirs together, etc.?
Now, I don't know that it's practical for this to be an "all the time" practice (or even a good idea at all), but what about starting by trying it twice a year? Seems to me it would teach us about the preconceptions we harbor about the various regions (are they even valid in today's climate of evolving styles?) and re-establish the relevant comparison points for today's global consumer. As difficult or unwieldy as it might be, I think it would better inform the scoring, make the reviews even more insightful, and give consumers a more meaningful picture for all wines. Has it been tried already? Would love to hear your comments.
Harvey Steiman
San Francisco — March 14, 2013 1:43pm ETWe have done some things like that in the past. I recall a 1998 three-way blind tasting of Burgundy, California and Oregon, in which Bruce Sanderson, Per-Henrik Mansson and I analyzed the results extensively. And we have staged blind tastings at the Wine Experience where several of us evaluated each other's wines, in public, along with the audience. It's always educational.
Staffan Bjorlin
Los Angeles — March 14, 2013 4:41pm ETEd Frankoski
Huntington, NY — March 14, 2013 10:07pm ETJordan Harris
Leesburg, Virginia — March 15, 2013 8:44am ETThanks for the great discussion and it should be mentioned here that it is great that Wine Spectator is open to forums like this. I am sure you know before publishing that it is going to create some discussion on both sides of issue and your openness to that it to be commended.
As I stated at the start of my first comment, obviously a lot of what I mentioned is simply impossible logistically for someone like W.S. There are also disadvantages to many of the ideas I stated in the grander picture for something like W.S. If editors tasted regularly by variety across many regions then many of the editors wouldn't build as much of their in depth knowledge of regions as they do now which could harm feature articles that are far more then blind tastings and ratiings for example.
As Thomas said, there really is no perfect answer, particularly when you include the fact that Wine Spectator is also a business and has to look at the costs and logistics of such changes. I do think it is great that there is a forum like this that I am sure is used by W.S. to constantly better themselves. Ed makes a great point about knowing your taster, which is very true and necessary for a consumer to get the best recommendation. The tough part is calibrating yourself with a taster can be very time consuming and expensive. Not everyone is about to spend the time comparing their notes with professionals notes in order to the best match.
One of the major reasons I am a great supporter of W.S, is because while you defend your stance and you should, you are open to these discussions and I imagine they are used to contantly learn how to better the publication for your readers. Thanks.
Hugh L Sutherland Jr-m
owens cross road,al 35763 — March 15, 2013 4:41pm ETEd Frankoski
Huntington, NY — March 15, 2013 10:00pm ETRichard Kim
Anaheim, CA, USA — March 18, 2013 6:27pm ETBut that doesn't mean we have to be scientific, necessarily. This isn't medical research, it's about enjoyment. You need to know and trust the reviewer. There are movie critics and restaurant critics with whom I share tastes, and there are those whose tastes differ radically from mine and I shy away from their recommendations.
So I guess each approach has its place.
Thank you Mr. Steiman for your insightful comments and the opportunity for this discussion!
Tim Mc Donald
Napa, CA USA — March 18, 2013 9:39pm ETKathleen
Pennsylvania — March 20, 2013 12:48pm ETJeffrey Matchen
New York, NY — March 25, 2013 11:27am ETFor enjoying wine, though, I prefer to have all the context and biases I can. I like to know the price, how difficult it was to find a bottle, the story about when you met the winemaker, reviewer notes, CellarTracker notes -- anything and everything. I agree all that context is wrong for rating a wine, but I think the experience of wine is so much more than simply what is in the glass. As with other pleasures, this one is mental as well as physical.
K & L Wine Merchants
San Francisco — March 28, 2013 8:34pm ETEvery now and then when my wife and I are looking to add a new 'daily drinker' wine, I buy about 6 different, but similar, bottles in the 10-30 dollar range and invite a few friends (who know wine) over for a tasting. We always do it blind, and even I am unaware of which wine is hiding in each numbered paper bag. We even add a stipulation that there can be no talking in the first round of tasting to avoid inadvertently influencing others. Everyone takes their own notes, and after about 20 minutes of silent sipping the discussion begins. I should add that all the wines in the tasting will be similar, such as all will be sauvignon blancs from NZ, for example. We do occasionally show similar old world/new world offerings side by side.
It’s always interesting to see how notes compare, male vs. female tastes differ, and hopefully there’s a clear consensus. It’s always a treat when a less expensive wine is a hands-down favorite! I usually run out and grab a few cases of the winner and happily drink it.
All that said, I would probably never serve a 1961 Lynch Bages to a friend without disclosing what the wine was first. Anticipation and expectation DO make a wine experience more exhilarating, and in this case especially, perception is reality.
Cheers!
Tristan Stringer
K&L Wines - Old & Rare / Auctions
Harvey Steiman
San Francisco — March 28, 2013 8:51pm ETYour blind-tasting exercise is a great way to keep the price out of the equation when you decide which wines you prefer. Thanks for your thoughts.