Wednesday, October 16, 2024

Crafting judgement - inside the GABF evaluation process: a guest post by Collin Zreet

Image: Brewers Association.

With another round of the Great American Beer Festival come and gone, there still seems to be a shroud of mystery around the competition process and what it really means to win one of these illustrious awards.

For 2024, there were 8,836 beers and 233 ciders entered into 109 categories (including 170 subcategories) evaluated by 285 expert beer judges. With the addition of judging ciders, this year also saw changes like the creation of three non-alcoholic beer categories (Pale Non-Alcoholic, Amber/Dark Non-Alcoholic, and Specialty Non-Alcoholic), as opposed to one overall non-alcoholic category last year. The largest categories were not surprisingly IPA related: Juicy or Hazy India Pale Ale (349 entries), West Coast IPA (292 entries), and American-Style India Pale Ale (217 entries), with the latter two categories being in one combined category just two years ago. Dortmunder or German-Style Oktoberfest (210 entries) and German-Style Maerzen (195 entries) were also large categories, likely due to the seasonality of the styles being so close to the competition dates.

Unlike homebrew competitions, professional beer competitions are more about hardware and winning than identifying off-flavors and needed process adjustments (though that doesn't mean professional beers are free from off-flavors or other glaring errors). Brewers want to compare their creations with the best across the country and need unbiased qualified judges to provide that feedback. Judges are not chosen at random, nor do they have a bias towards one region of the country over another, as others have speculated.

The judge selection process requires candidates to detail their qualifications in either sensory/QA, professional brewing, extensive beer judging, and/or raw material expertise. Upper-level qualifications from the Cicerone Certification Program and Beer Judge Certification Program (BJCP) help, but are not necessarily required. On top of those qualifications, judges also submit three letters of recommendation from colleagues that are familiar with their judging qualifications. Once all criteria are submitted, there is typically a two to three year waiting period and invitations are usually sent out a few months before the actual judging occurs. Judges are from all over the country and range from brewers to sensory analysts, yeast lab specialists, hop growers, and even non-commercial high ranking BJCP homebrew judges. There are typically 10% new judges each year, though this year there were 33% new judges, likely from bringing in specialized judges for the newly created cider categories.

But how does a beer progress from a competition entry to bringing home some hardware? Once a brewery decides which beers they want to enter into the competition, they have to declare which category they want to submit their beer into and pay a $180 per beer fee to register it. Choosing a category isn’t as simple as one might think. With so many categories, some beers might actually fit in a few different categories and breweries need to decide which is the “best” category to enter into. This might not even be the category the beer is marketed as (see Houston-based Saint Arnold Brewing Co.'s silver medal winning Scottish-style ale, Saint Arnold Oktoberfest). Being able to discern where a beer truly fits is a winning strategy within itself.

After a beer has been registered and then brewed, it then needs to be sent to the Brewers Association in Boulder, Colorado, where it will be held in refrigeration until it is judged. The Texas Craft Brewers Guild has collection points across the state that allow member breweries to have their beer driven via refrigerated truck all the way to allow for the beers to be in the best condition upon arrival. This does lend a slight advantage for Colorado breweries, as they are able to send fresher beer without the worry of longer travel times than other breweries.

The first session of judging typically happens a few weeks before the actual festival in Denver, with a second session occurring just days before the festival. All judging is done double-blind, so judges are not aware of what specific beer or what brewery’s beer they are judging, and they are not even aware after the festival unless they won a medal. There’s a dedicated team of non-judge volunteers pouring beer and another team running beers to the judging tables to avoid any kind of bias. Judges are also not allowed to judge their own beers.

A typical beer flight is between eight and twelve two-ounce samples and may have several rounds of judging based on the size of the category. Each table has six judges, and each pod of three will evaluate the same flight of beers. The first round is the only round that requires judges to provide written feedback to breweries. No longer are there quantitative scores for each beer, but more open-ended qualitative feedback is provided. The first round is mostly to eliminate beers with off-flavors and large stylistic errors, though sometimes good beers do not advance from this round in favor of other better beers. Of each round, only three move on to the following round. Beyond the first round, all other assessments are between the judges and not recorded. These rounds are typically twelve beers each and are served similarly, passing on three to the following round as well. 

The final round is the same, except all six judges at the table are then deciding between the twelve samples set in front of them. This is where the nitpicking really occurs. Imagine out of all the commercial beers in the U.S. for a certain category, you have the best twelve in front of you and not only have to decide the best three, but which receive gold, silver, and bronze medals. Usually no one agrees at first, and you have to systematically discuss each beer and argue against five other highly accredited beer judges why you think ones should or should not stay on the table for medal consideration. The margins are so small between them and sometimes reasoning even gets philosophical on why one aspect of one beer is more stylistically accurate over another. It gets even more complicated when you consider categories like Experimental Beer, Non-Alcoholic Beer, or Wood-Aged Beer when you are also judging across several different base styles that don’t all have the same criteria.

If you look at some of the larger categories that had three rounds of judging, it typically means that each beer that makes it to the final table has been assessed by twelve different judges, and the category itself has been assessed by 126 different judges in total. Even smaller two round categories have beers assessed by nine judges. For any beer to even make it that far, much less even medal, is a wonderful achievement. Participating breweries are only notified if their beer made it out of the first round and if they medaled. So, even the margin between medaling and not medaling (3rd and 4th place) could be the slimmest of margins, but no one would even know the difference, not even the judges. 

On a typical day, judges assess five to six flights per day across three or four days (depending on session). While it might seem like all fun and games to drink beer in Colorado for a few days (and for sure it is at times), the process and end goal is to objectively assess all of the beers (even the bad ones) and determine the best beers in the U.S., which is taken very seriously by all of the judges.

Congratulations to all of the D-FW local and Texas breweries that participated this year and even brought home some hardware. It is a difficult task to even make it out of the first round, much less progress any further. Please continue to support those local breweries that you love. It is still very difficult for all of them out there, and they all could use your support.



Collin Zreet is a former brewery owner (Funky Picnic Brewery & CafĂ©) and one of only eight Advanced Cicerones in the State of Texas. Throughout his experiences in the craft beer industry, he has specialized in sensory and beer quality, judging several professional beer competitions, including the Great American Beer Festival, and being an instrumental part of setting the styles and guidelines for the Texas Craft Brewers Guild’s annual state-wide Texas Craft Brewers Cup. He also specializes in beer and food pairing, creating and leading over 25 beer dinners across the Dallas-Fort Worth Metroplex.

No comments:

Post a Comment