Each December, the words that echo across the cigar universe are – “it’s that time of the year.” Specifically, it’s the time of the year when various outlets – both media and non-media related, give their perspective on what were the best cigars of the year. With the increased presence in both online and social media, the amount of outlets doing cigar of the year lists has increased. Online media has grown as the Internet has grown. At the same time, social media has essentially allowed anyone to weigh in on what is the best cigar of the year. It has also given rise to many methods to determine a cigar of the year. Today, we look at several methods used to identify a cigar of the year – and weigh in on the pros and cons.

We have identified five scoring methods that cover the majority of cigar of the year lists. Our focus will be on print and online media outlets.

1. Quantitative Scoring Method

Using this method builds an index that depends on specific quantitative data. This method ranks the cigars by how they scored throughout the year or during a specific period of time. This is a method used by Developing Palates.

Pro

Think of this method as the best “regular” season a cigar has during the cigar review period. It gives a real picture of which cigars had the best level of performance for that time. The cigars are ranked from highest score to lowest score.

Con

Ultimately there are intangible factors that many believe play into a Cigar of the Year list. These things include consistency and distribution. The quantitative method ultimately won’t factor these attributes into the ranking.

2. Subjective Method

This is probably the most popular method that is used by online media. This is a method where the curator/author of the list ranks the cigars based on personal criteria. It also can be done in aggregate by a panel representing the media outlet. Cigar Coop uses this method.

Pro

Unlike the quantitative method, the subjective approach is able to better incorporate intangible factors into the list.

Con

It’s subjective, and as a result, it’s the methodology that could easily have the most personal bias.

3. Quantitative Retail Sales Method

Many retailers like to get involved with doing a cigar of the year list. Some will employ a subjective approach, but some will also use quantitative data based on sales. While some retailers will use the subjective method to compile a list, in this cas,e it is being based on sales.

Pro

Ultimately people are in the cigar business to sell cigars, so it makes sense that the best-selling cigars should be recognized.

Cons

Unless there is a national effort to link retailer numbers, the retail sales method will be localized to a particular shop or group of shops. With the increase in small batch releases, a good number of cigars in a retailer’s humidor are not nationally available. As a result, it is not an accurate measure.

In addition, through discounting,  a retailer can influence the sales volume being generated.

4. General Electoral Method

This method actually combines elements of a quantitative and subjective method. It focuses on a vote done by a panel or the general public. Voters cast a vote subjectively, but ultimately the results are tabulated quantitatively. Tobacco Business Magazine is employing this concept.

Pro

When the general public is polled, one can say it’s cigar enthusiasts deciding and is an accurate barometer to gauge a cigar of the year.

Con

These can often turn into popularity contests. A significant amount of personal bias can be interjected. This might come down to what brand can best get people out to vote – and ultimately the curator of the vote can control what cigars are eligible.

5. Tasting Tournament Method

This involves tasting cigars and using a process to determine the best cigars. To some extent, this uses quantitative elements and subjective elements – although the subjective elements can be eliminated if the cigars are tasted blind (i.e. unbanded) by one or more people. This is the method used by Cigar Aficionado.

Pro

This provides a real-time way to measure cigar performance. If done blind, it eliminates subjectivity.

Con

This is an expensive and resource- intensive proposition for most online media brands. If this is not done blind, ultimately it becomes another exercise with heavy subjectivity.

The Ingredients Make the List

No matter what method you use, there are two ingredients that make for a successful cigar of the year list – a set of evaluation criteria and what cigars are eligible based on that criteria. The more consistent these attributes are, the more credible the list will be.

When it comes to media outlets, there is not a consistent set of evaluation criteria used. Many media outlets will include on their list only cigars they have covered or reviewed over the past calendar year. While on the surface this might seem like a good thing, I’ve found this to be a problem with many of the cigar lists. Charlie Minato, who curates the Halfwheel Consensus, an aggregation of cigar media lists has been on the record as stating the end of the year lists are more of a look at the relationship between cigar companies and cigar media. The more I think about it, the more I think he is correct. When there is a good relationship between cigar company and cigar media, a cigar will get covered. Reviewers are more likely to review cigars by people whom they like.  Eventually, these types of reviews play into what cigars are eligible for one’s end of the year list.

But the relationship between cigar company and cigar media doesn’t just extend to online media. If you look at Cigar Aficionado’s (C.A.) Top 25, often the #1 cigar is from a company that is feeding exclusive or first-breaking news to C.A. at the expense of all other media. In fact, many of the cigars on the Top 25 also fall into this category.

It’s no secret that last year I was quite disappointed with the online media as a whole in terms of the cigars which were included on the lists. Not only did the lists reflect Minato’s theory about relationships of media and companies, but they also didn’t have cigars that were “out of one’s comfort zone.” For example last year, top cigars such as the Diamond Crown Black Diamond and Archetype Sage Advice are cigars that were infrequently reviewed and as a result never got a chance to appear on lists. On the other hand, while companies such as Dunbarton Tobacco and Trust and Foundation Cigar Company made great cigars, it seems like each year there is an effort to include brands like this at the expense of brands that may not have as much of a cult following.

What Should You Look for?

The answer is that you can look at any Cigar of the Year list and enjoy it, but if you are looking for some good ones, here are some things to consider:

1. Look carefully and see if the list has a set of criteria, and how consistent it is. If the media outlet makes a lot of “exceptions”, then one has to question the credibility of the list in terms of why exceptions are being made. For example, an index may choose to rank the top 10 cigars under $10.00, but if an exception is made for a cigar that is $10.50, one has to question why that is being done.
2. While I feel strongly that a list is stronger when the media brand is reviewing cigars, I think producing consistent content is equally as important. If a media site has ‘dead publication” periods or simply doesn’t have a lot of content since the last time it published a list, it’s a red flag.
3. You don’t have to stick to one kind of list, but consider the method when deciding which cigars to smoke. Not all lists are created equal. The Cigar Aficionado list (which uses the tournament approach) is often compared to the online lists (that lean toward the subjective method).
4. Consider some basics: If a review site doesn’t consistently get right some basic things as: get the cigar name correct, get the cigar company correct, identify natural versus maduro, etc, then one must question how thorough and serious a job the review site is doing.
5. Several years ago, Cigar Coop used to rank the cigar line as a whole (considering all sizes), but over the past four years, we have gone to a vitola method. While there is nothing wrong with rating the “line,” ultimately a vitola-based list is a stronger proposition.
6. Is the reviewer or media brand  “buddy-buddy” with the manufacturer?  If that manufacturer appears high or dominates the list, then it is natural to question the objectivity of the list. Social media can tell you a lot here.
7. Is the list dominated by sponsored cigars?  Again, this might cause one to be concerned that the list is biased.
8. If the reviewer or media site is “apologizing” publicly to brands that are not on its list, then most likely the media brand is trying to make everyone happy, and to some extent, it sends a message that the reviewer or media site may lack some confidence.
9. Is there a wide range of brands represented on a list?  If a Top 25 list has only 10 brands versus a Top 25 list with 20 brands, it could be an indicator the media site is not broadening its horizons on coverage.
10. Nothing wrong with a cigar of the year list by a retailer, but it’s important to keep in mind that the goal of many retailers is to sell cigars. In particular, if the retailer is using the Subjective Method, that list may reflect what cigars they are looking to move.

Photo Credits: Cigar Coop