Centerlining in Cupping - A Follow Up

Previously I wrote about an article that I wrote on applying a manufacturing practice known as centerlining to the problem of deciding how to cup coffee.

One of the projects that I've been working on lately is a video series on different methods for evaluating coffee. The article was adapted from some of the material in the first episode of that series. This ties into my work on Typica as a way of forcing me to consider the different evaluative methods that I've found useful and ensuring that a future version of Typica can capture and allow meaningful analysis of the data collected across the product lifecycle from sourcing green coffees through finished product quality assurance as well as supporting experiments on process improvements.

There are two central points to this series. The first is that when you're evaluating a coffee, you're doing that because you have a question. Which of these coffees do I want to buy? How much am I willing to pay for these coffees? How much of this coffee can I sell in a reasonable amount of time? How do I want to roast this coffee? Am I roasting this coffee consistently from batch to batch? What characteristics of this coffee do my customers like and dislike? These are just a few of the more common questions that you might have in mind when evaluating a coffee and it should be clear that no single evaluative method will be appropriate for answering all of these questions, so it's important to be aware of different methods and choose tests that can produce data relevant to the questions you're trying to answer.

The second central point is that in developing any process, there are decisions that can be made, and the results of that process can depend greatly on what those decisions are. If you test different options, you can make those decisions deliberately in a way that takes advantage of the tools and skills you have available and makes it easier to get more useful information from your tests.

Before going into the different testing methods available, however, it's good to have a solid starting point. This is why I decided to start with centerlining. We're fortunate to have an excellent starting point in the form of SCAA and CQI standards, but some of the setup parameters are standardized as falling into a range or rely on equipment that smaller companies might not have access to. Centerlining provides a way to look at these different parameters and document a procedure that works with the equipment you have and allows you to optimize your process to get the most useful information you can out of your tests. It also doesn't have to rely on anything that you don't already have, making it especially applicable to companies that don't necessarily have the budget to assemble a complete coffee lab. When moving on to different testing methods, many of these are simple variations on cupping. Having an excellent starting point allows you to focus the changes in these other tests to only those needed to get information relevant to the question at hand.

Now that the article is no longer on the front page of Daily Coffee News, I thought that it might be interesting to see the original unedited draft that I sent.

Extraction and the mechanics of cupping are greatly impacted by grind.

Centerlining Your Cupping Protocol

Cupping is a widely used practice in the coffee industry and while good standards for cupping exist, few companies put in the effort to rigorously determine what cupping practices will work best for their needs or even verify that their practices match standards they claim to use. Centerlining your cupping protocol allows your company to be more consistent in sample evaluation while also making it easier to distinguish among similar coffees.

First, it's important to understand why the standards for cupping are what they are. There are mechanical considerations. If the crust breaks on its own before the cupper gets to it, it's hard to reliably judge the aroma of the coffee. The goals of cupping also play a role in determining standards. For example, darker roasts can mask defects that you would like to detect while excessively light coffees also make it difficult to discriminate on quality. Many of the variables in cupping are standardized to a broad range. This is in part because of the huge range of equipment used by different companies in the industry with different capabilities and people with different skill in roasting samples, but also because anywhere in those ranges is good enough for basic communication through the supply chain.

While the people who write standards intended to be used across an industry need to keep those standards realistic if there is any hope of those standards being adopted, individual companies are often in a position to set tighter process envelopes that are specific to their equipment and personnel skills. Doing this deliberately allows the creation of a standard that maximizes the chance that observed differences are the result of the coffee being different and not the result of variations in preparation. Documenting that internal standard makes it easier to ensure that the standard is being used consistently by everybody in the company.

To centerline a cupping protocol, start by identifying the variables. How is the coffee roasted? What is the coffee to water ratio used? How much time is allowed between roasting, grinding, and cupping? How is the coffee ground? What water source is used? How hot is the water? How much time is allowed between pouring water on the coffee and breaking the crust? For each of these questions, it is possible to devise an experiment in which coffees are prepared identically except for a variation on a single variable. I recommend repeating these with multiple coffees. Some coffees should be chosen to represent the range of coffees you are most interested in evaluating while others should be chosen because they are similar. This allows you to evaluate how easy it is to distinguish among similar coffees in each test while ensuring that the standard set is reasonably applicable across the range of coffees you're likely to evaluate.

When designing your experiments, make sure that you're testing things that are easily repeatable. For example, there are many sample roasters that people are using successfully every day that have no meaningful instrumentation. Can you reliably hit certain development milestones in a tight range despite this? Maybe, but if not, it's fine to skip the experiments on the roasting portion of the standard. There are, however, several roasters currently available that do have adequate instrumentation and controls that are good enough to standardize a roast profile for cupping. If you have that capability, there is no good reason not to use it. If you don't have that capability, it makes no sense to develop a roasting standard that assumes you do.

As you conduct these experiments, it's important to document the results. This can make your standard more relevant to the tools you have available. For example, the SCAA Cupping Protocol defines the grind as one in which 70-75% of the particles pass through a 20 mesh sieve. Nobody has a grinder with a setting marked in that way and many places that cup coffee also don't have a sieve set. By testing how your cupping sessions work with different grind settings, you can determine what setting works best for that grinder and document it in the same terms as the settings are indicated on that grinder. You're more likely to get everybody choosing the same grinder setting if the standard is documented as "grinder setting #5" than references to how the coffee passes through a sieve. Since grinders can get out of calibration, it may also be helpful to keep a grind sample to compare against in the future and if you do have tools to measure particle size distribution, those should still be used so you can periodically check that your documented grind setting is still producing roughly the same distribution.

Once you've done your experiments and documented your new internal cupping protocol, remember to keep that document up to date. If you get new equipment or perform maintenance that can affect the performance of that equipment, it's important to review how this might affect cupping and decide if any protocol changes are needed. If so, those changes also need to be communicated to everybody involved in cupping.

This might seem like a lot of work, but the end result is that you'll have a greater appreciation for why cupping standards are what they are and you'll have a tighter internal standard that makes it easier for you to make appropriate decisions based on the results of your cupping. It can also be a lot of fun to try these variations and centerlining can be a good training exercise.

Many people seemed to find the article interesting and I'm hoping that it encourages more companies to look at how they're cupping coffee and see if there are changes they can make to get more useful information out of their cupping sessions, allowing them to make better decisions. One of the responses that I got to the article came from Spencer Turer, Vice President of Coffee Analysts and chair of SCAA's Technical Standards committee. He brought up a very good point and has agreed to allow me to publish an excerpt here.

Centerlining is an excellent process to determine what is best for each individual company. Blindly following a testing protocol is never suggested. Once a company determines the most appropriate and efficient testing protocol is must be uniformly applied and consistently followed for the collected data to be actionable. Testing equipment must be calibrated regularly and testing variables must be confirmed to validate the accuracy of data collected.

When a testing protocol is changed any confidence in correlating or comparing the results to other analytical data is eliminated. When coffee is tested differently, quality ratings and sensory profiles will be different. The consistent application of a testing process is required to identify the intrinsic coffee quality; changing a testing variable only succeeds in identifying the results of the variable change.

It is critical for buyers and sellers of green coffee to agree on the evaluation process for quality determination and purchase decisions. This is the reason that the SCAA and CQI utilize a standard protocol for sample roasting, sample preparation and cupping. Controlling the process variables to segregate the intrinsic quality of the coffee, and establishing a common vocabulary is critical to identifying and communicating quality throughout a supply chain.

Neal your engineering approach to coffee cupping and challenging the status-quo is refreshing and applicable for intra-company operations. Thanks for sharing this concept with us.

There are some important points here. With some types of evaluation and for some companies, an ability to communicate through the supply chain is of critical importance. While I'm not personally a Q grader, I've cupped with many of them over the years on five continents and my impression has always been that they're doing excellent work and producing reliable, consistent data. For some types of analysis, the right thing to do is hire a Q grader, train to become one yourself, or arrange to calibrate against one and try to set up your cupping sessions to match as closely as possible to what your partners in other parts of your supply chain are doing.

It's also true that when you change your evaluation methods, your ability to compare results after changing the testing methods with tests performed before that change are limited and possibly eliminated. For a coffee roaster, this might not be much of a problem. You're most likely to want to compare results against coffees that you've purchased previously, which means that you should have a lot of information from other tests to supplement your understanding of those coffees and awareness of how and when testing processes changed might be enough to continue using that information to help inform future decisions. For a company that has never centerlined its processes, I suspect they would find that the improved immediate utility of cupping results would outweigh the loss of legacy data from older testing methods, but when periodically re-evaluating testing methods they might find that small improvements are not necessarily worth losing the ability to compare against past data. That's another decision that should be made deliberately, keeping in mind what sorts of analysis you're performing and how you're using that data.

For example, if you can match a sample roasting specification with roasts that are 8, 9, 10, 11, or 12 minutes, chances are good that one of those is going to give you more useful results than the others, so even if you're fully committed to using the SCAA cupping protocol, it still makes sense to test those options and see what gives you the most useful results across the range of coffees that you'll be evaluating. I've been informed that the Technical Standards committee is working on tightening that range up, but this might prove to be more challenging than it seems at first. I suspect that different companies working on different sample roasters centerlining their process would come to different answers. Some would find their best results on the shorter side of the range while others would find their best results on the longer side of the range. I suspect that these differences would correlate to the time spent prior to the start of color change from green to yellow. A sample roaster that makes that change very quickly probably benefits from a shorter total duration while one that takes longer to get to that initial color change would benefit from a longer total duration. The way to tighten this up would be to ignore that earliest portion of the roast and standardize a tighter duration range from the start of color change through the end of the roast. I doubt the committee intends to move in that direction, but I think that's the right change to make.

As people seem to find this sort of topic interesting, I've written another piece on the use of profile translation analysis as a way of connecting roasting data with cupping data, figuring out how changes in different parts of the roast impact the cup, and using that experience to direct product development in the future. I'll have a post linking to that once it's available.