Evolving the SCA Cupping Protocol and Form: An Overview of the Pilot Testing Process
If you’ve participated in any Specialty Coffee Association (SCA) events or activities over the past few years, it’s likely you’re aware that the SCA is evolving the existing cupping form and protocol, valid for close to 20 years with few changes despite the evolution of the specialty coffee industry and sensory science, into a coffee value assessment system.
MARIO R. FERNÁNDEZ-ALDUENDA, PhD explains the learnings of the evaluation phase of the SCA’s Coffee Value Assessment System work as well as an update on the pilot process at the heart of the project’s ongoing evolution phase.
In 2020, the SCA realized an evolution of the SCA Cupping Protocol was due, but well aware of the level of adoption of the current protocol along the coffee value chain, wanted to move cautiously. The first step of any project grounded in research principles is to complete what’s known as a “literature review,” or to understand the research that had been done by others on the subjects related to the work ahead. We combined the learnings from the project’s literature review—now condensed in the Coffee Sensory and Cupping Handbook—with input from the industry regarding the current cupping protocol’s strengths and weaknesses to design the concept of a new coffee value assessment system (CVAS). At the core of the new system, we use an updated definition of specialty coffee, based on the concept of “attributes,” a feature, aspect, or quality regarded as a characteristic or inherent part of something. Coffees have both intrinsic (measurable characteristics of the product, including flavor and quality) and extrinsic (information about the product, including origin, branding and sustainability) attributes. In a global and diverse specialty coffee industry, different cultures, markets, and individuals value attributes differently. For someone, the sustainability aspects of a coffee may be more important than its cup quality; for other people, it may be the other way round. We have the same situation when it comes to coffee flavor: someone might value highly a fruity and winey natural, for example, while other people may dislike it.
Evolving the Coffee Value Assessment System: Core Concepts and Underlying Logic
The basic concept of the new SCA Coffee Value Assessment System is to offer a systematic and transparent approach to discover a green coffee’s value by comparing “the attributes valued in a coffee” against “the attributes shown by a given coffee.” This requires an honest insight into the attributes one values in coffee as well as an unbiased, thorough assessment of a coffee’s attributes. There are so many different ways to evaluate green coffee—a green coffee assessment (physical appearance, moisture content, etc.), a sensory assessment (flavor profile or impression of quality), or an analysis of the extrinsic attributes linked to the coffee—but knowing the results from one of these assessments, while completing a different kind of assessment, biases the one doing the assessing (the cupper). To avoid this kind of bias, we proposed a compartmentalized approach, where a cupper would complete each assessment separately in order to conceal the extrinsic information about a coffee from the professionals doing the intrinsic assessments. In addition to this, we further separated the two types of sensory assessment—flavor profile (descriptive assessment) and impression of quality (affective assessment)—as sensory science advises disentangling these two types of assessment as sensory testing best practice.
The current cupping protocol emphasizes quality scoring, which is an affective type of assessment. Although some descriptive information is collected in the form, its purpose is primarily to justify the cupping scores; the type of descriptive data collected is not systematic and cuppers are not required to back up their cupping results with descriptive information. One of the overwhelming pieces of feedback we received during the survey element during the current protocol’s evaluation phase (and continue to receive across the pilot programs) is that the times have changed: for some segments of the specialty coffee industry, a coffee’s profile has become more important today than its score. In other words, the importance of the descriptive assessment has surpassed in many cases that of the affective one, while the current cupping method is not designed to compile descriptive information about a coffee. What would a new cupping form look like, if we were to apply both this understanding of what kind of information about a coffee is increasingly important to cuppers as well as our learnings about sensory testing best practice?
Figure 1. The current cupping form collects affective (preference), descriptive, and discriminative information, a combination that sensory science best practice currently advises against. This image highlights which parts of the form fall into which categories of information.
A Proposal for An Evolved Cupping Form
Combining the two logics suggests two separate sensory assessments are required as a part of the new coffee value assessment System: a descriptive assessment, designed to gather descriptive (and objective) information about a coffee’s flavor, and an affective assessment, designed to produce a score based on the coffee’s impression of quality. Both assessments would be broken out into sections, following the traditional procedure for cupping: fragrance/aroma, flavor, aftertaste, acidity, mouthfeel, etc. In the case of the descriptive form, the intensity of each section—for example, the acidity intensity—would be scored on a 15-point scale (widely used in descriptive sensory), and the character of each section would be described using a check-all-that-apply (CATA) list of descriptors, based on the main categories of the SCA/WCR/UC Davis Coffee Taster’s Flavor Wheel. In the case of the affective form, each section’s impression of quality would be rated on a 9-point hedonic scale, widely used in affective tests. The coffee’s score would then be calculated from the hedonic information, using a linear equation designed to produce a better differentiation along the scale than our current cupping method.
Figure 2. Advances in sensory science advocate a "discrete task" approach, which requires evolving the existing protocol into two separate kinds of evaluation: descriptive and affective. This approach could further be expanded by capturing extrinsic attribute information, which we know impacts how a coffee is valued, and using the combination of grading, descriptive, affective, and extrinsic assessments to more transparently assist with value discovery for buyers and sellers alike.
These concepts—separation between descriptive and affective assessments, a 15-point scale for intensities, Coffee Taster’s Flavor Wheel-based CATA lists for sensory characteristics, and the use of a 9-point hedonic scale for impression of quality—were very clear directions for the evolution of the sensory assessment, as they are taken directly from common practice in sensory science. However, the details of how we would integrate those ideas into a cupping form were far from clear, and bearing in mind one of the overwhelming pieces of feedback we received early on in this process during our user perception study was to “please move slowly and carefully,” we knew we needed a series of pilot activities to fine-tune the concepts iteratively.
Testing and Further Evolution
Lecture at CVAS Pilot session in Coatepec, Veracruz Mexico. November 7th, 2022
An initial version of the forms for both affective and descriptive assessments was piloted with three different groups of cuppers between June and August 2022. The first pilot test, using a Spanish-language version of the draft forms, was held in June in Lausanne, Switzerland, with the participation of cuppers from six different countries, including cuppers from several cooperatives. The second pilot, using an English-language version of the draft forms, was held in July at the Alliance for Coffee Excellence laboratory in Portland, Oregon, with the participation of Cup of Excellence stakeholders and other cuppers. A third activity, held during the Coffee Roasters Guild retreat in August with the English-language forms, offered an opportunity to trace results with a larger group of cuppers with diverse backgrounds, experience levels, and preferences. These initial pilot activities were designed to gather several layers of information: sensory data from the assessment forms, which was later analyzed through sensory statistical software; a formal survey, asking participants about their experience with the forms; and freely elicited comments from focus groups held at the end of each activity. Each of these activities included a lecture, explaining the new draft forms, followed by three separate cupping sessions to pilot three different iterations of the form (one that only included descriptive compartments, one that only included affective compartments, and one that had both descriptive and affective compartments), and a group discussion to gather feedback. The same coffees were used in all sessions, but in random order.
Patricia Moreno (left) and Mariana Moreno (right) preform a tasting at CVAS pilot session in Coatepec, Veracruz, Mexico. November 7th 2022.
After analyzing the sensory results and compiling participants’ feedback, we developed a second version of the forms, which we tested in October 2022 in San Diego, with participants from Southern California and Baja California. In addition to testing the new form using the same activities as the first three pilots, we added a training with a set of 17 sensory references prior to tasting to aid cuppers in the descriptive work. The 17 sensory references were carefully curated with two objectives: to represent the main categories of the Coffee Taster’s Flavor Wheel and to propose a set of generic references widely available throughout the world (like “brown sugar”).
The changes to the assessment forms resulting from the second iteration were slight. It feels like we are getting closer to the final version, which will be launched in 2023. In the meantime, we have just completed another pilot test, held in November 2022 in Mexico, with “value chain cuppers”—the cuppers who work closely with producers, cooperatives, and exporters to discover, improve, and control coffee quality. It was a unique opportunity to pilot a refined version of the Spanish-language forms in a true value chain context.
We are very excited about the direction these pilot activities have taken us in the development of the new coffee value assessment system, and are hoping to share technical details about the process and its results very soon.
MARIO R. FERNÁNDEZ-ALDUENDA, PhD is Technical Officer at the Specialty Coffee Association.
The author and the SCA wish to thank all the organizations who have made these pilots possible: Sustainable Harvest Coffee Importers, Alliance for Coffee Excellence, Keurig Trading, San Diego Coffee Training Institute, Escuela de Café Tueste Café, and all the cuppers who participated in these activities. For updates on the progress of this project, please visit sca.coffee/value-assessment.