SOP: Paired Comparison Test
The paired comparison is the simplest difference test: two samples, one question, “which is sweeter?” Its focused format gives more statistical power per assessor than the triangle test when you know which attribute might differ.
Human assessors remain the most sensitive instrument for flavor, texture, and aroma, but they are also the most variable. Good sensory practice controls everything around the assessor so that the only thing changing is the product. Discipline in preparation, serving, and data handling is what makes sensory results trustworthy.
Method Principle
The paired comparison presents two samples and asks a single focused question, which maximizes statistical power for that question. With a guessing probability of one-half, the binomial test determines significance from the response split. Directed tests (which is sweeter) are more powerful than preference tests because assessors attend to one defined attribute. The method’s simplicity makes it suitable for large consumer panels and quick QC decisions.
Purpose
To determine whether a perceptible difference exists between two products for a specified attribute, or which is preferred.
Scope
Applies to directed difference testing (which is sweeter) and preference testing (which do you prefer) in product development.
Equipment and Materials
- Two coded samples per assessor
- Ballots with the directed question
- Palate cleansers
- Binomial statistical tables
Procedure
- Recruit at least 30 assessors (50+ for preference tests, which need more power).
- Prepare and code the two samples identically. Balance serving order across assessors.
- For difference testing, ask: “Which sample is sweeter?” (directed). For preference: “Which do you prefer?”
- Require a choice; no ties.
- Count responses for the specified option and test against the binomial critical value.
- For preference tests, check that the preference is not driven by a defect in one sample.
Recording Results
Report the split (for example 38 of 50 chose A as sweeter) with the binomial test result and significance level.
Quality Control
Validate panel performance continuously: track each assessor’s repeatability on blind duplicates and their agreement with the panel consensus. Retrain or replace assessors whose discrimination falls below the program standard. Calibrate the panel with reference standards at the start of each session for descriptive work. Monitor for common biases including order effects, carryover, and fatigue, and design serving orders to counteract them. Review ballot completion rates; incomplete ballots indicate instruction or fatigue problems. Archive all raw ballots and code keys with the study records for audit.
Troubleshooting
- Order bias: balance orders; the first sample is often chosen in preference tests.
- Vague questions: the directed attribute must be crystal clear to assessors.
- Small panels: preference tests with under 50 assessors lack power; increase the panel.
- High no-show rate: over-recruit by 20 percent and confirm attendance the day before.
- Data entry errors: use digital ballots with range checks; double-enter paper ballots.
Interferences and Limitations
Panelist physiological state affects sensitivity: illness, smoking, strong foods before the session, and even time of day shift thresholds. Screen assessors at session start and excuse anyone compromised. Context effects bias ratings: a mediocre sample tastes better after a poor one. Controlled serving orders and palate cleansers mitigate but do not eliminate these effects. Sensory results describe perception under test conditions; they predict consumer response but do not replace in-market validation for major launches.
Safety Notes
- Allergen screening.
- Observe holding times.