SENSORY EVALUATION: DISCRIMINATION TESTING
Diagram
Two at a time, or all at once
Paired comparison
Two samples are served at a time, and the panellist picks one: the sweeter, the stronger, or the one they like. Only two samples are judged at once.
↓
Ranking test
Several samples are served at once, and the panellist puts them in order, from least to most, or from most liked to least liked.
↓
Same family
Both compare samples, and both are used in discrimination testing and in acceptance and preference testing (p. 430).
Figure 1: Paired comparison tests two samples at a time; a ranking test orders several at once. Drawn in our own style from the discrimination testing chapter (p. 116) and the acceptance and preference testing chapter (p. 430) of Lawless and Heymann, Sensory Evaluation of Food.
Introduction
A paired comparison test and a ranking test both compare samples, but one tests two samples at a time and the other orders several samples at once. In a paired comparison, two samples are served and the panellist picks one: the sweeter, the stronger, or the one they like. In a ranking test, several samples are served together and the panellist puts them in order, from least to most of an attribute, or from most liked to least liked. Lawless and Heymann cover both in the discrimination testing chapter (p. 116) and in acceptance and preference testing (p. 430).
The core difference is the number of samples judged at once: two in a paired comparison, and three or more in a ranking test.
Quick comparison
Table 1: Paired comparison compared with Ranking test
| Aspect | Paired comparison | Ranking test |
|---|---|---|
| Samples at once | Two | Three or more, often four to six |
| Task | Pick one of the two | Put all samples in order |
| Question asked | Which is sweeter, stronger, or liked more | What is the order, from least to most, or most liked to least liked |
| Result | One winner per pair | A full order of all samples |
| Chance of guessing right | One in two | Low: the chance of getting the whole order right by guessing is small |
| How much it shows | Only which of the two is more, not by how much | The order, but not how big the gaps between samples are |
| Memory load | Low: only two samples to hold in mind | High: all samples must be held in mind and compared |
| Fatigue | Low per pair, but many pairs are needed to compare many samples | High: tasting many samples at once is tiring |
| Best use | Comparing two products, or two recipes | Ordering several products at once, such as several sweetness levels |
| Analysis | Count how many chose each sample, and check against guessing | Count the ranks, and check whether the order is real or chance |
Scientific explanation
Definition
A paired comparison test is a test in which two samples are served and the panellist picks one, for a stated attribute or for preference. A ranking test is a test in which several samples are served together and the panellist puts them in order, from least to most of an attribute, or from most liked to least liked.
Both are covered by Lawless and Heymann in the discrimination testing chapter (p. 116) and in acceptance and preference testing (p. 430): the paired comparison is used both to find a difference and to ask for a preference, and the ranking test is used to order several samples at once.
Working principle
A paired comparison works by forcing a choice between two: the panellist tastes both, compares them, and picks one. Because there are only two choices, a panellist guessing at random is right one time in two, so many panellists are needed before the count proves a real difference or a real preference.
A ranking test works by ordering: the panellist tastes all the samples, compares them with each other, and puts them in order. The chance of getting the whole order right by guessing is small, so even a modest panel can show a real order, but the test cannot say how big the gaps are between the samples.
Apparatus
Both tests need a sensory test room with separate booths, coded samples, water and plain crackers to cleanse the palate, and a ballot form. A paired comparison needs two coded samples per pair, and a ranking test needs all the samples served together on a tray.
Measurement procedure
For a paired comparison, the panellist is served two coded samples in a balanced order, tastes both, and marks the one that is more of the stated attribute, or the one they prefer. For a ranking test, the panellist is served all the samples together, tastes them, and writes the codes in order, from least to most, or from most liked to least liked.
The answers are counted: in a paired comparison, the count for each sample is checked against guessing, and in a ranking test, the ranks are added up for each sample and checked to see whether the order is real.
Factors affecting readings
Fatigue matters most for ranking tests: tasting many samples at once is tiring, so the samples must be small and the palate cleansed between them. Order matters for both: the first sample tasted can seem stronger, so the order is balanced across panellists. The size of the real differences matters most of all: if the samples are nearly the same, no test can order them reliably.
Formulas and calculations
The guessing probability follows from the design. In a paired comparison there are two samples and one is picked, so the chance of guessing right is 1 in 2. Worked example: if 40 panellists take a paired comparison and guess at random, the number expected to pick a given sample is 40 x 1/2 = 20. In a ranking test the chance of getting the whole order right by guessing falls fast as samples are added: with 4 samples it is 1 in 24, because there are 4 x 3 x 2 x 1 = 24 possible orders. The count of picks, or the sum of ranks, is checked against chance to judge whether the result is real.
Industrial applications
- Paired comparison is used to choose between two recipes before a product is launched.
- Paired comparison is used to check whether a new sample matches a standard, or which of two is sweeter or stronger.
- Paired comparison is used in preference tests, where buyers pick the product they like more.
- Ranking tests are used to order several sweetness, saltiness or strength levels, to find the level buyers like most.
- Ranking tests are used to order several products at once, where testing every pair would take too long.
Advantages and limitations
- Paired comparison, advantage: simple and easy for the panellist, with only two samples to judge at once.
- Paired comparison, limitation: only two samples at a time, so comparing many products needs many pairs and many panellists.
- Ranking test, advantage: orders several samples in one test, so it is fast for comparing many products.
- Ranking test, limitation: tiring, because all samples must be tasted and held in mind, and it cannot say how big the gaps are between samples.
FAQ
Can a ranking test tell me how much sweeter one sample is than another?
No: it only gives the order, not the size of the gaps. To measure how much sweeter, use an intensity scale in a descriptive test.
Why does a paired comparison need so many panellists?
Because guessing right is easy: one in two. Many panellists are needed before the count proves that the winner was really picked, not guessed.
How many samples can a ranking test handle?
Usually four to six: more than that tires the panellist, and the order becomes unreliable. For more samples, test them in smaller groups.
Can a paired comparison ask about liking?
Yes: paired comparison is used both for difference, such as which is sweeter, and for preference, such as which is liked more. The question on the ballot must state which one is asked.
What if the panellists cannot agree on an order?
That is a result too: it means the samples are too alike to order reliably, or the panel is too small. More panellists, or a simpler test, may be needed.
