SOP: Testing Ranked Preference Data by the Chi-Square Test
GIFSQ — Global Institute for Food Safety & Quality
Principle
When panelists rank several samples in order of preference, the chi-square (χ²) test applied to the rank totals determines whether the differences among the samples are significant. If the null hypothesis of no real difference held, the rank totals would be nearly equal; a large χ² value establishes a genuine difference in preference.
Materials
- The ranking of the n samples by each of the P panelists (1 = most preferred)
- The book’s chi-square table for (n − 1) degrees of freedom
Procedure
Have each panelist rank the n samples in order of preference, 1 to n
↓
Add the ranks given to each sample over all panelists to obtain the rank total RT for each sample
↓
Square each rank total and add to obtain Σ(RT)²
↓
Compute the χ² value using the expression below
↓
Compare the calculated χ² with the tabulated value for (n − 1) degrees of freedom and judge the significance of the differences among the samples
Calculation
χ² = [12 / (P × n × (n + 1))] × Σ(RT)² − 3P(n + 1)
Where,
- P = number of panelists
- n = number of samples
- RT = rank total of a sample
- Σ(RT)² = sum of the squares of the rank totals
- χ² has (n − 1) degrees of freedom
Worked example — ranking of five beverages
Eight panelists ranked five beverage samples A–E (1 = most preferred):
| Panelist | A | B | C | D | E |
|---|---|---|---|---|---|
| 1 | 1 | 4 | 5 | 3 | 2 |
| 2 | 1 | 3 | 5 | 4 | 2 |
| 3 | 1 | 4 | 3 | 5 | 2 |
| 4 | 2 | 3 | 5 | 4 | 1 |
| 5 | 3 | 2 | 4 | 5 | 1 |
| 6 | 1 | 2 | 4 | 5 | 3 |
| 7 | 2 | 3 | 4 | 5 | 1 |
| 8 | 1 | 3 | 4 | 5 | 2 |
| Rank total | 12 | 24 | 34 | 36 | 14 |
P = 8; n = 5; Σ(RT)² = 12² + 24² + 34² + 36² + 14² = 144 + 576 + 1156 + 1296 + 196 = 3368
χ² = [12/(8 × 5 × 6)] × 3368 − 3 × 8 × 6 = (12/240) × 3368 − 144 = 168.4 − 144 = 24.4
For 4 degrees of freedom, a value as high as 14.9 will occur by chance only 0.5% of the time. The calculated value of 24.4 far exceeds it; therefore the difference among the samples has been established beyond the 5% level — the samples genuinely differ in preference.
Notes: (1) panelist 4’s rank for sample C is illegible in the available copy; it is carried as 5, the only value that makes the row sum to 15 (the ranks 1–5) and gives sample C the printed rank total of 34; (2) the printed χ² value is garbled in the scan and is carried here as recomputed (24.4), which is consistent with the printed conclusion.
Precautions
- Every panelist must rank all n samples using each rank exactly once, so that every row sums to n(n + 1)/2.
- The test establishes that the samples differ; it does not say which pair differs.
References
Ranganna, S., Handbook of Analysis and Quality Control for Fruit and Vegetable Products, 2nd edition.