The Guinness Question: Is the Difference Real?
Run any experiment with two groups and the averages will differ — they always do. A Guinness brewer invented the tool for deciding when a difference is real, and you can rediscover it with your own experiment.
Where this idea comes from
Start here — these are the sources that inspired this exploration.
Introduction
Around 1900, Guinness brewer William Gosset had a problem: he could only afford small samples of barley, and small samples lie — two batches always taste of different numbers even when nothing has changed. The tool he invented (published secretly as 'Student') is now a standard tool in clinical trials worldwide. Your version: run a two-group experiment on yourself or your class — reaction time with music vs without, free throws warmed-up vs cold, memory in the morning vs afternoon. The means will differ. The exploration is about the harder question underneath: how big must a difference be, for the sample size you have, before chance stops being a good explanation? The maths is the subject; whether music actually helps is just your test case.
Guiding Questions
- Design a fair two-condition experiment you can really run, and collect at least 15 measurements per condition. What did you do to stop other differences sneaking in?
- Compare the two groups: means, spreads, and a plot of all points. The means differ — but glancing at the overlap, are you convinced?
- Put chance on trial: shuffle all your measurements, deal them randomly into two fake groups, and record the difference in means. Do this many times. How often does pure chance beat the real difference you found?
- Gosset's t-statistic packages your simulation into one number, balancing the gap between means against the spread and the sample size. Compute it for your data — does its verdict agree with your shuffling?
- How would your conclusion change with five measurements per group? With fifty? Investigate how sample size changes what you're able to detect — Gosset's actual problem at the brewery.