“Who think they can taste the difference between regular and diet coke?”
One by one, some hands go up. About half the class raises their hand.
“Suppose someone takes a sip from a cup and states: it is regular coke. Does that prove that they can taste it?”
Students shake their heads. “No. It could be a lucky guess.”
“Exactly. If they predict it correctly twice, does that prove that they can taste it?”
Again, students shake their heads. “Nope. Could be two lucky guesses”
“And what if they predict it correctly six times in a row?”
Students are in doubt. “Eh, then I would believe they can taste it, yes..” “Would it prove they can taste it?”
“Ehm, no, I guess? It could still be a coincidence, but that would be quite unlikely.. So yeah, I would believe it if someone would correctly predict five or six out of six cups.”
“All right. Then let’s do what we do in experimental statistics: let’s put it to a test!”
From under the table I pick four large bottles of cola. Half is regular coke, the other half is diet coke.
The class is suddenly filled with noise and excited chatter.
“Everyone who raised their hand, claiming that they could taste the difference between regular and diet coke: congratulations! You have just volunteered to be in the experimental group.
What we will test is simple: can those of you who say they can taste the difference between regular and diet coke actually taste the difference and correctly label cups filled with coke.
All the others who did not raise their hand, Science will be grateful to you, as you will be the preparers, responsible for preparing and administering our experiment.
Now, we want to be sure that we actually test if the tasters can taste the difference. That means we want to minimise any factors that would make it seem that the tasters can taste the difference, without them actually being able to taste the difference. We want to create a “controlled” experiment.
Everyone, please think about this for a minute: what can we do create some essential controls this experiment?”
All students are quiet and thinking.
“Please discuss with your neighbour and then list what we can do to create a controlled experiment?” Chattering all around again. I ask some students what they came up with.
“The tasters should not see what we pour into the cups!”
Yes, indeed, great one. That is often called a ‘blind’ experiment.
All right, let’s commence the preparation of the experiment. Tasters, this is a ‘blind’ experiment, you shall not peek, so please leave the room and wait outside.
Still a little dumbfounded, the tasters leave the class.
Dutifully, the preparers start filling the cups, writing down the codes and creating lists they will use to write down the predictions of the tasters.
After five to ten minutes, the tables are filled with lines and lines of paper cups.
One of you, please tell the tasters we are ready for them!” Slowly, the tasters walk in.
“Tasters, please stand at a line of cups. Researchers, does each of you have a taster you will observe? What will happen now is this: tasters, you will take a sip of a cup, use your incredible tasting abilities, and just name the correct type: regular or diet.
Researchers, you write down what the tasters say. When the taster has made predictions for each cup, you’re done. Any questions you may have that would block you from doing this task? No? Then let the experiment commence! “
The room fills with sound. Some tasters quickly take sips. Others seem to be at a wine tasting, slowly sifting the cola through their mouth, eyes closed, the preparers eagerly awaiting their verdict - diet or regular?
After ten minutes, all the tasters are done. Everybody returns to their seat. The big question looms large: can the tasters taste the difference?
The answer…
I leave that up to you to find out in your own in-class experiment!
Which learning activities do you use to help students understand inferential statistics?
P.S. Really, really want to know the outcomes of this class? Just email me and I’ll happily send you the data from our class experiment ;-)