The Math of Numb3rs 104: Uncertainty Principle


Background Reading material from Cornell University.

PRIZES. If you present the maths/stats content in the reading material that interests you or address a challenge posted on a previous episode during our 10 minutes discussion session (1700-1710 hours) then you will get a ``special lunch package''!

Interviewer Effect by Richard Penny of Statistics New Zealand (edited by Raaz)

The use of the Heisenberg Uncertainty Principle in the show is a typical misunderstanding of what it is and why it arises. The Heisenberg Uncertainty Principle is quite specific and better left to the realms of statistical experiments with mathematical models of sub-atomic particles. What they possibly mean in the show can be better explained by using elementary concepts in statistical theory. Charlie seems to be conflating the Heisenberg Uncertainty Principle with the observer effect. Observer effect refers to changes that the act of observation will make on the phenomenon being observed. It is certainly true that data collected from a data generating agent can be severely affected if the agent is aware of being observed. A well-known example is the Hawthorne Effect, and this would probably be a better name for what they are talking about in the show as opposed to the sub-atomic analogy.

As a survey statistician, it is necessary to consider how the very act of data collection can affect the data one collects. An area with a great deal of research is on the "interviewer effect". This is where you have so many data points to collect that more than one interviewer is required to collect the data. This is a common occurrence for interviewer surveys. Many studies have shown that the answers one gets for some types of questions can vary if the interviewer is similar to the the subjects being interviewed compared to those obtained if the interviewer is different. An example from the U.S. is where a person's data collected by an interviewer regarding race relations will vary depending on whether the interviewer / interviewee is Caucasian/white or African/black American. If you want to know more about the theory of interviewer effects see InterviewerEffect.pdf.

Another effect is the "observer effect". This is where the data generating agent gives a different result if the agent is aware of being observed. Recently, in our STAT 312 Sampling Methods course, the students were asked to collect some data in order to appreciate that data collection problems are part of the survey design. Three of the student groups considered observer effects.

The first group was measuring the time it took between ordering and getting a pizza at various local pizza outlets. I quote from one of the researchers. "When sampling pizza waiting times, we chose not to ask for permission to sample from the pizza places because we thought that it might alter our data. If they knew we were recording times, then they may speed up their cooking and delivery times." Sarah Ramsey

Another group was looking at how many people park at the university by taking the risk of getting a ticket, i.e. a fine for parking violation. To find this out they had to look for payment slips in cars, which is what parking wardens would do. One of the data collectors was actually approached by anxious students believing her to be a parking warden!

The third group was trying to see what kind of people are more likely to pick up a coin in the street. They needed to consider whether the person's knowledge of being observed would affect her/his likelihood of picking up the coin.

CHALLENGES for numb3rs 104:

  1. This episode focussed on Charlie's psyche too much and offered little concrete material to latch on to for challenges.

Maintained by Raazesh Sainudiin and mirrored by Paul Brouwers.