Data science simplified: Hypothesis testing with Isildur and Gandalf

Edward Teller, the famous Hungarian-American physicist, once said:

“A fact is a simple statement that everyone believes. It is innocent unless found guilty. A hypothesis is a novel suggestion that no one wants to believe. It is guilty until found effective.”

The application of hypothesis testing is predominant in data science. It is imperative to simplify and deconstruct it. Like a crime fiction story, hypothesis testing, based on data, leads us from a novel suggestion to an effective proposition.

Concept

The word hypothesis originates from the Greek words hupo (under) and thesis (placing). It means an idea made from limited evidence. It is a starting point for further investigation.

The notion is simple yet powerful. We perform hypothesis testing intuitively every day. It is a seven-step process:

1. Make assumptions
2. Take an initial position
3. Determine the alternate position
4. Set acceptance criteria
5. Conduct fact-based tests
6. Evaluate results. (Does the evaluation support the initial position? Are we confident that the result is not due to chance?)
7. Reach one of the following conclusions: reject the original position in favor of the alternate position or fail to reject the initial position.

Process

Let me use an example to explain the concept of hypothesis testing.

Holmavik is a small town in the western part of Iceland. This little town is unique and is known for the Museum of Witchcraft. Even now, there are people in Westfjords who claim to be wizards.

Let’s take fictional characters Isildur and Gandalf as examples. Both Isildur and Gandalf claim to be wizards and are clairvoyant. A statistician wants to prove or disprove this claim. They play a clairvoyant card game. Isildur and Gandalf are shown the backs of 10 randomly selected cards from a set of playing cards, and they have to identify each cards’ suit.

It is also determined that for a normal person, the average number of times the prediction is correct is around six. This is the basis on which we will perform the hypothesis test and statistically determine if they are wizards or not.

Step 1: Make assumptions

Different kinds of hypothesis tests have different assumptions. Assumptions are related to the distribution of data, sampling, and linearity. Two of the common assumptions are:

Conclusion