[TTStats] What is a t-test?

In 2014/2015, the material used to make table tennis balls changed from celluloid to a non-flammable plastic. These balls are a tiny bit bigger than the 40mm celluloid balls, but the main difference is the type of material they are made from. According to the information online, the balls have less spin and bounce after the change of materials.

As a data person, I want to measure whether there are big differences in the bounciness of the two types of balls. I get a box of plastic balls and another of celluloid balls, both of which have 10 pieces of new balls inside. The way I measure the bounciness is that I let each ball free fall at the height of 300mm onto the table tennis table and measure what’s the height of its first bounce.

After 20 free falls, I have obtained the following data:

With these data, we can calculate the average of the height of each type of balls’ first bounces:

From the mean figures, we can see that the average bounce height of plastic balls is almost 10mm smaller than celluloid balls. However, what does it really tell us? Does it mean that there is a big difference between the heights of the first bounce of the two types of balls? Is it possible that the 10mm differences are just by chance (e.g. in a new test, plastic balls may have higher first bounce heights than the celluloid balls)? How can we draw the conclusion that there are a big enough differences in the heights that we can prove that the difference in the bouncy level does exist?

The distribution of the heights is plotted on the picture below. There are obvious differences and the minimum of the celluloid height is already greater than the maximum height of plastic, however, is it conclusive enough?

In order to figure out this question, we need to introduce the t-test.

The t-test is a commonly used technique that measures the differences between sample means.  It calculates the differences between the groups and compares it to the differences within the groups

T= variance between groups/variance within groups

A big t-value indicates different groups and a small t-value indicates similar groups.

How do we know a t-value is big enough to decide that the groups are very different? Each t-value corresponds to a p-value which indicates the possibility of ‘there is no real difference between the groups’. A p-value of less than 0.05 says that the probability of two groups being indifferent is less than 5%. The smaller the p-value, the more likely that the two means are different. Usually, we take 5% as a threshold of judging whether the difference is significant.

Now back to our example, we run a t-test on the data we collected and below are the outputs:

The p-value is 3.955e-10, which means that there is less than 0.00001% probability that the average bounce heights of plastic balls and celluloid balls are indifferent. This proves that the bounce heights of plastic balls are significantly lower than the celluloid balls and they are much less bouncy.

The t-test that we use to compare the means between two groups is called an independent t-test. There are also paired t-test and one-sample t-test.

The paired t-test is to test the means of the same group twice. The advantage of this t-test is that it reduces possibly variability of subjects, however, it’s prone to order effects. e.g. the body temperature of a group of runners before and after a marathon.

The one-sample t-test is to compare the group mean with a hypothesized/known sample mean. e.g. the average IQ of a group of Cambridge graduate vs. 100

*The plastic vs. celluloid ball example I gave is just for storytelling purpose. I did not conduct that experiment. The tests and the figures are from the experiment done by Preston Table Tennis Association. If you are interested in the experiment, kindly check out their video: Plastic & Celluloid Table Tennis Balls Compared: Bounce and Conformity