As a marketer, I have always been interested in facts, figures, and stats. It made me feel like I know something for sure without doubts, and I can act upon the information. But the process of coming to the result was new to me until I started running ad campaigns and tried to understand what performed better. It was the key to understanding what kind of ads I would run in the future.
Take, for example, Google ad campaigns; it gives you all the information about impressions, clicks, demographics, etc. Still, if you cannot infer what it means in terms of likelihood to click based on Gender or location, it means you cannot know whether you should run a gender or location-specific ad campaign in the future to focus on getting better results.
Here I learned about t-tests and how it is essential for a marketer looking to maximize conversions by testing call to action button’s or subject line performance. Typically, one which gives better CTRs being the winner, and we use it in the future.
But we perform a t-test to see if there is ‘significant difference’ because of the slight change in 2 versions of the testing material (webpage, email, ad)
With the t-test, we can understand if the result had to do something with the respondent’s characteristics or demographics.
What is a t-test?
The t-test is used to determine the difference between the two groups, which may be related to certain features. It can also be called a hypothesis testing tool, which can be used to test an assumption about a population.
The best thing is that this test can be done on Excel using the Function in the formula bar or Data Analysis plug-in.
In the above example, the p-value is 0.000006776 which is less than 0.05.
When we find the p-value, we see whether it is greater or lesser than the significance level (most commonly 0.05 at a Confidence interval of 95%).
The p-value higher than 0.05 is not statistically significant, or in simpler words, it is safe to say that the likelihood to choose one version over the other is due to preference towards one and is not by chance.
The p-value is lesser than or equal to 0.05 is statistically significant, or both versions performed the same, and we can use any of them.
In conclusion, the t-test is one of the most useful statistic tests I have ever used in judging the performance of an ad campaign over the other. Some of the most profound observations and insights that we have used in designing layouts are also the result of this test. I hope you also find a t-test as impressive as I have.
If you enjoyed reading this article, be sure to read my last week’s article on Data Points and Data Insights.