Hypothesis Testing: One-Sample Tests

In the data-driven landscape of modern business, managers cannot rely on gut feeling alone to justify strategic decisions. Hypothesis testing provides the formal, statistical framework for making data-driven decisions, allowing you to evaluate claims about a population—such as average product quality, customer satisfaction scores, or process performance—using a sample of data. Mastering this tool transforms raw numbers into actionable evidence, enabling you to validate marketing campaigns, audit supplier claims, and optimize operational efficiency with confidence.

The Foundational Framework: Stating Your Hypotheses

Every hypothesis test begins with a clear, testable claim. You formally state two competing hypotheses. The null hypothesis ($H_0$) represents the status quo or a statement of "no effect" or "no difference." It is the hypothesis you assume to be true until evidence suggests otherwise. The alternative hypothesis ($H_a$ or $H_1$) is what you seek evidence for; it represents a new theory, a change, or an effect.

For example, if a supplier claims their components have an average weight of 100 grams, your null hypothesis would be $H_0:\mu =100$ grams. If you are auditing to see if the weight is different (either less or more), your alternative is $H_a:\mu \ne 100$ grams. This is called a two-tailed test. If you are only concerned if the weight is less than 100 grams, your alternative is $H_a:\mu <100$ grams, making it a one-tailed test. Defining the directionality upfront is a critical business decision, as it affects the sensitivity of your test.

Significance, Errors, and the P-Value: Weighing the Evidence

Once you collect sample data, you calculate a test statistic that measures how far your sample result is from the null hypothesis. To interpret this distance, you use the significance level, denoted by $\alpha$ (alpha). This is the probability threshold you set for rejecting the null hypothesis, typically 0.05 or 5%. It represents your tolerance for a false alarm.

This leads directly to the concepts of error. A Type I error occurs when you reject a true null hypothesis (a false alarm). The probability of making a Type I error is exactly $\alpha$. A Type II error occurs when you fail to reject a false null hypothesis (a missed detection). The probability of a Type II error is denoted by $\beta$. The power of a test is $1-\beta$—the probability of correctly rejecting a false null hypothesis. In business, balancing these errors is crucial: setting a very low $\alpha$ (e.g., 0.01) reduces false alarms but increases the chance of missing a real problem.

The key metric for making the reject-or-not decision is the p-value. The p-value is the probability, assuming the null hypothesis is true, of obtaining a sample result at least as extreme as the one observed. A small p-value (typically $p\le \alpha$) provides strong evidence against $H_0$, leading you to reject it. A large p-value ($p>\alpha$) means you do not have sufficient evidence to reject $H_0$. It is vital to remember that "failing to reject" is not the same as "proving" the null hypothesis true.

The One-Sample Z-Test for a Population Mean

The one-sample z-test is used when you want to test a hypothesis about a population mean ($\mu$) and you know the population standard deviation ($\sigma$). This scenario is less common in practice but is a foundational concept. The test statistic is calculated as:

$$z=\frac{\bar{x}-{\mu }_0}{\sigma /\sqrt{n}}$$

where $\bar{x}$ is the sample mean, ${\mu }_0$ is the hypothesized population mean from $H_0$, $\sigma$ is the known population standard deviation, and $n$ is the sample size.

Business Scenario: A bottling plant's machinery is calibrated to fill bottles to 500 ml, with a known long-term standard deviation ($\sigma$) of 5 ml. To audit the process, a quality manager takes a random sample of 50 bottles, finding a sample mean ($\bar{x}$) of 498 ml. Is this statistically significant evidence that the machine is underfilling? Testing $H_0:\mu =500$ vs. $H_a:\mu <500$ with $\alpha =0.05$, the z-statistic is:

$$z=\frac{498-500}{5/\sqrt{50}}=\frac{-2}{0.707}\approx -2.83$$

A z-score of -2.83 corresponds to a very small p-value (one-tailed) of about 0.0023. Since $p<0.05$, we reject $H_0$. There is strong statistical evidence the machine is underfilling, warranting recalibration.

The One-Sample T-Test for a Population Mean

In reality, you rarely know the population standard deviation. The one-sample t-test is the workhorse for testing a population mean when you must estimate variability from the sample itself, using the sample standard deviation ($s$). The test statistic follows a t-distribution (which has fatter tails than the normal z-distribution, accounting for the extra uncertainty):

$$t=\frac{\bar{x}-{\mu }_0}{s/\sqrt{n}}$$

The degrees of freedom for this test are $df=n-1$. As sample size increases, the t-distribution converges to the normal distribution.

Business Scenario: A company claims its employees complete a standard task in 8 minutes on average. An efficiency consultant times a random sample of 20 employees, finding a sample mean of 8.8 minutes and a sample standard deviation of 1.5 minutes. Is there evidence the true average is different from 8 minutes? This is a two-tailed test: $H_0:\mu =8$, $H_a:\mu \ne 8$.

$$t=\frac{8.8-8}{1.5/\sqrt{20}}=\frac{0.8}{0.335}\approx 2.39$$

With $df=19$ and $\alpha =0.05$, the critical t-value from a t-table is approximately ±2.093. Our calculated t (2.39) exceeds this critical value, so we reject $H_0$. The p-value for this t-score is approximately 0.027, which is less than 0.05, confirming the decision. The consultant has evidence the average task time is statistically different from 8 minutes.

The One-Sample Test for a Population Proportion

Business questions often concern proportions, such as the percentage of defective items, customer conversion rates, or market share. To test a hypothesis about a population proportion ($p$), you use a z-test for proportions. The test statistic is:

$$z=\frac{\hat{p}-p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$$

where $\hat{p}$ is the sample proportion, $p_0$ is the hypothesized population proportion from $H_0$, and $n$ is the sample size. This test relies on the normal approximation to the binomial distribution, which is generally valid when $n{p}_0\ge 10$ and $n(1-p_0)\ge 10$.

Business Scenario: A company's historical defect rate is 5% ($p_0=0.05$). After a process overhaul, a quality audit of 300 randomly selected items finds 10 defects ($\hat{p}=10/300\approx 0.033$). Has the defect rate significantly decreased? Test $H_0:p=0.05$ vs. $H_a:p<0.05$.

First, check conditions: $n{p}_0=300\ast 0.05=15$ and $n(1-p_0)=300\ast 0.95=285$, both ≥10. The test statistic is:

$$z=\frac{0.033-0.05}{\sqrt{\frac{0.05(0.95)}{300}}}=\frac{-0.017}{\sqrt{0.0001583}}\approx \frac{-0.017}{0.01258}\approx -1.35$$

For a one-tailed test at $\alpha =0.05$, the critical z-value is -1.645. Our z-score of -1.35 is not less than -1.645, and the corresponding p-value is approximately 0.0885. Since $p>0.05$, we fail to reject $H_0$. There is not sufficient statistical evidence at the 5% level to conclude the new process has reduced the defect rate, despite the observed improvement.

Common Pitfalls

Misinterpreting the P-value: The p-value is not the probability that the null hypothesis is true, nor is it the probability that your results occurred by chance alone. It is the probability of your data given that the null is true. Confusing this can lead to overstating the certainty of your conclusions.

Ignoring Practical Significance (Effect Size): A result can be statistically significant but practically meaningless. With a very large sample, a tiny difference (e.g., a 0.1% increase in click-through rate) can yield a very small p-value. Always ask: "Is the detected difference large enough to matter for our business decision?" Complement p-values with confidence intervals to see the range of plausible effect sizes.

Neglecting Test Assumptions: Each test has underlying assumptions. The t-test assumes the data comes from an approximately normal population or a large enough sample (n > 30) for the Central Limit Theorem to apply. The proportion z-test requires the success/failure conditions to be met. Violating these assumptions can invalidate your results.

Failing to Consider Power Before Data Collection: Conducting a test with a sample size that is too small is a recipe for a Type II error—you'll likely fail to detect an effect even if one exists. Before collecting data, especially in high-stakes A/B testing or quality control, perform a power analysis to determine the sample size needed to reliably detect a meaningful effect.

Summary

• Hypothesis testing is a structured process for using sample data to evaluate claims about a population, beginning with the clear formulation of a null hypothesis ($H_0$) and an alternative hypothesis ($H_a$).
• The p-value, weighed against a pre-set significance level ($\alpha$), is the primary decision tool. A low p-value provides evidence against the null hypothesis, but you must balance the risks of Type I errors (false positives) and Type II errors (false negatives).
• Use the one-sample z-test for a mean when the population standard deviation is known; use the one-sample t-test when it is unknown and estimated from the sample. Use the one-sample z-test for a proportion to analyze percentages and rates.
• Always contextualize statistical significance with practical significance (effect size) and ensure the assumptions of your chosen test are reasonably met to draw valid business conclusions.