Further Hypothesis Testing and Confidence Intervals

Mastering hypothesis testing and confidence intervals is essential for moving beyond simple data description into meaningful statistical inference. These tools allow you to make reliable claims about entire populations based on sample data, forming the backbone of scientific research, quality control, and data-driven decision-making. This guide builds on foundational concepts to tackle realistic scenarios where key population parameters, like variance, are unknown.

Foundational Concepts and the t-Distribution

Before diving into new procedures, it's crucial to solidify the logic of statistical inference. Hypothesis testing is a formal process for evaluating claims about a population parameter. You start with a null hypothesis ($H_0$), which is a statement of no effect or status quo (e.g., a population mean $\mu =k$), and an alternative hypothesis ($H_1$), which is what you seek evidence for (e.g., $\mu >k$, $\mu <k$, or $\mu \ne k$). The process involves calculating the probability of observing your sample data, or something more extreme, if the null hypothesis is true; this probability is the p-value.

Previously, you likely used the z-test when testing a population mean with a known population variance (${\sigma }^2$). In reality, ${\sigma }^2$ is rarely known. When you must estimate the population variance using the sample variance ($s^2$), the standardised test statistic no longer follows a standard normal (Z) distribution. Instead, it follows a t-distribution. The t-distribution is similar in shape to the normal distribution—symmetric and bell-shaped—but has thicker tails. This accounts for the extra uncertainty introduced by estimating ${\sigma }^2$ from the sample. The exact shape of the t-distribution depends on the degrees of freedom ($\nu$), which for a one-sample test is $n-1$, where $n$ is the sample size. As $n$ increases, the t-distribution converges to the standard normal distribution.

Hypothesis Tests for Means with Unknown Variance (The t-test)

When the population variance is unknown, you conduct a t-test. The procedure is analogous to a z-test but uses the sample standard deviation and the t-distribution.

Step-by-Step Procedure:

1. State Hypotheses: Define $H_0:\mu ={\mu }_0$ and $H_1$ (one- or two-tailed).
2. Calculate the Test Statistic: The formula for the t-statistic is:

$$t=\frac{\bar{x}-{\mu }_0}{s/\sqrt{n}}$$ where $\bar{x}$ is the sample mean, ${\mu }_0$ is the hypothesised population mean, $s$ is the sample standard deviation, and $n$ is the sample size.

1. Determine the Critical Value or p-value: Using tables or software, find the critical t-value from the $t_{n-1}$ distribution at your chosen significance level ($\alpha$, commonly 0.05). Alternatively, calculate the p-value associated with your observed t-statistic.
2. Make a Decision: If $|t|>t_{critical}$ (or if p-value < $\alpha$), reject $H_0$. Otherwise, do not reject $H_0$.
3. Interpret in Context: State your conclusion in plain language related to the original problem.

Example: A manufacturer claims its lightbulbs last 1200 hours. A sample of 15 bulbs has a mean life of 1150 hours with a standard deviation of 100 hours. Test the claim at the 5% level.

• $H_0:\mu =1200$, $H_1:\mu <1200$ (one-tailed).
• $t=\frac{1150-1200}{100/\sqrt{15}}\approx -1.936$.
• Degrees of freedom $\nu =14$. The critical $t$-value for a one-tailed test at $\alpha =0.05$ is approximately -1.761.
• Since $-1.936<-1.761$, we reject $H_0$. There is significant evidence at the 5% level that the bulbs last less than 1200 hours.

Constructing Confidence Intervals for Means and Proportions

While hypothesis testing gives a yes/no answer to a specific claim, confidence intervals provide a range of plausible values for the population parameter. A 95% confidence interval means that if we were to take many samples and construct an interval from each, we would expect 95% of those intervals to contain the true population parameter.

For a Population Mean (σ unknown): The general form is: $\text{point estimate}\pm (\text{critical value})\times (\text{standard error})$. For a mean with unknown $\sigma$, the formula is: $$\bar{x}\pm t_{n-1,\alpha /2}\times \frac{s}{\sqrt{n}}$$ Here, $t_{n-1,\alpha /2}$ is the critical t-value for a two-tailed probability $\alpha$. For a 95% CI, $\alpha =0.05$.

For a Population Proportion: When estimating a proportion $p$, from a large sample, we use the normal approximation. The confidence interval is: $$\hat{p}\pm z_{\alpha /2}\times \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ where $\hat{p}$ is the sample proportion and $z_{\alpha /2}$ is the critical z-value.

Example (Proportion): In a poll of 500 voters, 280 support Candidate A. The 95% CI for the true proportion is: $\hat{p}=280/500=0.56$, $z_{0.025}\approx 1.96$. $0.56\pm 1.96\times \sqrt{\frac{0.56\times 0.44}{500}}=0.56\pm 0.0435=(0.5165,0.6035)$. We are 95% confident the true population proportion supporting Candidate A lies between 51.7% and 60.4%.

The Relationship Between Width, Sample Size, and Confidence Level

The width of a confidence interval is not arbitrary; it communicates the precision of your estimate. Three factors directly control it:

1. Confidence Level: Choosing a 99% CI over a 95% CI increases the critical value ($t$ or $z$), making the interval wider. Higher confidence requires a wider net to be more sure of catching the true parameter.
2. Sample Size ($n$): Increasing the sample size decreases the standard error (the $s/\surd n$ or $\surd (\hat{p}(1-\hat{p})/n)$ part). A smaller standard error makes the interval narrower, increasing precision. Width is proportional to $1/\surd n$.
3. Sample Variability ($s$ or $\hat{p}$): More variable data (larger $s$) produces wider intervals. For proportions, the standard error is maximised when $\hat{p}=0.5$.

You can use this relationship in reverse to determine the sample size required to achieve a desired margin of error (half-width) for a proportion: $n\approx (z_{\alpha /2}^2\times \hat{p}(1-\hat{p}))/E^2$, where $E$ is the margin of error. If $\hat{p}$ is unknown, use $\hat{p}=0.5$ for a conservative (largest) estimate.

Common Pitfalls

1. Misinterpreting the Confidence Level: A 95% CI does not mean "there is a 95% probability that the true parameter lies in this specific interval." The parameter is fixed; the interval is random. The correct interpretation is about the long-run success rate of the method.
2. Using a z-test when a t-test is required: The most common error is using $z$ critical values or the normal distribution when the population standard deviation is estimated from the sample. Always ask: "Did I know $\sigma$, or did I calculate $s$?" Use $t$ for $s$.
3. Ignoring the conditions for inference: The t-test and confidence intervals for means assume the data comes from an approximately normal population or a large enough sample (n > 30 typically, but depends on skew) for the Central Limit Theorem to apply. For proportion intervals, you must check that $n\hat{p}>5$ and $n(1-\hat{p})>5$ to use the normal approximation.
4. Confusing significance with practical importance: A statistically significant result (e.g., rejecting $H_0$) may arise from a tiny but real effect when the sample size is huge. Always consider the context and the magnitude of the effect, not just the p-value. Examine the confidence interval to see the range of plausible effect sizes.

Summary

• When the population variance is unknown, use the t-distribution and t-tests for hypothesis tests concerning a population mean. The test statistic is $t=(\bar{x}-{\mu }_0)/(s/\surd n)$ with $n-1$ degrees of freedom.
• Confidence intervals provide a range of plausible values for a parameter (mean or proportion). For a mean with unknown σ, the interval is $\bar{x}\pm t_{n-1,\alpha /2}\times (s/\surd n)$. For a proportion, it is $\hat{p}\pm z_{\alpha /2}\times \surd (\hat{p}(1-\hat{p})/n)$.
• The width of a confidence interval is controlled by the confidence level (higher level = wider interval), sample size (larger n = narrower interval), and sample variability.
• Always interpret results in the context of the real-world problem, and be vigilant about the underlying assumptions and common misinterpretations of both p-values and confidence intervals.