IB Math AI: Bivariate Statistics and Regression

Bivariate statistics form the backbone of understanding relationships between two variables in the real world. Whether you're analysing economic indicators, scientific measurements, or social trends, these tools allow you to move beyond describing single data points to modelling how they interact. For IB Math Applications and Interpretation, mastering these techniques is essential for turning raw data into meaningful, actionable insights and predictions.

Starting with Scatter Diagrams

The journey into bivariate analysis always begins with a scatter diagram (or scatter plot). This is a graphical representation that plots paired numerical data $(x,y)$ for two variables on a Cartesian plane. The independent or explanatory variable (often denoted $x$) is placed on the horizontal axis, while the dependent or response variable ($y$) is placed on the vertical axis.

The primary purpose of a scatter diagram is to visually assess the form, direction, and strength of a relationship. You look for a correlation—a statistical association between the two variables. The form might be linear (the points roughly follow a straight line) or non-linear (following a curve). The direction is described as positive (as $x$ increases, $y$ tends to increase) or negative (as $x$ increases, $y$ tends to decrease). The strength is judged by how closely the points cluster around an apparent underlying pattern; a tight cluster suggests a strong relationship, while a widely scattered cloud suggests a weak one. This visual inspection is your first and most crucial diagnostic step before any calculation.

Quantifying the Relationship: Pearson's Correlation Coefficient

While a scatter plot gives a visual impression, Pearson's correlation coefficient, denoted $r$, provides a precise numerical measure of the strength and direction of a linear association. Its value always lies between -1 and +1. An $r$ value of +1 indicates a perfect positive linear relationship, -1 indicates a perfect negative linear relationship, and 0 indicates no linear correlation.

The formula for calculating $r$ for a dataset with $n$ pairs is: $$r=\frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum (x_i-\bar{x}{)}^2\sum (y_i-\bar{y}{)}^2}}$$ where $\bar{x}$ and $\bar{y}$ are the means of the $x$ and $y$ variables, respectively. In practice, you will typically use your GDC to calculate $r$. It's critical to remember that $r$ only measures linear correlation. A strong non-linear relationship can yield an $r$ value near zero, which is why always looking at the scatter plot is non-negotiable. Furthermore, a high $r$ value does not, by itself, imply that one variable causes the change in the other.

Modeling the Relationship: The Least-Squares Regression Line

Once a linear correlation is established, the next step is to model it with a line of best fit. The least-squares regression line is the unique straight line that minimizes the sum of the squares of the vertical distances (residuals) between the observed $y$-values and the $y$-values predicted by the line. This method provides the best linear unbiased estimator for the relationship.

The equation of the regression line is written in the form $y=a+bx$. Here, $b$ is the slope (or gradient) and $a$ is the y-intercept. The slope $b$ is calculated as $b=r\cdot \frac{s_y}{s_x}$, where $s_y$ and $s_x$ are the standard deviations of the $y$ and $x$ variables. The intercept is then found using the point of means: $a=\bar{y}-b\bar{x}$. Your GDC will compute $a$ and $b$ directly. The slope $b$ tells you the estimated change in the $y$ variable for a one-unit increase in the $x$ variable. For instance, if modeling study hours ($x$) against test score ($y$), a slope of $b=2.5$ suggests each additional hour of study is associated with an average score increase of 2.5 points.

Interpreting the Model: The Coefficient of Determination

A vital companion to the regression equation is the coefficient of determination, denoted $r^2$. Simply, it is the square of Pearson's correlation coefficient ($r$). While $r$ tells you the strength and direction of the linear relationship, $r^2$ has a more powerful interpretation: it represents the proportion of the variation in the dependent variable ($y$) that is explained by the variation in the independent variable ($x$) using the regression model.

An $r^2$ value of 0.85 means that 85% of the total variation in the $y$-values can be accounted for by the linear relationship with $x$. The remaining 15% of the variation is due to other, unmeasured factors or random error. This makes $r^2$ a key measure of the model's usefulness. A high $r^2$ indicates a model that fits the data well and can make reliable predictions within the range of the data.

Making and Evaluating Predictions

The practical utility of a regression model lies in its ability to make predictions. Interpolation is the process of using the regression equation to predict a $y$-value for an $x$-value that lies within the range of the original data set. This is generally considered reliable, as you are working within the observed boundaries of the relationship.

Extrapolation, in contrast, is when you use the model to predict a value for an $x$-value that is outside the range of the original data. This is risky and often unreliable. The assumed linear relationship may not hold beyond the observed data. For example, using a regression line built from data for children's ages 2-10 to predict height at age 25 would be a faulty extrapolation, as growth patterns change dramatically after puberty. A good analysis always clearly states the domain of the independent variable for which predictions are valid.

Common Pitfalls

1. Confusing Correlation with Causation: Finding a strong correlation ($r$ near ±1) does not prove that changes in $x$ cause changes in $y$. There may be a lurking (confounding) variable that influences both, or the relationship may be coincidental. Always consider logical, real-world explanations before inferring cause and effect.
2. Using Regression for Non-Linear Data: Applying a linear regression model to data that shows a clear curved pattern on a scatter plot will produce a misleading model and invalid predictions. Always check the scatter plot first. The IB Math AI syllabus may require you to linearize certain non-linear relationships (e.g., by using logarithms) before applying these techniques.
3. Ignoring the Limitations of Extrapolation: As discussed, making predictions far outside the original data range is one of the most common and serious misuses of regression. It can lead to wildly inaccurate results. Always qualify your predictions and note when they are based on extrapolation.
4. Misinterpreting the Coefficients: The slope $b$ represents the average change in $y$ per unit change in $x$, assuming the linear model holds. It is not a deterministic law for individual cases. Similarly, the intercept $a$ is often meaningless if an $x$-value of zero is not within the plausible scope of the data (e.g., predicting company revenue based on years since founding, where "0 years" has no practical context).

Summary

• Visualize First: Always begin bivariate analysis by creating and interpreting a scatter diagram to assess the form, direction, and strength of a potential relationship.
• Quantify Linearly: Use Pearson's correlation coefficient ($r$) to measure the strength and direction of a linear association, remembering it does not imply causation.
• Build the Model: The least-squares regression line ($y=a+bx$) provides the best-fit linear model, where the slope $b$ indicates the estimated rate of change.
• Assess Model Fit: The coefficient of determination ($r^2$) tells you the proportion of variation in the $y$-variable explained by the model—a crucial measure of its predictive power.
• Predict with Care: Use the model for interpolation (predicting within the data range) reliably, but treat extrapolation (predicting outside the range) with extreme caution due to its inherent limitations.