IB AI: Descriptive Statistics and Representation

Data is the raw material of Artificial Intelligence, but without proper organization and analysis, it remains meaningless. Descriptive statistics provide the essential toolkit for summarizing, visualizing, and interpreting data, transforming it into actionable information for making predictions, training models, and ultimately, driving informed decisions in the real world.

Understanding Your Data: The First Step

Before any calculation or graph can be made, you must understand what you are measuring. Data can be categorized into two primary types: quantitative and qualitative. Quantitative data deals with numbers and measurements, such as height, temperature, or test scores. It can be further split into discrete data (countable, like the number of students) and continuous data (measurable, like time or weight). Qualitative data, or categorical data, describes qualities or characteristics, such as eye color, brand preference, or blood type. This fundamental classification dictates every subsequent choice in representation and analysis.

Organizing raw data begins with a frequency table. This simple tool tallies how often each data value or category occurs. For quantitative data, especially continuous data, data is often grouped into class intervals (e.g., 10-19 cm, 20-29 cm). The frequency table then shows the number of data points within each interval. This process condenses a long list of numbers into a clearer picture of where the data concentrates.

Visualizing Distributions: Histograms and Cumulative Frequency

For a visual summary of grouped quantitative data, a histogram is the primary tool. Unlike a bar chart, a histogram's bars touch each other because the data is continuous on the x-axis. The height of each bar represents the frequency (or sometimes the frequency density) of data within that class interval. By examining a histogram, you can quickly assess the distribution of the data—whether it is symmetrical, skewed left or right, or has multiple peaks (bimodal).

To answer questions like "what percentage of data falls below a certain value?", we use the cumulative frequency curve (or ogive). You first construct a cumulative frequency table by adding up frequencies as you go. Plotting the upper boundary of each class interval against its cumulative frequency creates a rising curve. This powerful graph allows you to estimate medians, quartiles, and percentiles directly. For example, the median corresponds to the 50th percentile, found by drawing a line from the cumulative frequency axis (at 50% of the total) to the curve and then down to the data axis.

Summarizing with Numbers: Measures of Center and Spread

Visuals tell one story; summary statistics tell another. Measures of central tendency identify a typical or central value. The mean ($\bar{x}$) is the arithmetic average, sensitive to every value. The median is the middle value when data is ordered, robust against extreme scores. The mode is the most frequent value. The choice depends on the data's distribution and the presence of outliers.

Knowing the center is insufficient; you must also measure variability. Measures of spread quantify how much the data scatters. The range (max - min) is simple but easily distorted by outliers. The interquartile range (IQR) is far more robust. It is the range of the middle 50% of the data: $IQR=Q_3-Q_1$, where $Q_1$ is the first quartile (25th percentile) and $Q_3$ is the third quartile (75th percentile). The IQR is the cornerstone of the box-and-whisker plot (box plot), a standardized visual summary showing the median, quartiles, and potential outliers in one compact graphic.

Identifying Outliers and Making Informed Decisions

An outlier is a data point that lies an abnormal distance from other values. In a box plot, outliers are typically identified using the 1.5 × IQR rule. Any data point below $Q_1-1.5\times IQR$ or above $Q_3+1.5\times IQR$ is considered an outlier and is plotted as an individual point. Identifying outliers is critical because they can significantly skew the mean and standard deviation, potentially indicating measurement error, unique events, or important discoveries.

The ultimate goal of descriptive statistics in IB AI is interpreting data in real-world contexts for informed decision-making. For instance, a company might analyze customer age distribution (histogram) and median purchase value (box plot) to target marketing campaigns. A medical researcher might use cumulative frequency to determine what percentage of patients responded to a treatment within a certain time frame. Your interpretation must connect the statistical findings—the shape of the distribution, the center, the spread, and the presence of outliers—back to the original context to draw valid, meaningful conclusions.

Common Pitfalls

1. Misusing Charts: Using a bar chart for continuous data or mislabeling a histogram's axes. Remember, histograms are for grouped quantitative data, and the bars must be adjacent. Always label axes clearly with the variable and units.
2. Overreliance on the Mean: Automatically using the mean to describe the center of a skewed distribution or one with outliers. For data like house prices or income, the median is almost always a more representative measure of a "typical" value because it is not pulled by extreme figures.
3. Ignoring Spread When Comparing: Stating that two datasets are different based solely on a difference in means. If the spreads (IQRs) are large and overlap significantly, the difference in means may not be practically important. Always consider center and spread together.
4. Misidentifying Outliers: Forgetting to investigate outliers before deleting them. An outlier is not an error by default; it may be the most important data point. Your first step should be to check for measurement or entry mistakes. If none exist, you must consider what the outlier represents in the real-world context.

Summary

• Descriptive statistics transform raw data into understandable summaries through frequency tables, visualizations like histograms, cumulative frequency curves, and box-and-whisker plots, and numerical measures.
• The choice of measures of central tendency (mean, median, mode) and spread (range, IQR) depends on the data type and distribution; the median and IQR are resistant to outliers.
• Outliers should be identified using the $1.5\times IQR$ rule and investigated contextually, not simply removed.
• All statistical analysis must culminate in a clear interpretation that links the mathematical results back to the original real-world problem to support informed decision-making.
• A box plot provides a powerful, standardized five-number summary (min, $Q_1$, median, $Q_3$, max) that allows for quick comparison between different datasets.