AP Statistics: Experimental Design FRQ Questions

Mastering experimental design is not just about passing the AP Statistics exam—it’s about learning to think like a scientist. The Free Response Questions (FRQs) on this topic require you to translate statistical principles into clear, logical procedures, a skill essential for evaluating real-world research. Success hinges on your ability to design, critique, and explain experiments with precision.

Core Components of an Experiment

Every experiment is built from a set of fundamental parts. Precisely identifying these components is the first and most critical step in answering any experimental design FRQ.

The experimental units are the smallest collection of material to which a treatment is independently applied. In a drug trial, the experimental unit is an individual patient; in an agricultural study, it might be a plot of land. The specific conditions applied to these units are the treatments. For example, if you are testing three different fertilizers and a control group (no fertilizer), you have four distinct treatments. The outcome you measure to evaluate the treatments is the response variable. This must be a quantifiable measure, like crop yield in kilograms or patient recovery time in days.

Crucially, you must distinguish between random assignment and random selection. Random selection is a sampling method used to choose a representative sample from a larger population, which is the focus of surveys. Random assignment is the cornerstone of experiments. It is the process of using a chance mechanism, like a random number generator or drawing names from a hat, to assign experimental units to different treatment groups. Its sole purpose is to create treatment groups that are roughly equivalent in all aspects before the treatments are applied. This helps control for confounding variables—extraneous factors that are associated with both the treatment and the response, making it impossible to determine if differences in the response are due to the treatment or the confounder. Random assignment spreads these potential confounding variables evenly across groups.

Foundational Design Principles: Randomization, Replication, and Control

A well-designed experiment actively implements three key principles to ensure its results are valid and generalizable.

Randomization, through random assignment, is your primary defense against confounding and bias. When describing this procedure in an FRQ, you must be specific. A strong answer goes beyond "I randomly assigned subjects." Instead, you might write: "Number the 60 patients from 1 to 60. Use a random number generator to select 30 unique numbers. Those patients are assigned to the new therapy (Treatment A). The remaining 30 patients are assigned to the standard therapy (Treatment B)." This specificity demonstrates understanding.

Replication refers to applying each treatment to multiple experimental units. Having only one unit per treatment tells you nothing about the variation in the response. Replication allows you to estimate this inherent variability, which is essential for determining if observed differences between treatment groups are statistically significant or just due to random chance. In essence, replication gives you a measure of reliability.

Control involves comparing treatment groups to a baseline. This can be a control group that receives a placebo or standard treatment, or it can involve controlling environmental conditions. For instance, in a plant growth experiment, controlling for sunlight and water ensures that any differences in growth can be more confidently attributed to the fertilizer treatments alone.

Advanced Design: Blocking and Matched Pairs

When you are aware of a specific variable that is likely to influence the response, you can use a more sophisticated design to reduce variability and increase the precision of your experiment.

Blocking involves grouping experimental units that are similar with respect to this nuisance variable (the blocking variable) and then randomizing within those groups. Suppose you are testing a new teaching method on student test scores. You know that prior academic performance (e.g., honors vs. standard track) strongly affects scores. To block on this variable, you would first separate students into two blocks: "honors" and "standard." Then, within each block, you would randomly assign half the students to the new method and half to the standard method. This ensures each teaching method is tested on comparable students, making the comparison within each block more precise. Blocking controls for known sources of variation to increase the sensitivity of the experiment.

A matched-pairs design is a special case of blocking where blocks are of size two. This is common when experimental units are naturally paired (like twins, or two eyes of the same person) or when units are paired based on similar characteristics. One member of each pair is randomly assigned to Treatment A, and the other to Treatment B. This design is particularly powerful for controlling for many confounding variables at once, as the pairs are very similar to each other.

Synthesis and FRQ Strategy

An experimental design FRQ will often present a scenario and ask you to "describe a completely randomized design" or "design a blocked experiment." Your response must be a coherent, step-by-step procedure.

1. Identify and Define: Start by explicitly stating the experimental units, treatments, and response variable. This sets the stage.
2. Detail the Procedure: Write your steps as instructions for someone to follow.

• For a completely randomized design: "Number all experimental units from 1 to N. Use a random number generator to assign each unit to a treatment group until the predetermined group sizes are met."
• For a randomized block design: "First, divide the units into blocks based on [blocking variable]. Within each block, number the units. Use a random number generator to randomly assign the units within each block to the treatment groups."

1. Explain the 'Why': Weave in explanations for your choices. For example, "Blocking on soil type is used here because it is a major known factor affecting plant growth. By randomizing within blocks, we ensure the comparison between fertilizers is fair within each soil type, reducing variability in the response."
2. Connect to Conclusion: A final, powerful sentence can tie your design back to inference: "This random assignment process creates comparable groups, allowing us to conclude that any significant difference in the mean response is caused by the treatment."

Common Pitfalls

Mistake 1: Confusing Random Selection with Random Assignment. Students often write, "Randomly select 30 subjects for the treatment group." This phrasing describes sampling, not the assignment to treatments needed for an experiment. This error shows a fundamental misunderstanding. Correction: Always use "randomly assign." The selection of subjects into the study sample is a separate step that may or may not be random, but the assignment to treatments within the experiment must be.

Mistake 2: Vague Randomization Description. Writing "I randomly put them into groups" earns no credit. The procedure is not replicable. Correction: Be mechanistic. Describe the tool (random number generator, random digit table, drawing slips of paper) and the specific steps (e.g., "assign numbers," "generate numbers," "assign based on even/odd" or a range of numbers).

Mistake 3: Misidentifying the Experimental Unit. In a study where multiple plants in one pot receive the same fertilizer treatment, the experimental unit is the pot, not the individual plant, because the treatment is applied to the whole pot. The plants within are subsamples. Analyzing them as independent units violates the assumption of independence. Correction: Carefully ask: "To what is the treatment directly and independently applied?" The answer is your experimental unit.

Mistake 4: Forgetting the Purpose of Principles. It’s not enough to say you will block; you must state what you are blocking on and why. Correction: Always pair the implementation of a design principle with its purpose. "We block by gender to account for differences in metabolism that could confound the results of the drug trial."

Summary

• The key feature of an experiment is random assignment, which helps eliminate confounding by creating roughly equivalent treatment groups before the intervention.
• Clearly and specifically identify the experimental units, treatments, and response variable as the foundation for any design description.
• Implement and explain the core principles: randomization (to avoid bias), replication (to measure variability), and control (to establish a baseline for comparison).
• Use blocking for known, influential sources of variation to increase the precision of your experiment. A matched-pairs design is an effective form of blocking with pairs of units.
• In your FRQ response, write a detailed, step-by-step procedure that another person could follow, and explicitly justify your design choices by linking them to the goal of establishing a cause-and-effect relationship.