Statistics guide, Unit 1, step 1 of 22Previous step Next step

Step 1: Statistical Thinking

By the EconTutorials team. 10-minute read with 6 practice questions.

SyllabusAP Stats 1.1–1.2

In 30 seconds

Statistics is the science of learning from data: collecting it sensibly, describing it clearly and drawing honest conclusions that admit uncertainty.
You use a sample (which you can see) to learn about a population (which you can’t). A parameter describes the population; a statistic is calculated from the sample and estimates it.
Bias is a systematic distortion. A bigger sample does not fix it; random selection is your main defence.
Every study follows four steps: ask, collect, analyse, conclude. Exam marks come from naming the population and sample separately and never claiming more certainty than the data allows.

Video lesson coming soonA short animated lesson for this step will appear here.

What statistics really is

Ask your friends what “statistics” means and most will say “numbers”: football scores, unemployment rates, the “95% of users agree” line on a shampoo bottle. That answer is not wrong, but it misses the main point.

Imagine you run a small café and want to know whether customers like your new cinnamon bun. You can’t ask everyone who will ever walk in. So you ask 40 customers this week, count how many say “yes”, and then make a guess about all your customers. You have just done two things. First, you described what you measured (“28 out of 40 liked it”). Second, you concluded something beyond what you measured (“most of my customers probably like it”). That second step is where statistics becomes powerful, and also where it becomes tricky, because you might be wrong.

Key termStatistics: the science of learning from data, which means collecting data sensibly, describing it clearly and drawing trustworthy conclusions while being honest about how uncertain those conclusions are.

Think of it as detective work. You never see the whole truth directly. You get clues (data) and reason carefully from them to a conclusion, resisting the urge to see what you want to see.

The poll that got it spectacularly wrong

In 1936 an American magazine called The Literary Digest tried to predict the U.S. presidential election. It mailed a survey to about ten million people and got back around 2.4 million replies. That is an enormous sample. Based on it, the magazine predicted a comfortable win for Alf Landon over Franklin D. Roosevelt. Roosevelt won in a landslide, taking 46 of the 48 states.

How can more than two million replies be so wrong? The mailing list came from telephone directories, magazine subscribers and car-registration records. During the Great Depression, people who owned phones and cars and paid for magazines were wealthier than average, and wealthier voters leaned towards Landon. The poll was not too small. It reached the wrong mix of people. Keep this story in mind: how you collect data matters more than how much you collect.

Population and sample, parameter and statistic

Almost every statistical question is about a large group, but you can almost never look at the whole group. So you look at a slice and reason back to the whole.

Suppose you want to know how many hours undergraduates at your university sleep per night. Measuring every student is impractical, so you survey 200 of them. The 200 students are your sample; all undergraduates at the university are your population.

Key termPopulation: the entire collection of people, objects or events you want to know something about.

Key termSample: the smaller subset of the population that you actually collect data from.

There is a matching pair of words for the numbers involved.

Key termParameter: a number that describes the whole population, such as the true average sleep of all undergraduates; it is usually unknown.

Key termStatistic: a number calculated from your sample, such as the average sleep of your 200 students; it is known and is used to estimate the parameter.

A quick memory hook: population goes with parameter, sample goes with statistic. Same first letters, so they pair up.

Worked example

A quality manager wants to know what percentage of all smartphones from a production line are defective. She randomly selects 500 phones and finds 15 defective. Name the four parts and calculate the sample defect rate.

Step 1Population: all phones produced on that line. Sample: the 500 phones she tested.

Step 2Sample defect rate \( = \dfrac{15}{500} \times 100 = 0.03 \times 100 = 3\% \)

Step 3Statistic: the 3% she measured (known). Parameter: the true defect rate for the whole line (unknown).

The 3% is her best estimate of the unknown true rate. It probably won’t be exactly right, and later steps in the book show how to say how far off it might be.

We use a sample, which we can see, to learn about a population, which we can’t. Almost everything else in statistics is machinery for making that leap safely.

Common mistakeWriting “3% of the phones are defective” when you mean “3% of the sampled phones were defective, so about 3% of all phones probably are.” The first version quietly claims a certainty you don’t have. Name the sample and the population separately every time.

Descriptive versus inferential statistics

The whole subject splits neatly into two jobs. Go back to the sleep survey. If your 200 students average 6.4 hours, then saying “the average in my sample was 6.4 hours” simply reports your data. Saying “undergraduates here average about 6.4 hours of sleep, give or take” jumps from 200 students to all of them, and that jump needs justifying.

Key termDescriptive statistics: summarising and displaying the data you actually have, using averages, percentages, tables and graphs, with no claim beyond that data.

Key termInferential statistics: using sample data to draw conclusions about the wider population, with a stated level of confidence.

Here is a quick test you can use on any sentence: does it talk only about the data in front of you, or does it reach out to a bigger group?

Statement	Type	Why
“In our survey of 1,000 shoppers, 62% preferred the new packaging.”	Descriptive	A plain report of the sample.
“Based on the survey, a majority of all our customers prefer the new packaging.”	Inferential	A claim about the whole customer base.
“The tallest player on the team is 2.01 metres.”	Descriptive	A fact about the data at hand.

Where bias sneaks in

The Literary Digest story is a story about bias. Avoiding bias is most of what “collecting data sensibly” means, so it is worth naming clearly.

Key termBias: a systematic tendency for your data to misrepresent the population, pushing your results consistently in one direction however much data you gather.

The word systematic does the heavy lifting. Compare two problems:

Random error is bad luck. By chance you happen to survey a few unusually sleepy students. As your sample grows, these accidents tend to cancel out.
Bias is a fault in the method. If your method keeps reaching wealthy voters, collecting ten times more data just gives you a bigger, more confident wrong answer.

Think of a bathroom scale that always reads 2 kg too heavy. Weighing yourself a hundred times will not fix it; you will simply be very sure of the wrong number. That is bias.

The best defence is randomness: giving every member of the population a fair, known chance of being in your sample. Deliberately letting chance decide who is selected is one of the most important ideas in the subject.

Common mistakeBelieving that a huge dataset must be trustworthy. The Literary Digest had 2.4 million replies and still got it badly wrong. Size protects you from random error, not from bias. A small, representative sample beats a giant biased one.

The four-step investigative process

Statistics is best learned as a process rather than a pile of formulas. Almost every study, from a school project to a clinical trial, goes through the same four stages.

Ask a question. State clearly what you want to find out, in a form data could answer. (“Do students who use this textbook score higher on the final exam?”)
Collect data. Decide who or what to measure and how, guarding against bias at every turn. This step makes or breaks everything after it.
Analyse the data. Summarise it, draw graphs and apply the right descriptive and inferential methods.
Draw a conclusion. Answer the original question, say how confident you are and be honest about the limitations.

Notice that the formulas most people think of as “statistics” live only in step 3. A brilliant analysis of badly collected data is worthless, because you cannot calculate your way out of a biased sample.

The café loyalty card

A café owner thinks a new loyalty card is boosting repeat visits. Here is the process in action:

Ask: Do customers with the loyalty card visit more often than those without?
Collect: Randomly sample transactions over a month, recording visit frequency and whether the customer has a card. Be careful not to sample only regulars, which would bias the result.
Analyse: Compare the average number of visits for the two groups, draw a graph, then test whether the difference is bigger than chance alone would produce (a technique from later in the book).
Conclude: State the finding, attach a confidence level and note limits, such as one month possibly missing seasonal effects.

Exam tipWhen a question gives you a study, label every part out loud: population, sample, parameter, statistic. Then say whether a statement is descriptive or inferential and give a one-line reason. If you are asked to criticise a survey, use the word “bias” correctly, say who was over- or under-represented and explain why a larger sample would not help.

Practise: 6 questions

Try each one on paper first, then check the worked answer.

EasyQ1. Fill in the blanks: a number describing a whole population is a ______, while a number calculated from a sample is a ______. Show worked answer

A parameter; a statistic.
Memory hook: population goes with parameter, sample goes with statistic.

EasyQ2. Is each statement descriptive or inferential? (a) “The mean score of the 30 students in this class was 74%.” (b) “Because our sample of 500 voters favoured Candidate A, we believe Candidate A leads among all voters in the city.” Show worked answer

(a) Descriptive: it reports data for this particular class and claims nothing beyond it.
(b) Inferential: it generalises from 500 sampled voters to all voters in the city.

MediumQ3. True or false: a larger sample automatically removes bias. Explain. Show worked answer

False. A larger sample reduces random error, because chance accidents tend to cancel out. It does nothing about bias, because bias comes from the method and pushes results in the same direction every time. If the selection method is skewed, a bigger sample just gives a more confident wrong answer, as in the Literary Digest poll.

MediumQ4. A researcher wants to estimate the average daily screen time of teenagers in a country. She randomly selects 2,000 teenagers and finds their average is 5.2 hours. Identify the population, the sample, the parameter and the statistic. Show worked answer

Population: all teenagers in the country.
Sample: the 2,000 randomly selected teenagers.
Parameter: the true average screen time of all teenagers in the country (unknown).
Statistic: 5.2 hours, the average calculated from her sample (known).

MediumQ5. A fitness app claims: “Users who logged workouts lost more weight, so our app causes weight loss.” Give one reason this conclusion might not be justified, thinking about how the data was collected. Show worked answer

The data was observed, not created by randomly assigning people to use the app. People who choose to log workouts are probably more motivated or disciplined to begin with, and that motivation could explain the weight loss. So the app may not be the cause (this is self-selection; Chapter 5 of the book returns to correlation versus causation).

ExamQ6. In 1936 the Literary Digest received about 2.4 million replies to its election survey but predicted the wrong winner. Explain why such a large survey failed. [4 marks] Show worked answer

Mark scheme (1 mark each):
1. The mailing list came from telephone directories, magazine subscribers and car-registration records.
2. In 1936 these people were wealthier than average, and wealthier voters leaned towards Landon, so the sample did not represent the people who would actually vote.
3. This is bias: a systematic distortion that pushes results in one direction.
4. Bias does not shrink as the sample grows; a larger sample only reduces random error, so 2.4 million biased replies still gave the wrong answer. A smaller random sample would have done better.

Check yourself

1. The average sleep of the 200 students you surveyed is an example of a…

It is calculated from the sample, so it is a statistic; the true average of all students would be the parameter.

2. What is the single best defence against bias when choosing a sample?

Random selection gives everyone a fair, known chance of being chosen; size alone cannot remove bias.

3. In which step of the investigative process do the familiar formulas appear?

Calculations belong to step 3, and they are only as good as the data produced in steps 1 and 2.

Your score: 0 out of 3

The short version

Statistics is about reasoning from data, not just about numbers: collect sensibly, describe clearly, conclude honestly.
Population and parameter go together; sample and statistic go together. A statistic estimates an unknown parameter.
Descriptive statistics reports the data you have; inferential statistics generalises to a wider population and needs justifying.
Bias is systematic and more data cannot fix it. Random selection is the main defence.
Ask, collect, analyse, conclude. The full chapter in Statistics Made Simple includes 20 practice questions with a complete answer key.