What it is
The population is the whole group you want to know about. It need not consist of people: all the screws a machine makes in a week, all the pupils at a school and all the readings a thermometer could give are populations. A sample is the part of the population that is actually examined. Examining the whole population is called a census.
A sample is taken when the population is too large, too costly or impossible to examine in full. A test that pulls a screw until it breaks destroys it, so not every screw can be tested.
| Population | Sample | |
|---|---|---|
| Size | N | n |
| Mean | μ | x̄ |
| Standard deviation | σ | s |
| Proportion | p | p̂ |
The population’s numbers are fixed but usually unknown. The sample’s numbers are calculated from the data and vary from sample to sample.
A school has 600 pupils. 60 pupils chosen at random are asked how they get to school, and 21 cycle. The sample proportion is p̂ = 21/60 = 0.35. Another random sample of 60 pupils might give 18 cyclists (0.30) or 24 (0.40), and the proportion for the whole school is a single number that none of them need hit. How far off a sample tends to land is worked out with the confidence interval for a proportion: for 21 out of 60, the 95% interval runs from about 0.23 to 0.47.
How the sample is chosen decides what can be said:
- Simple random sample. Everyone has the same chance of being included, for example when 60 names are drawn from a list of all 600. The formulas for the standard error and the confidence interval assume a sample like this.
- Stratified sample. The population is split into groups, such as year groups, and a random sample is drawn from each group in proportion to its size.
- Systematic sample. Every tenth name on a list, from a randomly chosen starting name.
- Cluster sample. Whole groups, such as classes, are chosen at random.
- Convenience and self-selected samples. Those who are easy to reach, such as the first 60 in the lunch queue, or those who choose to answer, as in an online poll.
The first four rely on chance, and there are methods for calculating their uncertainty. The last does not, and then the formulas have nothing to stand on.
Sources of error. The difference between sample and population has two kinds of cause. Sampling error is the random variation between samples, and it shrinks as the sample grows. Systematic errors do not shrink with sample size: the people chosen are not like the population, those who do not answer differ from those who do (non-response), the question steers the answer, or the measurement is wrong.
A large sample does not fix a biased one. Before the 1936 US presidential election, the magazine Literary Digest sent questionnaires to 10 million people and received 2.38 million replies. It predicted 57% for Landon, but Roosevelt won by a wide margin. The addresses came from subscribers, car owners and telephone subscribers, groups that were richer than average during the Depression of the 1930s, and according to later analyses non-response was the main cause of the error.
Why it matters
- Statistics about large groups usually rest on samples. Asking everyone takes too long or costs too much, and sometimes measuring destroys what is measured.
- Which formula applies. If the values are the whole group, the population standard deviation σ and variance are used, with n in the denominator. If they are a sample, the sample standard deviation s and the sample variance are used, with n − 1. The list 2, 4, 4, 4, 5, 5, 7, 9 gives σ = 2 as a population and s ≈ 2.138 as a sample. The statistics calculator shows both.
- How far from the true value the result can be. The standard error s/√n describes how much a sample mean varies, and the confidence interval is built on it.
- How large the sample should be. At 95% confidence and a proportion of 0.5, the worst case, 1,068 answers are needed for ±3 percentage points and 385 for ±5; see sample size for a proportion and sample size for a mean. The size of the population does not matter as long as it is much larger than the sample.
- What the conclusion covers. A conclusion holds for the population the sample was drawn from. A survey of the pupils at one school does not tell you what pupils at other schools think. Two samples are compared with the p-value calculator.
Common misconceptions
- “A larger sample is better.” A larger sample reduces random variation but not bias in how it was chosen. Literary Digest had 2.38 million replies and still got it wrong.
- “The sample should be a certain percentage of the population.” The uncertainty depends on the size of the sample, not its share of the population, as long as the population is much larger. 1,000 answers give about the same uncertainty for a population of 100,000 as for one of 10,000,000.
- “The sample mean is the population mean.” It is an estimate, and a new sample gives a different number.
- “Those who answered are the sample.” The sample is those who were chosen. Those who answer are some of them, and if those who do not answer differ from the rest, the result is biased.
- “A random sample is representative.” A random sample can also turn out skewed, especially when it is small. What you gain is that you can calculate how large the skew tends to be.
Common questions
When is a census better than a sample?
When the population is small, like a class of 25 pupils, and it is just as easy to measure everyone. Then there is
no sampling uncertainty to calculate. A sample pays off when the population is large or when measuring destroys
what is measured.
How large should the sample be?
That depends on how much uncertainty you accept. The sample-size formulas work it out for a chosen margin of error
and confidence level. Halving the margin of error takes four times as many answers.
Why divide by n − 1 for a sample?
The mean is calculated from the same sample, so the sum of squares around it is slightly too small on average.
The denominator n − 1 makes up for that; see the
sample variance.