Statistics and Probability
A test for a rare disease is per cent accurate, in both directions: it catches per cent of those who have it and clears per cent of those who do not. The disease affects one person in ten thousand.
You test positive. What is the chance you have it?
The number in the question is per cent, so that is the answer everyone gives.
Take a million people and count.
| Group | How many | Test positive |
|---|---|---|
| Have the disease | ||
| Do not | ||
| Total positives |
Not per cent. About one per cent, a hundredfold error.
The healthy group is ten thousand times larger, so even a one per cent error rate applied to it produces a hundred times more false positives than there are true ones. The test's accuracy was never the answer, because it describes and the question asked for . Those are different numbers, and Bayes' theorem is the machinery for turning one into the other.
The statistics half of the chapter fails in the same way. Two classes both average out of .
Same mean, and nothing else in common: the first has standard deviation , the second . One number described both and distinguished neither.
Every question in this chapter reduces a lot of information to one number, and the marks go to noticing what that number leaves out.
| The number | What it hides |
|---|---|
| the mean | the spread, entirely |
| it is not | |
| "mutually exclusive" | this is not the same as independent |
1. Central Tendency
| Measure | Definition | Best when |
|---|---|---|
| Mean | data is symmetric, no extreme values | |
| Median | the middle value in order | outliers are present |
| Mode | the most frequent value | data is categorical |
For grouped data the median and mode need interpolation inside the modal or median class.
Here is the lower boundary of the relevant class, its width, the cumulative frequency before it, and the frequencies of the previous, modal and next classes.
Trap. The mean is dragged by a single extreme value and the median is not. A dataset of nine salaries near thirty thousand and one of ten million has a mean that describes nobody in it.
Illustration 1
For the distribution below, find the mean, median and mode.
| Class | – | – | – | – | – |
|---|---|---|---|---|---|
| Frequency |
Mean uses class midpoints and .
Median needs the cumulative frequencies . Half of is , which first exceeds a cumulative total inside the class –.
Mode sits in the class with the highest frequency, –, with and on either side.
The three answers differ, and their ordering carries information: mean below median below mode signals a distribution with a longer tail to the left. A question asking which measure best represents the data is asking you to read exactly that.
2. Measures of Dispersion
Squaring rather than taking a modulus is what makes the variance algebraically tractable, and it produces the shortcut that does almost all the work.
Mean of the squares minus the square of the mean. Standard deviation is , in the same units as the data; variance is , in squared units.
Illustration 2
Find the variance and standard deviation of , both directly and by the shortcut.
Check directly. The deviations are , whose squares total .
The shortcut needed no deviations at all, which matters when the mean is not a whole number: subtracting from every value is where arithmetic errors come from, and the shortcut avoids it entirely.
3. Transformations and Combined Data
| Change to every value | Effect on mean | Effect on variance |
|---|---|---|
| add | increases by | unchanged |
| multiply by | multiplied by | multiplied by |
Shifting the whole dataset moves it without stretching it, so the spread cannot change. Scaling stretches every deviation by , and variance is built from squared deviations.
For two groups combined, the variance picks up the gap between the group means as well as the variances within them.
Illustration 3
The data of Illustration 1 has mean and variance . Find the mean and variance after adding to each value, and after multiplying each by .
Adding shifts the whole set without stretching it.
Multiplying stretches every deviation threefold, and variance squares deviations.
The standard deviation, being the square root, is only tripled: . That is the practical reason to keep standard deviation and variance apart in your head — one scales by , the other by .
Illustration 4
Group A has values with mean and variance . Group B has values with mean and variance . Find the variance of the combined data.
The combined mean comes first, because the gaps are measured from it.
Notice that exceeds both group variances, and . Combining groups whose means differ always increases the spread, because the separation between the means becomes part of the variation in the pooled data. Averaging the two variances to get is the standard error, and it is always too small.
4. Probability: Events and the Axioms
That counting definition needs the outcomes to be equally likely, which is a real assumption and not a formality.
The complement rule is the most under-used line in the chapter. "At least one" is almost always faster as one minus the probability of none.
5. Addition Theorem, and Two Words That Are Not Synonyms
The subtraction is inclusion-exclusion again, correcting for the overlap counted twice.
| Term | Condition | Meaning |
|---|---|---|
| Mutually exclusive | they cannot both happen | |
| Independent | one happening tells you nothing about the other |
Illustration 5
Show that two events with non-zero probability cannot be both mutually exclusive and independent.
Suppose they were. Exclusivity gives , and independence gives .
But both probabilities were assumed non-zero, so their product is non-zero. Contradiction.
The intuition is sharper than the algebra. Mutually exclusive events are maximally dependent: learning that happened tells you certainly did not, which is the largest possible amount of information one event can carry about another. Independence means learning tells you nothing at all. The two are opposite ends of the same scale, not two names for the same idea.
Illustration 6
A card is drawn from a standard pack. Find the probability that it is a king or a heart.
The two events overlap in exactly one card, the king of hearts, and that card would otherwise be counted twice.
Are they independent? Test the definition rather than guessing.
They are, which surprises most people. Being a heart really does not change the chance of being a king, because each suit contains exactly one king. Change the pack by removing one non-heart king and independence breaks immediately.
6. Conditional Probability
Conditioning shrinks the sample space to and re-measures inside it. That is the entire content of the definition, and reading it that way solves most questions without algebra.
Illustration 7
A family has two children. Given that at least one is a boy, what is the probability that both are?
The reflex answer is , reasoning that the other child is equally likely to be either.
List the sample space, in birth order, and then shrink it.
"At least one boy" removes and leaves three equally likely outcomes.
The reflex fails because "at least one is a boy" does not identify which child, so it eliminates one outcome rather than fixing one child's sex. Contrast the different question "the elder is a boy", which leaves and does give .
Trap. Read exactly what the condition specifies. "At least one", "the first one" and "a randomly chosen one" produce three different answers to the same-sounding question.
7. Bayes' Theorem
The denominator is the total probability of , assembled from every way it could happen. The theorem reverses the conditioning: it converts , which experiments give you, into , which is what you actually want to know.
The prior is what people leave out. In the opening example the prior was one in ten thousand, and ignoring it moved the answer by a factor of a hundred.
Illustration 8
Bag A holds red and black balls; bag B holds red and black. A bag is chosen at random and a red ball is drawn. What is the probability it came from bag B?
Set out the priors and the conditionals before combining anything.
The denominator is every route to a red ball.
Slightly above a half, because bag B has the marginally higher proportion of red: against . The evidence shifted the answer from the prior of only a little, which is what weak evidence should do.
8. Random Variables, Mean and Variance
A random variable assigns a number to each outcome. Its probability distribution lists the values with their probabilities, which must sum to .
The variance formula is the same "mean of the squares minus square of the mean" as in the statistics half, and that is not a coincidence: a probability distribution is a dataset with the frequencies replaced by probabilities.
Illustration 9
Find the mean and variance of the score on one throw of a fair die.
Each value has probability , so both sums are straightforward.
The mean of is a value the die can never show, which is worth noticing: an expected value is a long-run average, not a prediction of any single throw.
9. Bernoulli Trials and the Binomial Distribution
Bernoulli trials are repeated, independent, with two outcomes and a constant probability of success.
The binomial coefficient counts the orderings in which successes can occur, which is the Binomial Theorem's counting argument reappearing in a new setting.
Since , the variance is always less than the mean for a binomial distribution. That is a fast check on any answer.
Illustration 10
A fair die is thrown times. Find the probability of exactly sixes, and the mean and variance of the number of sixes.
The trials are independent with constant , so the binomial applies.
The variance is below the mean, as it must be. The mean under also matches intuition: five throws will most often produce no six at all.
Illustration 11
A die is thrown times. Find the probability of at least one six.
Listing the cases for one, two, three, four and five sixes means five binomial terms. The complement needs one.
Just under per cent, which is worth remembering as a corrective: five throws is not enough to make a six likely-but-certain, and the intuition that six throws should "guarantee" one is wrong too, giving only about per cent.
Trap. "At least one" almost always means take the complement. Adding the individual cases gives the same answer and takes five times as long, with five times as many chances to slip.
10. A Note on Syllabus Emphasis
The unit is titled Measures of Dispersion and Probability, and the statistics half is exactly that: means, medians and modes of grouped and ungrouped data, and mean deviation, variance and standard deviation for both.
| Where the marks are | What it looks like |
|---|---|
| variance and standard deviation | one computation, often with a transformation attached |
| conditional probability and Bayes | the most examined idea in the unit |
| binomial distribution | mean, variance, and one specified probability |
Probability carries most of the weight, and within it Bayes' theorem carries most again. The statistics half is short, formulaic and worth securing completely, because the questions are nearly identical from year to year.
Summary
Every question here reduces a lot of information to one number, and the marks go to noticing what that number leaves out.
A per cent accurate test for a one-in-ten-thousand disease gives a positive result that means about a one per cent chance, because the healthy group is vastly larger. is not .
Two datasets can share a mean and share nothing else. The mean hides the spread completely.
For grouped data, the median and mode are interpolated inside their classes, and mean below median below mode signals a left tail.
Variance is the mean of the squares minus the square of the mean, which avoids computing any deviations.
Adding a constant leaves the variance unchanged; multiplying by multiplies it by , while the standard deviation is multiplied by .
Combining groups with different means always increases the variance, because the gap between the means becomes part of the spread.
Mutually exclusive and independent are opposite ends of one scale, and events with non-zero probability can never be both.
Conditioning shrinks the sample space and re-measures inside it, which is why "at least one is a boy" gives and "the elder is a boy" gives .
Bayes' theorem reverses the conditioning, and the prior is what people leave out.
An expected value is a long-run average and need not be an attainable value, as the die's shows.
For a binomial distribution the mean is and the variance , so the variance is always the smaller of the two.
"At least one" almost always means one minus the probability of none.
