Probability and Statistics

Probability contributes a dependable share of the 13-mark Engineering Mathematics block, and it also underpins average-case analysis in Algorithms, hashing in Data Structures and load modelling in Operating Systems. It is worth more than its own marks.

Two questions settle almost every problem: what is the sample space, and what is being conditioned on.

The sample space is the set of outcomes being counted, and getting it wrong is the source of most errors — not the arithmetic. Conditioning is what shrinks that sample space, and a conditional probability is nothing more than a probability computed inside a smaller world.

Every named distribution in the syllabus is a shortcut for a sample space too large to enumerate. Binomial counts successes in a fixed number of trials, Poisson counts events in an interval, exponential measures waiting time. Recognising which one the stem describes is the whole of the modelling step, and it is decided by what is being counted or measured, not by the wording.

The third principle is that expectation is linear whether or not variables are independent, while variance adds only when they are. That asymmetry is examined directly and is the most useful single fact in the chapter.

1. Sample Spaces and the Axioms

A probability assigns each event a number between 0 and 1, with the whole sample space getting 1, and with probabilities of disjoint events adding.

The addition rule corrects for double counting:

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$

For mutually exclusive events the intersection is empty and the correction vanishes, which is the special case, not the rule.

The complement rule is the workhorse. Whenever a question says "at least one", compute the probability of none and subtract from 1, because "none" is a single case while "at least one" is many.

For equally likely outcomes, probability reduces to counting, and the counting techniques from Quantitative Aptitude apply unchanged: block adjacent items, count complements, and choose combinations when order does not matter.

2. Conditional Probability and Independence

Conditional probability restricts attention to a smaller sample space:

$P(A\mid B)=\frac{P(A\cap B)}{P(B)}$

defined whenever $P(B)>0$. Reading it as "the fraction of $B$ that is also $A$" makes most questions immediate.

Rearranging gives the multiplication rule $P(A\cap B)=P(A\mid B)P(B)$, which chains for several events.

Two events are independent when $P(A\cap B)=P(A)P(B)$, equivalently when $P(A\mid B)=P(A)$. Independence means knowing one tells you nothing about the other.

Independent is not the same as mutually exclusive, and the two are almost opposites. If $A$ and $B$ are mutually exclusive with non-zero probabilities, then knowing $A$ occurred tells you $B$ did not, so they are strongly dependent. Confusing these is the commonest conceptual error in the topic.

Pairwise independence does not imply mutual independence. Three events can be independent in every pair while the triple fails, which is a standard counterexample question.

3. Bayes' Theorem

Bayes' theorem reverses the direction of conditioning:

$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}$

The denominator is usually expanded by the law of total probability over a partition:

$P(B)={\sum }_{i}P(B\mid A_i)P(A_i)$

The recurring exam shape is a test with known accuracy applied to a population with a known base rate, asking for the probability that a positive result is genuine.

The answer is almost always far lower than intuition suggests, and the reason is the base rate. If a disease affects 1 in 1000 and a test is 99 per cent accurate, then among 1000 people there is roughly 1 true positive and about 10 false positives, so a positive result is genuine only about 9 per cent of the time.

Working the problem in counts rather than probabilities makes this transparent and is faster under exam conditions. Choose a convenient population size, tabulate the four cells, and read the answer off.

4. Random Variables and Expectation

A random variable assigns a number to each outcome. Discrete variables have a probability mass function; continuous ones have a density whose integral over a range gives the probability.

For a continuous random variable, the probability of any single value is zero, so strict and non-strict inequalities give the same probability. This does not hold in the discrete case, and mixing them up costs marks.

Expectation is a weighted average:

$E[X]={\sum }_i{x}_i{p}_{i}E[X]=\int xf(x)dx$

Linearity of expectation holds always, whether or not the variables are independent, and whether or not they are even defined on related experiments:

$E[aX+bY]=aE[X]+bE[Y]$

This is the single most useful fact in the chapter. Problems that look intractable become one-line computations when a complicated random variable is written as a sum of simple indicator variables, each contributing its own probability.

For a function of a random variable, $E[g(X)]=\sum g(x_i)p_i$, and note that $E[g(X)]$ is generally not $g(E[X])$. The two agree only for linear $g$.

5. Variance

Variance measures spread and has a computational form that is almost always faster than the definition:

$\text{Var}(X)=E[X^2]-(E[X]{)}^2$

Scaling behaves differently from expectation. $\text{Var}(aX+b)=a^2\text{Var}(X)$, so the additive constant disappears entirely and the multiplier is squared.

Variance adds for independent variables only:

$\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)\text{if }X,Y\text{ independent}$

In general the covariance term appears, and $\text{Var}(X-Y)$ is still the sum of the variances for independent variables, not the difference — a standard trap.

The standard deviation is the square root of the variance and carries the same units as the variable, which is why it is quoted in practice.

Covariance measures how two variables move together:

$\text{Cov}(X,Y)=E[XY]-E[X]E[Y]$

Independence forces the covariance to zero, but zero covariance does not force independence. A variable and its own square on a symmetric range have zero covariance and are obviously dependent, which is the standard counterexample.

The correlation coefficient rescales covariance to lie between $-1$ and $1$ by dividing by the two standard deviations, which makes it dimensionless and comparable across problems.

Chebyshev's inequality bounds how much mass can sit far from the mean, using nothing but the variance: the probability of being more than $k$ standard deviations away is at most $1/k^2$.

It holds for every distribution, which is precisely why the bound is loose. For a normal distribution the true probability beyond two standard deviations is about 5 per cent, while Chebyshev only guarantees at most 25 per cent.

Markov's inequality is weaker still but needs only a non-negative variable and its mean, bounding $P(X\ge a)$ by $E[X]/a$. Both appear when a question supplies a mean and variance but no distribution.

6. Discrete Distributions

Four discrete distributions cover the syllabus, and each is identified by what it counts.

The Poisson distribution has mean equal to variance, which is its identifying signature and is asked directly.

The binomial requires a fixed number of independent trials with constant success probability. If the trials are dependent — sampling without replacement, for instance — the distribution is hypergeometric, not binomial.

The Poisson arises as the limit of the binomial when $n$ is large and $p$ small with $np=\lambda$ held fixed, which is why it models rare events over an interval: arrivals at a server, defects per wafer, packet losses per second.

The geometric distribution is memoryless: given that the first success has not occurred in the first $k$ trials, the distribution of the remaining wait is the same as the original. It is the only discrete distribution with this property.

7. Continuous Distributions

Three continuous distributions appear, and the exponential is the one examined most.

The exponential distribution has $P(X>t)=e^{-\lambda t}$, and it is memoryless, exactly like the geometric of which it is the continuous analogue:

$P(X>s+t\mid X>s)=P(X>t)$

This is why exponential service times make queueing analysis tractable, and it is the only continuous distribution with the property.

The exponential and Poisson describe the same process from two angles. If events arrive as a Poisson process at rate $\lambda$ per unit time, then the waiting time between consecutive events is exponential with the same $\lambda$.

The normal distribution is symmetric about its mean, and the standard normal is obtained by the transformation $Z=(X-\mu )/\sigma$. Roughly 68 per cent of the mass lies within one standard deviation, 95 per cent within two and 99.7 per cent within three.

A sum of independent normal variables is again normal, with means and variances adding, which is a property no other distribution in the syllabus shares in that form.

The central limit theorem explains why the normal appears so widely: the sum of a large number of independent variables with finite variance is approximately normal regardless of what the individual variables look like.

The practical consequence for GATE is that a question about the average of many independent trials can be treated as normal even when each trial is a coin toss, which is what makes the standard-deviation percentages usable.

8. Descriptive Statistics

The mean is the arithmetic average, the median is the middle value when sorted, and the mode is the most frequent value.

The median is robust to outliers and the mean is not, which is why questions supply a data set with one extreme value and ask which measure changes.

For a symmetric unimodal distribution all three coincide. For a right-skewed distribution the mean exceeds the median, which exceeds the mode, and the ordering reverses for left skew.

Population variance divides by $n$ while sample variance divides by $n-1$. The distinction is examined, and the $n-1$ correction exists because using the sample mean rather than the true mean systematically underestimates spread.

Linear transformations of a data set behave predictably, and the asymmetry mirrors the one for random variables.

Adding a constant shifts the mean and leaves the standard deviation unchanged; multiplying by a constant scales both. So adding 10 to every value raises the mean by 10 and the variance not at all, while doubling every value doubles the mean and quadruples the variance.

Quartiles split sorted data into four equal parts, and the interquartile range is the third quartile minus the first. Like the median, it ignores the tails entirely, which is why it is reported alongside the median whenever outliers are a concern.

9. Worked Examples

Example 1. A fair die is rolled twice. What is the probability that the sum is 8, given that the first roll is 5?

Conditioning shrinks the sample space to the six outcomes where the first roll is 5.

Within that world, the sum is 8 exactly when the second roll is 3, which is one outcome.

So the conditional probability is $1/6$.

Compare this with the unconditional probability of a sum of 8, which is $5/36$ — the pairs $(2,6),(3,5),(4,4),(5,3),(6,2)$. The two differ, which is precisely the statement that the events are not independent.

Example 2. A disease affects 1 per cent of a population. A test detects it with probability 0.99 in an affected person and gives a false positive with probability 0.05. If a person tests positive, what is the probability they have the disease?

Work in counts on a convenient population of 10,000, which is faster and far less error-prone than manipulating probabilities.

Affected: 1 per cent of 10,000 = 100 people. Of these, 99 test positive.

Unaffected: 9,900 people. Of these, 5 per cent = 495 test positive falsely.

Total positives: $99+495=594$.

The probability that a positive is genuine is $99/594\approx 0.167$, about 17 per cent.

The answer is far below the test's 99 per cent sensitivity because the unaffected group is 99 times larger, so even a small false-positive rate produces many more false positives than there are true cases. This base-rate effect is what the question exists to test.

Example 3. A fair coin is tossed 10 times. What are the expected number of heads and the variance?

Each toss is a Bernoulli trial with $p=1/2$, and the tosses are independent with a fixed count, so the total is binomial with $n=10$ and $p=1/2$.

Mean: $np=10\times 0.5=5$.

Variance: $np(1-p)=10\times 0.5\times 0.5=2.5$.

The standard deviation is $\sqrt{2.5}\approx 1.58$.

The linearity route gives the mean without the binomial formula: write the total as a sum of 10 indicator variables, each with expectation $1/2$, and add. That route also works when the coins have different biases, where the binomial formula does not apply but linearity still does.

Example 4. Packets arrive at a router as a Poisson process at 3 per second. What is the probability that exactly 2 arrive in a given second, and what is the expected gap between consecutive arrivals?

For the count, use the Poisson distribution with $\lambda =3$:

$P(X=2)=\frac{e^{-3}\cdot 3^2}{2!}=\frac{9e^{-3}}{2}\approx 0.224$

For the gap, use the exponential distribution, which describes inter-arrival times of the same process with the same rate.

The mean of an exponential with rate $\lambda$ is $1/\lambda$, so the expected gap is $1/3$ of a second.

The pairing is the point of the question: the same $\lambda$ serves as a count parameter for the Poisson and a rate parameter for the exponential, and recognising that they describe one process from two angles saves modelling time.

Example 5. $X$ and $Y$ are independent with $\text{Var}(X)=4$ and $\text{Var}(Y)=9$. Find $\text{Var}(2X-3Y+5)$.

Handle the constant first: adding 5 shifts the distribution without changing its spread, so it contributes nothing.

Handle the multipliers: each is squared.

$\text{Var}(2X-3Y+5)=4\text{Var}(X)+9\text{Var}(Y)=4(4)+9(9)=16+81=97$

Two things are worth noting. The coefficient $-3$ contributes $(-3{)}^2=9$, so the sign disappears and the variances add rather than subtracting. And the independence assumption is what allows the two terms to be added at all; without it, a covariance term of $-2\times 2\times 3\times \text{Cov}(X,Y)$ would appear.

Example 6. A biased coin has $P(\text{head})=0.3$. It is tossed until the first head. What is the expected number of tosses, and what is the probability that more than 5 tosses are needed?

The number of tosses until the first success is geometric with $p=0.3$.

The expected number of tosses is $1/p=1/0.3\approx 3.33$.

For the tail probability, more than 5 tosses are needed exactly when the first 5 tosses are all tails.

$P(X>5)=(1-p{)}^5=(0.7{)}^5=0.168$

The memorylessness of the geometric distribution is what makes this so simple: no bookkeeping over which toss produced the success is needed, because the only way to need more than 5 tosses is for the first 5 to fail.

The same property means that having already tossed 5 tails, the expected number of further tosses is again $1/0.3=3.33$, not less. The coin has no memory of the failures.

Summary

Name the sample space and identify what is being conditioned on; a conditional probability is a probability inside a smaller world.

For "at least one", compute the complement.

Independent and mutually exclusive are almost opposites: mutually exclusive events with non-zero probability are strongly dependent. Pairwise independence does not give mutual independence.

Bayes' theorem reverses conditioning, and base-rate questions are answered fastest by tabulating counts in a convenient population.

Linearity of expectation holds always, independent or not, and writing a complicated variable as a sum of indicators is the standard technique.

Variance uses $E[X^2]-(E[X]{)}^2$, kills additive constants, squares multipliers, and adds only for independent variables — and it adds for a difference too.

Binomial counts successes in a fixed number of independent trials; Poisson counts events in an interval and has mean equal to variance; geometric counts trials to the first success and is memoryless.

Uniform, exponential and normal cover the continuous cases. The exponential is memoryless and is the inter-arrival time of a Poisson process with the same rate.

The median resists outliers and the mean does not. Sample variance divides by $n-1$ rather than $n$.