Expectation & Variance

E[X] is a random variable's long-run average; Var[X] is its spread.

Maths · Intermediate · ~4 min

In plain English

Expectation is what you'd get on average over many tries. Variance is how much any single try tends to differ from that average.

Why it's worth your time

It's the language every loss function, every A/B test and every bias-variance argument is written in.

If you remember three things

E[aX + b] = aE[X] + b — expectation is linear, always
Var(aX) = a²Var(X) — variance scales with the square
Variance adds for independent variables; covariance appears when they aren't

Overview

Expectation E[X] is a random variable's probability-weighted average — the value you'd converge to over many draws. Variance measures its spread around that mean, and its square root is the standard deviation. Linearity of expectation lets you add expectations freely, while variances add only when the variables are independent.

In an interview

E[X] is the long-run average of a random variable: each value weighted by its probability, then summed. Variance is the expected squared distance from that mean. The rule to remember is linearity of expectation — E[X+Y]=E[X]+E[Y] always, even for dependent variables — whereas variances add only when the variables are independent.

Production defaults

Averaging n samples: cuts the standard error by √n — the reason ensembles and larger test sets work
Sample size: to halve your uncertainty you need 4× the data. Plan experiments with that in mind
Correlated data: variance formulas that assume independence understate uncertainty badly

What breaks

A/B test says significant, the effect vanishes later — Underpowered test plus repeated peeking. Fix the sample size in advance.
Averaging didn't reduce noise as expected — The samples are correlated, so the √n rule doesn't hold.

Watch it explained

Mean (expected value) of a discrete random variable | AP Statistics | Khan Academy — Khan Academy, 4:31