Probability Basics

Outcomes, events, and the rules for combining chances with AND and OR.

Maths · Beginner · ~4 min

In plain English

How often something happens if you could rerun the world many times. Everything else is bookkeeping about how those chances combine.

Why it's worth your time

Every model output you'll ever ship is a probability, and misreading one is how confident systems become wrong systems.

If you remember three things

P(A and B) = P(A)·P(B) only when they're independent
Conditional probability is the workhorse: P(A|B)
Probabilities compose by multiplication, so use logs

Overview

Probability starts with the sample space of all outcomes, and events are the subsets you care about. Every probability sits between 0 and 1, and all outcomes together sum to 1. Two rules combine events: addition for 'or' and multiplication for 'and', with independence and mutual exclusivity as the special cases that simplify them.

In an interview

Probability assigns each event a number from 0 to 1. For 'A or B' you add the probabilities and subtract the overlap; for 'A and B' you multiply. Independence means one event doesn't affect the other, so the multiplication simplifies to P(A)·P(B). Mutually exclusive events can't co-occur, so their overlap is zero.

Production defaults

Log space: always, for chains of probabilities. Products of small numbers underflow to zero
Independence: assume it only when you can defend it. Most real features aren't
Calibration: a model outputting 0.8 should be right 80% of the time. Check with a reliability plot

What breaks

Multiplied probabilities all become 0 — Floating-point underflow. Sum log-probabilities instead of multiplying probabilities.
The model's 90% confidence is right 60% of the time — Miscalibration. Temperature scaling or isotonic regression on a held-out set.

Watch it explained

Probability explained | Independent and dependent events | Probability and Statistics | Khan Academy — Khan Academy, 8:18