Correlation & Covariance

Do two variables move together? Covariance gives direction, correlation strength.

Maths · Intermediate · ~4 min

In plain English

Do these two things move together? Covariance says yes/no and by how much in raw units; correlation rescales that to a clean −1 to +1.

Why it's worth your time

It's the fastest read on feature redundancy and the single most abused statistic in analytics.

If you remember three things

Correlation is covariance normalized by both standard deviations
It only measures LINEAR association
Correlation is not causation, and a zero correlation is not independence

Overview

Covariance measures whether two variables deviate from their means together, and its sign gives the direction of the relationship. Because its magnitude depends on units, correlation rescales it to the interval −1 to +1 for a unit-free measure of strength. A strong correlation signals association, never proof of causation.

In an interview

Covariance multiplies the two variables' deviations from their means and averages them — positive when they rise together, negative when they move oppositely. But its size depends on units, so correlation divides by both standard deviations to land in −1…+1. Even a perfect correlation doesn't prove one variable causes the other.

Production defaults

Screening: correlation matrix on features; |r| > 0.9 means one of the pair is probably redundant
Non-linear: Spearman (rank) correlation catches monotonic relationships Pearson misses
Always plot: Anscombe's quartet: four wildly different datasets, one correlation coefficient

What breaks

r ≈ 0 but the scatter plot is a clear arc — Pearson only sees straight lines. Use Spearman or mutual information.
Highly correlated features made coefficients unstable — Multicollinearity. Ridge regularization, or drop one of the pair.

Watch it explained

Introduction to Correlation (Statistics) — Cody Baldwin, 4:17