Group users by when they joined, then follow each group forward — the only way to tell a product that is improving from one that is just growing.
Follow the class of 2024 through school rather than averaging everyone in the building. Only then can you tell whether teaching improved or the intake changed.
It's the only way to separate 'the product is getting better' from 'we acquired more users'.
A single overall metric hides everything interesting, because it mixes people who joined years ago with people who joined this week. A cohort analysis fixes the group at a starting event — usually signup month — and then measures that fixed group at 1, 2, 3 months of age. Reading down a column tells you whether newer cohorts behave better than older ones, which is the question 'is the product getting better'. Reading across a row tells you the lifecycle of a single group. The overall average tells you neither, and moves whenever the mix of new and old users moves.
A cohort analysis groups users by their start period and tracks that fixed group over subsequent periods. Rows are cohorts, columns are age since joining. Reading across a row gives one cohort's lifecycle; reading down a column compares cohorts at the same age, which is how you tell product improvement from growth. An aggregate metric conflates the two.
What is a Cohort? How to Read a Cohort Analysis Chart... — Peel, 5:31