All concepts

From Question to Metric

"Is the new checkout working?" is not a question a query can answer — turning it into one is most of the job.

Analytics Foundations · Beginner · ~5 min

In plain English

Someone asks if the shop is doing well. Before you count anything you have to agree what 'well' is, who counts as a customer, and over what week.

Why it's worth your time

Most useless analyses aren't wrong — they're precise answers to a question nobody actually asked.

If you remember three things

  • Name the decision, the population, the window and the exact event
  • Write down what result would change the decision, before querying
  • A number without a comparison isn't evidence

Overview

Analysts are handed vague questions and are expected to return specific numbers. The gap between the two is where the work happens, and rushing it is the most common cause of an analysis that is technically flawless and completely useless. The move is to decompose: what decision follows from the answer, which population are we talking about, over what window, and what exactly counts as the event. "Is checkout working?" becomes "among users who reached the payment step on web in the last 14 days, what share completed a payment within 30 minutes, split by old and new flow?" — which is answerable, checkable, and hard to misread.

In an interview

Turn a vague question into a metric by naming four things: the decision it informs, the population, the time window, and the precise event definition. Then state what result would change the decision, before you run anything. Most bad analyses are not wrong queries — they are precise answers to a question nobody actually asked.

Production defaults

Definition
one sentence covering population, window and event
Exclusions
internal, test and bot traffic — always explicit, always stated
Sanity check
estimate the magnitude by hand before anyone else sees it

What breaks

  • Stakeholder disputes the definition after seeing the result — Agree the definition and the decision threshold in writing first. It's negotiation-proofing, not bureaucracy.
  • The metric moved but nothing about the feature changed — The denominator moved. Check the population before investigating the numerator.

Watch it explained

Simple Strategies for Turning Data into Insights — Ruben Ugarte, 5:05

Related