How Activation Claims Are Actually Made
Every mature product has a famous activation number. Facebook's is the best known: users who added 7 friends in their first 10 days stuck around, and users who didn't mostly left. That number went on to organize onboarding, growth targets, and product priorities for years.
But where does a number like that come from? Not from a brainstorm, and not from a single experiment. It comes from a process with five steps, and the step most people skip is the one that decides whether the number is real.
This guide walks through that process, using the Facebook example throughout, and ends with how to talk about it in a PM interview.
The short answer: the observation usually comes first
Facebook did not start with a theory and build friend-adding. They mined their retention data, grouped users by early behaviors, and noticed the pattern: users who hit 7 friends in 10 days retained, users who didn't mostly churned. That is a correlation found in the wild, not a hypothesis confirmed in a lab.
The hypothesis-first path exists too, and it is what you do with new products or genuinely new features: strong theory (network effects help retention), build the thing, then test. The two paths are not rivals. They are the same process entered at different points, and which one you use depends on whether you already have data worth mining.
The five steps
1. Observation
The pattern in the data. "7 friends in 10 days" predicts retention.
An observation is not a claim yet. It is a candidate. Plenty of patterns in product data look strong and mean nothing: users who change their profile photo retain better, users who open settings retain better, users who do almost anything in the first session retain better. The question is never "did we find a pattern?" It is "does this pattern point at something we can act on?"
2. Mechanism story
The hypothesis about why. For Facebook: network effects. More friends means more content in the feed, more notifications, more reasons to come back. Without a mechanism story, a pattern is just a curiosity. With one, you have something testable, because the mechanism tells you what should happen if you intervene.
Ask this of any activation candidate: if we pushed a user toward this behavior, why exactly would they retain? If you cannot answer in one sentence, you do not have a mechanism. You have a correlation wearing a story's clothes.
3. Confound checks
Are the 7-friend users just more social people who would have retained anyway? This is the step that separates a serious team from a team that ships a dashboard number and calls it strategy.
The method is straightforward: segment the users and check whether the pattern survives inside each group. Does 7 friends predict retention among users who signed up through invites, and also among users who signed up organically? Among mobile users and desktop users? Among users in different countries? If the pattern holds inside every segment, it is robust. If it vanishes inside segments, it was an artifact of composition (that is Simpson's paradox territory: the overall pattern was driven by who was in each group, not by the behavior itself).
The kinds of segments a team would try: signup source, geography, device, age, whether the friends were imported contacts or added on the platform. To be straight, the exact segments from Facebook's original analysis are not public. The story that survives is the finding, not the workbook. But the method transfers: segmenting is how you interrogate a correlation before trusting it.
4. The encouragement test
Nudge new users toward 7 friends and see if retention moves. This is the step that separates a real lever from a proxy.
Why a nudge and not a forced change? Because you cannot force friendship, and more importantly, you should not want to. The encouragement test asks the only question that matters for a decision: if we push people toward this behavior, does the outcome move? A randomized nudge (some new users get the friend-finding prompt, some do not) gives a clean comparison, because randomization balances the confounds whether you measured them or not.
5. Adoption as a target
If pushing toward the behavior moves retention, it becomes the activation metric: the number the whole team organizes around. Onboarding flows get redesigned to get users there faster. Growth reviews track the percentage of new users hitting it. It becomes, for practical purposes, the definition of a successful new user.
The key nuance: lever vs mirror
An activation metric is a decision tool, not a law of nature. It is almost always partly proxy and partly cause. The 7-friend users are both people who got network value and people engaged enough to add friends in the first place. That mixture is fine, as long as pushing people toward the metric moves the outcome.
The test is what tells you whether you are holding a lever or a mirror. If the nudge moves retention, it is a lever, use it. If the nudge does nothing, the pattern was just a mirror reflecting engaged users, and you drop it as a target. (It can stay as a diagnostic: a mirror still tells you something about who your engaged users are. It just cannot be the thing you optimize.)
The nudge test does not prove "friendship causes retention" in the abstract. It proves "pushing people toward friendship moves retention," which is the only version of the claim you can act on.
Why bother with confound checks if a randomized test follows?
This is the obvious objection, and it is worth answering honestly: step 3 is not needed for step 4's validity. A properly randomized test balances confounds on its own, measured or not. If you run the nudge test, the comparison is clean whether or not you did step 3. So step 3 earns its keep in four other places:
1. Triage. Confound checks are free, tests are expensive. You can do step 3 today on data you already have. If the pattern collapses the moment you account for engagement, you just saved yourself a test. Kill bad ideas cheap.
2. Design. This is the direct answer to how confounds flow into the test. You do separate the population, but by traits known before the test. If engagement is your top confound, you randomize within engagement levels (low-engagement users split 50/50, high-engagement users split 50/50) and you pre-specify that you will analyze results within each segment. That tells you whether the nudge works where it matters most (the unengaged) or only for people who were already engaged. Segments must be pre-treatment traits. Segmenting by what people did after the nudge rebuilds the trap: you would be comparing groups that the treatment itself created.
3. Precision and sanity checks. Measuring confounds before the test lets the analyst adjust for them, which sharpens the read, and lets you verify the arms are actually balanced. If the treatment arm is mysteriously older than control, something broke, and you want to know that before you ship on the result.
4. Interpretation. If the test comes back null or confusing, your confound list is your first set of suspects for why.
Step 3 makes step 4 smarter, better targeted, and interpretable. Step 4 is what makes the claim causal.
How to talk about this in an interview
Activation questions show up in PM interviews in two forms: "How would you define activation for this product?" and "Here's a correlation we found, what would you do with it?" Both are really testing whether you know the difference between a pattern and a lever.
A strong answer has four beats:
- Start from the data, not the theory. "I would look at what early behaviors predict retention, then build the mechanism story for the strongest candidate."
- Name the confound out loud. "The obvious worry is that these users were already more engaged, so I would check whether the pattern holds inside segments before trusting it."
- Propose the encouragement test. "Then I would run a nudge toward the behavior and see if retention moves. That is the test that decides whether we adopt it as a target."
- State the lever-vs-mirror condition. "If the nudge moves retention, we have a lever and we organize around it. If it does not, the pattern was a mirror, and we keep it as a diagnostic, not a target."
That fourth beat is the one most candidates miss, and it is the one interviewers remember. Anyone can recite the Facebook story. Fewer candidates can say what would make them stop believing it.
Practice: three drills
Drill 1. A fitness app finds that users who log a workout in their first 3 days retain at 3x the rate of users who don't. Write the mechanism story in one sentence, then name the two most likely confounds.
Drill 2. Same app. Design the encouragement test: what nudge, what population, what you measure, and what result would make you adopt "first workout in 3 days" as the activation metric.
Drill 3. A marketplace finds that sellers who upload 5 listings in week one earn 4x more in month two. Argue the lever case and the mirror case, each in three sentences. Then say what test would settle it.