The Paywall Trade-Off: Revenue Up, Retention Down, What Do You Do?
This is the scenario that ends more PM interview answers than any other. Not because it is hard to understand, but because it has two true facts pointing in opposite directions, and the candidate has to hold both at once instead of picking the convenient one.
The scenario below is fictional, with round numbers. The reasoning is the part that transfers.
The scenario
A running app tests putting personalized training plans behind Premium ($12 a month). Free users keep basic plans. After 4 weeks:
- Revenue per user in the test group is up 32%.
- 30-day retention is 34% in test vs 40% in control.
Two stakeholders read the same result and disagree:
- Finance lead: "Ship it, revenue per user is up a third."
- Growth lead: "Hold on, we're trading future subscribers for revenue today."
Both are reasoning from real numbers. The question is which number matters more, and over what time horizon.
The first question: what is the shape of the retention curve?
30-day retention is a snapshot, not a story. The decision depends on what happens after day 30, and there are three shapes the curve can take:
The gap holds. Test retention stays about 6 points below control at day 60 and day 90. The paywall permanently costs you users. Every cohort you put through the gate is a smaller cohort forever, and the revenue gain has to be large enough to compensate for a permanently smaller base.
The gap converges. Test retention catches up to control by day 60 or 90. The paywall delayed some users rather than losing them: they took longer to come back, or they came back through a different path. A converging gap is a much weaker reason to hold, because the cost was temporary.
The gap crosses. Test retention ends up above control after day 60. This happens when the gate selects for the users who were going to be serious anyway, and the serious users retain better once they are paying. A crossing gap turns the trade-off into a win: fewer users, but better ones, and the revenue gain compounds.
You cannot tell which shape you have from a 30-day snapshot. The correct response to the Finance lead is not "you're wrong," it is "ask for day 60." Specifically: ask for the retention curve by cohort week, not a single number, because the shape is the information. A candidate who asks for the curve sounds like someone who has seen this movie before. A candidate who argues from the 30-day number alone sounds like someone picking a side.
The second question: is the revenue real, or pulled forward?
Revenue per user up 32% in 30 days means one of two things, and they have opposite implications:
Real value. Users who hit the paywall and pay are getting more from the product, and they keep paying. The 32% persists at day 60 and day 90, and lifetime value in the test arm ends up higher.
Pull-forward. Users who would have subscribed eventually subscribed sooner because the gate forced the decision, while some users who would have subscribed later churned instead. Revenue arrives sooner, churn arrives sooner, and lifetime value ends up flat or lower. The 30-day number looks great because you compressed the timeline, not because you created value.
The test that separates them is the same as before: cohort lifetime value over a longer window. Ask for the LTV trajectory, not the 30-day revenue number. If LTV in test is still ahead at day 60 and day 90, the revenue is real. If LTV converges or falls behind, you pulled earnings forward while causing sooner churn, and the Finance lead's number was a mirage with a 30-day horizon.
This is the coexistence point, and it is worth stating in exactly this form in an interview: "Higher 30-day revenue and lower lifetime value can both be true at once, when the gate moves money sooner without creating it."
The third question: what guardrail would change your mind?
A primary metric tells you what you hope to move. A guardrail tells you what you refuse to break. For a paywall test, the guardrail candidates are:
- Uninstalls. A defensible signal of extreme dissatisfaction, but a weak guardrail on its own, because it misses the silent churners: users who stop opening the app without uninstalling it. Uninstalls measure the angriest users, not the lost ones.
- Retention itself. In a trade-off test, retention is arguably the guardrail, not a secondary metric: the test is only shippable if retention does not degrade beyond a pre-set bar.
- Support contacts and refund requests. Rising complaints about the gate are an early warning that the paywall experience feels unfair rather than valuable, which predicts churn that has not shown up in the numbers yet.
The interview answer that stands out names a guardrail and a pre-registered bar: "I would ship only if day-60 retention in test is within 2 points of control, and hold otherwise." A bar set before the results arrive is worth more than a bar invented after them, because a post-hoc bar always ends up wherever the results already are.
The decision structure
A strong interview answer to this scenario has five beats:
- Restate both facts without picking a side. "Revenue per user is up a third, and 30-day retention is down 6 points. Both are real."
- Ask for the curve. "Before deciding, I want day-60 retention by cohort week, to see whether the gap holds, converges, or crosses."
- Ask for the LTV trajectory. "And I want cohort LTV over the same window, to check whether the revenue is real value or pull-forward."
- Name the guardrail and the bar. "My guardrail is day-60 retention within 2 points of control. If it misses that bar, we hold regardless of the revenue number."
- Make the call and say what would change it. "On the current numbers alone, I would hold, because a 6-point retention gap with only 30 days of revenue evidence is not enough to ship. If day-60 LTV in test is ahead and the retention gap has converged, I would ship."
That last beat, what would change the call, is the difference between an answer and a verdict. Interviewers score the reasoning, and reasoning that names its own reversal condition is reasoning that can be trusted.
Why this scenario is so common in interviews
The paywall trade-off is a template, not a one-off. The same structure appears whenever a change makes money and costs users: ads, fees, reduced free tiers, stricter limits. The transferable skill is not "know the paywall answer." It is the habit of asking for the longer window, the curve shape, and the guardrail before deciding, every time two metrics point in opposite directions.
If you practice one thing from this post, practice the two questions: "Does the gap hold, converge, or cross?" and "Is this real value or pull-forward?" Those two questions turn a trade-off argument into a decision procedure, and that is what interviewers are listening for.