Why case studies lie (without lying)
Most agency case studies are technically accurate and substantively misleading. The numbers in them are real. The thing the numbers are claiming is not.
There are three recurring patterns. We see them on roughly four out of every five case studies we audit on the open web.
Tell #1: no baseline
"+312% organic traffic year-over-year" is a meaningless number if you don't see what the previous year was. A site that went from 100 visits/month to 412 visits/month did, technically, grow 312%. So did a site that went from 100,000 to 412,000. The work involved is not remotely the same.
Watch for this phrase: "organic traffic grew X%". If there is no chart showing absolute numbers, assume the absolute numbers are small.
Tell #2: no time horizon
"4.6× ROAS" without a date range is uninterpretable. Was that one week during a holiday push? A peak month? A year-long average?
The rule of thumb: any performance claim that isn't attached to a clear window (Q3 2025, March 2026, the 90 days after launch) is structured to be flattering. Demand the window.
Tell #3: no attribution method
The case study says paid social drove $3.2M in revenue. Who decided? Last-click? Platform-attributed? A custom MMM? The author?
Every paid channel will overstate revenue by 2–4× if measured naively. Without a sentence on how the number was attributed, every "the channel drove $X" claim is a sales pitch, not a data point.
A better template — what a credible case study contains
- Client name (or a clear reason for anonymity).
- Engagement dates, including start and end.
- Baseline metrics in absolute terms.
- The specific plays run, by month.
- Attribution method spelled out — one sentence is enough.
- What didn't work, and what the team learned.
If a case study has all six, it is probably honest. If it has fewer than three, it is probably a brochure.
How we write ours
Our own case studies index uses the six-part template above. Named clients (where contractually possible), real dates, real attribution. If we can't show one of the six, we say why.