Measure

Holdouts & incrementality

Keep a random group from receiving a campaign at all. The difference between that group and everyone else is the lift the marketing actually caused.

How it's used

A retention email goes to 90% of subscribers; 10% get nothing. If renewals are about the same in both groups, the email is not doing much, no matter what its open rate says.

How I help

I size holdouts so they can detect the lift you expect, choose outcome metrics that matter, and report results with honest ranges.

ProofHoldout-tested email, SMS and push for a media client

Common toolsRetention email, SMS and push programs