DirectToGrowth Co.
A founder told me recently they'd tied their CRO agency's bonus to site conversion rate. "Simple," they said. "If conversion goes up, they get paid."
I asked what happened in the month they ran a 30%-off sale. Conversion rate jumped. The agency got its bonus. Nobody had shipped a single test that month.
Conversion rate is one of the most important numbers in ecommerce and one of the worst numbers to pay people on. It moves for a dozen reasons the landing page team doesn't control, and it can go up while the business gets worse.
If you pay for a number someone can't control, you'll get either luck or gaming. Usually both.
Everything that moves conversion rate that isn't the page
| What changed | What happens to conversion rate | Did the page team do it? |
|---|---|---|
| A sitewide sale | Up, often sharply | No |
| More prospecting spend on cold audiences | Down — new visitors convert lower | No |
| Retargeting budget increased | Up — warm visitors convert higher | No |
| Hero product goes out of stock | Down | No |
| Influencer post sends a traffic spike | Down on the day, then up | No |
| A new test on the product page wins | Up | Yes |
One row out of six. Pay on blended conversion rate and you're mostly paying for the media plan, the merchandising calendar and the weather.
It's also easy to game
Want a higher conversion rate fast? Cut prospecting traffic. Push more discount codes. Remove the higher-priced products from the landing page. Each of those lifts the ratio and can hurt revenue, average order value or margin. A team bonused on the ratio is quietly rewarded for all of them.
What I pay for instead: the story
When I set up incentives for a landing page or CRO team — in-house or agency — I pay for three things that together make up the story of the work.
1. Impact: did a tested change make money?
Measured inside a proper A/B test, against a control, on revenue per visitor rather than conversion rate alone. Revenue per visitor catches the tricks: if a change lifts conversion by pushing cheaper products, revenue per visitor tells on it. Only tests that reach the sample size you agreed in advance count.
2. Learning: do we know something we didn't before?
Most tests lose. That's normal, and a team that's punished for losing tests stops running brave ones. So I reward clear learning: a written hypothesis before the test, a result after it, and a one-paragraph answer to "what does this tell us about our customer?" A well-documented loss that kills a bad idea for the whole company is worth paying for.
3. Follow-up: did the learning get used?
The best CRO programs compound. A winning headline angle gets carried into the ads, the emails and the next landing page. I reward the team when a learning gets rolled out beyond the page it was tested on — that's where the real money is.
How to structure the bonus
Half on impact: total revenue-per-visitor lift from shipped winners, measured in-test and verified for at least two weeks after rollout.
A quarter on learning: a quarterly review of the test log — hypotheses written up front, results recorded, insights stated plainly.
A quarter on follow-up: learnings that were rolled out to ads, email or other pages.
Keep blended conversion rate on the dashboard. Watch it every week. Just don't put a bonus on it.
The monthly test report I ask for
Tests launched, tests concluded and tests still running.
For each concluded test: hypothesis, result on revenue per visitor and confidence.
What we learned about the customer, in one sentence.
Where that learning is being used next.
Here's the reframe. You don't want a team that makes a number go up. You want a team that makes your understanding of the customer go up — because that's what keeps paying after the test is over. Pay for the story, and the numbers follow.