Skip to the lesson
Back to the home page and map, from TryOwning · Marketing · Lesson 3
Part 1 of 3Part 1 of 3, 0 done

Marketing · 14 min

Measuring what works

Some ads bring customers. Some only take the credit for customers who were coming anyway. Marketing teams test, track and turn things off to tell them apart. Here's how.

Ticket 01 · The guess

What did one small change earn?

At Bing, Microsoft's search engine, an employee suggested showing the headlines of its ads a little differently. Nobody thought it mattered much, and the idea waited for months. Then someone tested it on some of Bing's users. How much did Bing's revenue go up?

2%
0%30%
Words to know

Spot something wrong?

Tell us what looks off. No name or email needed.

What kind of problem?

0 / 1000

We'll also send this page, the lesson, ticket 1, so we can find it. Nothing else. Privacy

·
Sources and every number (1)

Sources

  • The Surprising Power of Online Experiments, by Ron Kohavi and Stefan Thomke (Harvard Business Review, September–October 2017): the researchers who ran Bing's experiments report the ad-headline test · Harvard Business Review, 2017

Every number

  • 12%hbr.org ·2017 · How much Bing's revenue went up from a small change to how ad headlines were shown, as the researchers who ran Bing's experiments reported in Harvard Business Review (2017).
  • $100Mhbr.org ·2017 · More than this a year, in the United States alone, from the same change, as reported in the same article.
  1. Lessons›
  2. Inside a business›
  3. Marketing
  • ·Before this: Who it's for
Skip to the lesson
Back to the home page and map, from TryOwning · Marketing · Lesson 3
Part 1 of 3Part 1 of 3, 0 done

Marketing · 14 min

Measuring what works

Some ads bring customers. Some only take the credit for customers who were coming anyway. Marketing teams test, track and turn things off to tell them apart. Here's how.

Ticket 01 · The guess

What did one small change earn?

At Bing, Microsoft's search engine, an employee suggested showing the headlines of its ads a little differently. Nobody thought it mattered much, and the idea waited for months. Then someone tested it on some of Bing's users. How much did Bing's revenue go up?

2%
0%30%
Words to know

Spot something wrong?

Tell us what looks off. No name or email needed.

What kind of problem?

0 / 1000

We'll also send this page, the lesson, ticket 1, so we can find it. Nothing else. Privacy

·
Sources and every number (1)

Sources

  • The Surprising Power of Online Experiments, by Ron Kohavi and Stefan Thomke (Harvard Business Review, September–October 2017): the researchers who ran Bing's experiments report the ad-headline test · Harvard Business Review, 2017

Every number

  • 12%hbr.org ·2017 · How much Bing's revenue went up from a small change to how ad headlines were shown, as the researchers who ran Bing's experiments reported in Harvard Business Review (2017).
  • $100Mhbr.org ·2017 · More than this a year, in the United States alone, from the same change, as reported in the same article.
  1. Lessons›
  2. Inside a business›
  3. Marketing
  • ·Before this: Who it's for