4 Product Testing Results from Booking.com’s Experimentation Machine (The Irrational Mind Podcast)

January 31, 2025  |  By: The Irrational Mind

Most companies run A/B tests. Booking.com has turned testing into a science, running thousands of experiments at once. Kristina Gibson, former Director of Product at Booking.com, reveals their most powerful testing discoveries in her conversation with Kristen Berman for The Irrational Mind podcast—and what product teams everywhere can learn from their results. Note that Kristina was at Booking.com around 2019, so some of these tests may not be currently live on the site.

Enjoy these insights? Your team may be interested in working with Irrational Labs to deep dive on customer psychology and behavioral design.

Here are four key insights from their conversation that product teams can apply.

1. Sold out: Create strategic scarcity (even when it seems counterintuitive)

It’s no secret that when an online shop says they have “one left”, some consumers will jump and purchase that item to avoid missing out. Booking.com goes a step further. They show actual hotels that are currently “Sold out”. You’d imagine that this would decrease site conversion. They’re taking up valuable online space to show you something you cannot buy!

However, they tested “sold out” labels and something surprising happened. Showing sold-out properties alongside available ones increased overall bookings. The unavailable options heightened travelers’ desire to book what they could still get. Why? Likely it showed travelers that the label “1 left” may not be cheap talk—hotels were actually selling out. Booking.com amplified this mindset by showing real-time signals like “13 people are looking at this hotel” and “85% booked for your dates.”

Product team takeaway:

These days we’re used to seeing “limited time offer” attached to online products. We may even glance over this and discount that it’s actually real. Adding something that signals you’re NOT lying/misleading consumers could get them to take you more seriously. Obviously, abusing this is risky. Consumers catch on and your brand value will erode. But thoughtfully designed, real scarcity can motivate action—especially for big decisions like travel where a lot is on the line if you make a mistake and it’s tempting to procrastinate.

A few companies who have done this include: 

  • Hinge limits daily matches to increase engagement
  • Farm-to-table restaurants change menus regularly based on what’s locally available
  • National parks limit daily visitors to preserve the environment and experience
  • McDonald’s creates time-limited Monopoly promotions

2. Toilet paper included: Reduce uncertainty with small details

Small property details had an outsized conversion impact. Telling people things that may seem obvious or feel trivial drove bookings:

  • For hostels, mentioning the availability of charging outlets by beds
  • For hostels, reassuring people “Toilet paper included”
  • For hotels, specific descriptions (“very good breakfast”) outperformed generic ones (“breakfast included”).

These precise details reduced uncertainty at the moment of decision.

Product team takeaway:

The potential customer/buyer is really trying to figure out if what you’re selling is any good and if it will meet or exceed their needs. Your job during that evaluation period is to reduce uncertainty. Test surfacing specific details that help address their concerns or potential worries. Sometimes tiny pieces of information can tip the scales.

A few companies who have done this include: 

  • Airbnb displays specific host response times
  • Amazon shows exact delivery dates, not ranges
  • Zappos lists detailed shoe measurements

3. Pick your pics: Pick visuals that convey the right value

For hostels, Booking.com discovered showing photos of social spaces outperformed standard room shots. The bunk bed wasn’t selling the stay – the community vibe was. They reorganized their photo carousels to show common areas first.

Product team takeaway:

The right visuals can speak volumes and convey the value quickly. Show customers what they’re really buying, not just the obvious product features.

Some companies who have done this include: 

  • Peloton emphasizes community over bike specs
  • WeWork sells collaboration, not just desk space
  • Airbnb features people enjoying spaces, not empty rooms

4. Free cancellation: Remove risk from big decisions

Free cancellation became one of Booking.com’s most powerful conversion tools. While it created operational costs, making the decision feel less permanent drove significantly more bookings. When facing a big purchase decision, people need a safety net.

Product team takeaway:

For high-stakes decisions, focus on removing risk rather than just optimizing the funnel.

Examples of companies who have pulled this off: 

  • Warby Parker’s home try-on program
  • Zappos’ free return shipping
  • Apple’s 14-day return policy on devices

The bottom line

Booking.com doesn’t just test button colors. They systematically explore how people make travel decisions and build tools to make those decisions easier. Their success comes from balancing quick, iterative tests with strategic thinking about what drives customer behavior.

Check out the full episode of The Irrational Mind for more insights on building an experimentation culture and optimizing digital products through behavioral science. Be sure to subscribe to The Irrational Mind so that you don’t miss future episodes. Have a question for Kristen? Hit her up: [email protected].


🔑📈  Enjoyed Kristen’s deep dive into behavior change and Booking.com? Irrational Labs works with top companies to unlock product growth and customer value through behavioral design. Get in touch to learn more about our consulting services. Or get a taste of behavioral science by joining one of our bootcamps.

EXPLORE MORE

Our Services

From concept to code, explore how we get our hands dirty with research, product, and marketing challenges.

Our Areas of Expertise

Learn how we are helping change behaviors across the domains of health, education, finance, and more.

Join our Bootcamp

Understand your customers’ choices and learn how to change their behavior for the better — in our 8-week online Behavioral Design course.