Using experiments to make better product decisions

Teach Your Monster creates educational games that help children learn to read, improve maths skills and explore science through play.

The team approached me with a deceptively simple question about a soon-to-be-released game.

How much should we charge?

The team had already done the hard work. They’d spoken to customers, explored different pricing models and built detailed financial forecasts. The remaining uncertainty wasn’t in the spreadsheets. It was in how parents would actually behave when presented with different options.

Rather than spending more time refining the model, we asked a different question.

What’s the quickest, smallest thing we could ship to test it?

Over the next two weeks we built five landing pages, each presenting a different pricing model. By sending real traffic to each variation, the team could observe what people actually did rather than relying on what they thought they might do.

The experiment gave the team confidence in the pricing model they eventually chose. Just as importantly, it showed how quickly relatively small experiments could answer important product questions.

Since then, I’ve found myself asking the same question time and again:

What’s the quickest, smallest thing we could ship to test this?

Customer interviews, surveys and financial modelling all have an important place in product development. But there comes a point where the best way to reduce uncertainty is to put something in front of real people and see what happens.

Curious why experimentation plays such an important role in my approach?

It’s one of several principles that shape how I think about building products and helping teams make better decisions.