Just a great way to think about A/B testing in a different way (this was from DeepLearning.AI’s newsletter, the Batch):
- Build two versions of your product.
- Have the product team make predictions about which version will perform better.
- Test both versions and collect data on user behavior.
- Show the results to the team, and let them influence their beliefs about users and their reactions. If someone says, “Oh, that’s weird. I didn’t realize our users wanted that!” then we’ve learned something valuable.
- Based on the team’s revised intuitions, have them decide what to launch. It could be version A, version B, or something else.
- Repeat until you reach diminishing returns in terms of learning.