S2E37: Pre-test planning: The Red-Headed Stepchild of Experimentation with Haley Carpenter
Teams are horrible at this! They have no idea it should happen, and if they do, they're usually lost and the approach is terrible. When you break it down, it's really not difficult. This shouldn't be a black box anymore. I will cover the following: (1) What each of the test planning aspects are and how to do them including MDE and duration estimates, (2) CXL's pre-test calculator unless our new Optimizely one is live by the time I record, (3) how to turn pre-test calculations into a testing roadmap, and (4) how to determine when a business can't test or when a swimlane isn't viable so teams can stop wasting time. Chapters --- 0:00 - Introduction 1:50 - Imagine you’re with your team building an experimentation program 2:30 - Common struggles 6:26 - What is pre-test planning? 6:56 - Primarily…MDE and Duration estimates 26:28 - Roadmapping 28:38 - Should we test everywhere? Should we test at all? 31:00 - Example scenario 32:20 - Wrap up