facebook hr推荐A/B testing 学习资料

过了hr screening正在准备第一轮电面，看到这份fb hr推荐的ab testing的资料，非常浅显易懂，其中一些内容高屋建瓴，看完后觉得被理顺了脉络https://www.shopify.com/blog/the-complete-guide-to-ab-testing
其中对于我们申请data science岗位有帮助的experimentation思维脉络，我整理了一下：

• Collect existing or historical data and make sure analytics implementations are accurate.
• Analyze previous data and find insights. In my opinion, this is the core of any good testing program. In the analysis stage, the goal is to examine your analytics data, survey or UX data, or any other sources of customer insight you might have in order to understand where your opportunities for optimization are.
• Turn insights into hypotheses. Think about how you could potentially fix or improve these areas of optimization.
• Prioritize based on impact and ease, and maximize the allocation of resources (especially technical resources).
• Determine how many variations you can test (based on your traffic level), and then pick your best one to two ideas for a solution to test against control.

• Have your front-end developer implement the treatments in your testing tool. Set up necessary integrations (Google Analytics), set appropriate goals.
• Conduct QA on the test (broken tests are by far the biggest A/B testing killer) to make sure it works with every browser/device combo.
• Run a test (following statistics best practices to the best of my knowledge and ability). Be sure to run them for a reasonable amount of time (I default to two weeks to ensure I’m accounting for the day of week effect or anomalies)
• Analyze results to determine your winner and implement or not according to the results.
• Iterate and repeat.

感谢分享

谢谢总结！