Recently there was a discussion around what is AB testing, why is it important, and how it has been used. A couple years ago, I had the opportunity to speak at a conference about AB (split testing). Here is the presentation as reference. Below are some links we put together recently that highlight the process of A/B testing. There is also a website GoodUI.org that lists a series of AB testing design tips. The information here is merely a springboard to understanding the process.
How User Feedback and A/B Testing Work Together (Article)
For many online projects, I have used this method successfully. You collect feedback, analytics and then run AB testing together to made educated decisions. If you just collect feedback, it's not enough and the same with analytics and AB testing. This article breaks down how they can both work together and why it matters. It actually would be something we could theoretically setup quickly but it would take time to analyze and make decisions.
http://blog.usabilla.com/user-feedback-ab-testing-need/
Nexflix Case Study (Article)
Netflix’s UX group ran an A/B Test on different on how changing the box art on their movies affect conversion rates. This is a good example of a thorough A/B. Because it breaks down exactly what their testing by stating their hypothesis, their control, their test variable and their success criteria.
https://uxdesign.cc/how-netflix-does-a-b-testing-87df9f9bf57c#.9jgr0f3rb
Benefits & Limitations of A/B Testing (Article)
Nielsen Norman Group talks about the benefits and limitations of A/B testing. They talk about how you need to have a goal measurable via analytics to run a successful A/B test.
https://www.nngroup.com/articles/putting-ab-testing-in-its-place/
A/B Test Checklist (Article)
This checklist outlines the key steps you need to take to setup an A/B test. The first step is to pick on variable to test and the second is to choose a goal.
http://blog.hubspot.com/marketing/a-b-test-checklist
How to Turn Data into Insights & Customers (Video)
When I spoke at an AB test conference back in 2014, I met this speaker Peep Laja and saw him give this presentation. It's clear and outlines the process of setting up an AB test. Most of the information you find around AB testing is all about conversions - he starts here but moves into just finding the problems. This runs about 24 minutes.
https://www.youtube.com/watch?v=TxsXApadBCA
