Landing page · Market research

What is the best way to execute A/B testing on a landing page?

Haya Levine Independent Medical Practice Professional

June 9th, 2015

Friends, I am in the process of finalizing a landing page for a new product. My intention is to test various price points, by creating various versions of the page.

Ideally, I would like an equal number of visitors to to see each version of the page. Is there some hi-tech way to do this? Or would I have to run two separate experiments?

Anthony Zeoli Digital Strategy and WordPress Consultant and Trainer

June 9th, 2015

A/B testing is usually done dynamically, so that page A is served to the first user and the page B is served to the second user and it flips back and forth. Both pages should have some marked differentiation of content, style, and offer in order to ascertain which does better.


If using WordPress, there is a plugin called Landing Pages with A/B testing built in. I'm using it myself.

Mikhail Gorelkin Principal AI Architect & Developer

June 9th, 2015

I would recommend more sophisticated testing based on the bandit algorithms. Look for example at https://www.chrisstucchio.com/blog/2012/bandit_algorithms_vs_ab.html 

Jessica Alter Entrepreneur & Advisor

June 9th, 2015

Very easy to do with Optimizely or competitors >> 

Anonymous

June 9th, 2015

Just create one version in your own HTML/CMS. Then use Optimizely to create variations.
Please note that you need to have a decent amount of traffic to be able to do A/B testings. At an average of lets say 1 % conversion with 20 % difference you want to detect at 95% significance you need over 35,000 views. https://www.optimizely.com/resources/sample-size-calculator/?conversion=1&effect=20&significance=95

Stephen Huson Leader in Internet lead generation, SEM / PPC / SEO and analytics

June 10th, 2015

Agree with Jessica...Optimizely or Visual Website Optimizer are good choices.

Haya Levine Independent Medical Practice Professional

June 10th, 2015

Thanks all. Do you know if I can use Optimizely with a Wix website? 

Gil Allouche Founder @ Metadata

June 10th, 2015

Hey Haya! Yes you should use a tool like: Optimizely (http://optimize.ly) or Visual Web Optimizer (vwo.com) -- they have a free version and generally speaking are not expensive. You install an embed code on the landing page, then you create a few experiments (same page diffent price, changing location of price etc) decide what % of visitors get that experiment (you can have few versions of the landing page at a time) -- and the rest happens magically. In fact -- it can automatically learn the best price and optimize accordingly. Happy to help further. --Gil | founder at metadata.io

Mikhail Gorelkin Principal AI Architect & Developer

June 11th, 2015

Here is a very fresh post: How to Evaluate Machine Learning Models: The Pitfalls of A/B Testing:
http://blog.dato.com/how-to-evaluate-machine-learning-models-the-pitfalls-of-ab-testing
 

Mike Masello

June 11th, 2015

I'm a fan of VWO, but be aware of sample size requirements and measurements of statistical significance when running an AB test.  I've seen plenty of tests with low data be declared statistically significant as a "winner" only to end up in a tie or even flip-flop.  The smaller the difference in the 2 groups the larger the sample you want to collect.

If you're running this on a small site setup your test to maximize the exposure.

Kent Lewis Anvil President & Founder | Purveyor of measurable marketing that moves you (analytics, search & social media marketing)

June 12th, 2015

I recommend utilizing a platform like Unbounce, Optimizely or LiveBall to automate the testing. I've written about the topic here:

Turn Website Visitors into Customers via Conversion Optimization
http://www.business2community.com/online-marketing/turn-website-visitors-into-customers-via-conversion-optimization-0329212