Lean startup · Landing page

"Lean Startup"-style Landing Page testing: Am I supposed to be simultaneously A-B testing my landing page AND my AdWords ad?

Matty Sallin System Financial, Inc.

November 5th, 2013

So, I just created a landing page (http://www.trythesystem.com if you'd like to give me your opinion) and I created my first AdWords campaign for it. I know I'm supposed to be A-B testing both the landing page by creating two versions, but am I also supposed to be A-B testing the AdWords campaign that links to it? Is this how it should work:

[A] Campaign --> [A] Landing Page
[A] Campaign --> [B] Landing Page
[B] Campaign --> [A] Landing Page
[B] Campaign --> [B] Landing Page

...right?

Carl Timm

November 5th, 2013

Though a main pillar in Lean is testing and learning, the end goal of the framework is to reduce waste. It's good to keep that in mind when thinking about endless possibilities.

In your case specifically, have you established a baseline yet? If not, it would be wasteful to start testing, because without a baseline you don''t even know for sure where (or if) you have issues in your conversion pipeline and therefore you don't know where to begin testing. And again, that is wasteful.

Dave McClure--through his AARRR framework--advocates you should first spend time on activation and retention before moving on to other metrics, like acquisition. I mention this specifically because I'd argue your landing page is part of your activation metric whereas your AdWords campaign is part of your acquistion metric. So, according to McClure, you'll want to spend time improving/testing your landing page before moving on to improving/testing your AdWords campaign.

That said, I'd argue you'll want a baseline of how well your landing page is converting before investing time building an A/B test. If through establishing your baseline you find your landing page is converting at or above industry standards, you won't even need an A/B test for your landing page. If you happen to find this result, you can avoid the waste of building an unnecessary A/B test....

Just a few thoughts--hope they help! Best of luck!

Mohammad Forouzani CEO at Forecast.net

November 6th, 2013

sorry to be so unspecific, but even adwords traffic varies a lot too.

For example, if you put things like "free xxxx" into your adword text, you might get a click from freebie seekers, but they wont convert well for paid items or other actions such as signups.
Also, the broader the keywords you target, the lower the conversion rate e.g. "travel guides".

Stick to highly targeted (longtail) key phrases such as "short travel guide for Sydney, Australia", and you can expect at least 10% conversion assuming you have everything else right on your landing page (good marketing copy, real value product for the customer etc.)
The problem with longtail is that there wont be much traffic coming to your site.

Of course there is also industry to factor in too - all things being equal, you wont get the same conversion rates for socks as for laptops.

Matty Sallin System Financial, Inc.

November 5th, 2013

@ Alex: Thank you!

Matty Sallin System Financial, Inc.

November 5th, 2013

@ Mohammad: Can you clarify what you mean by highly targeted vs. untargeted? I'm not buying traffic from anyone (didn't know you could even do that), I'm just spending $5/day on AdWords.

Alex Murray

November 5th, 2013

I wouldn't worry about A/B testing anything until you start bidding on a lot of SEM traffic. For now, just read up on best practices and try to keep it simple. This may help you out: http://unbounce.com/landing-pages/the-complete-guide-infographic/ Best, Alex

Carl Timm

November 6th, 2013

Mohammad has some great advice. Between his post and your Quora findings, you have some good guidelines. As far as establishing your baseline, you might need ~1 week of somewhat steady traffic. And by steady, I mean 5 - 10 people a day. This isn't huge and by no means will give you statistically significant data. But you'll begin to get a sense for where your baseline is and how well the page is converting. And at this point, getting enough feedback to form that sense quickly is a huge win. Again, good luck! 

Mohammad Forouzani CEO at Forecast.net

November 5th, 2013

you can if you want to - this is basically multivariate testing. Do some research on it to find out how to apply it.

The only thing to keep in mind is that you will need more data than if you were doing a single test - it's as if you are running multiple tests at once, so to gather statistically significant results you will have to send enough clicks for both tests. Translation: double your budget. 

Matty Sallin System Financial, Inc.

November 6th, 2013

@ Mohammad: Thank you, that helps a lot! 

Morten Jensen Founder at UserChamp; Chief Experimentation Officer at Playpen Labs

November 5th, 2013

What questions are you trying to answer? And which one is most important? I don't know that you're supposed to A/B test, but it is a great method if you have articulated what decision you will be making based on your test. Multivariate is great but gets messy.

Regarding the ads, different keyword lists (different ad groups) help you understand what searches are being performed. Testing different ad texts against each keyword list tells you what concepts and calls to action people doing those searches respond to. Finally, testing conversion rate on your site for each ad text tells you how each group of users respond to the content and call to action on your site.

Mohammad Forouzani CEO at Forecast.net

November 5th, 2013

No. Conversion stats vary greatly by many different factors, but the biggest one is traffic quality.

If you get highly targetted traffic, 30-40% can be achieved.
Untargetted - 1% is average.
If you buy traffic from places like thepiratebay, you would be lucky to get 0.1%

And there is everything in between.