Lean startup · Landing page

"Lean Startup"-style Landing Page testing: Am I supposed to be simultaneously A-B testing my landing page AND my AdWords ad?

Matty Sallin System Financial, Inc.

November 5th, 2013

So, I just created a landing page (http://www.trythesystem.com if you'd like to give me your opinion) and I created my first AdWords campaign for it. I know I'm supposed to be A-B testing both the landing page by creating two versions, but am I also supposed to be A-B testing the AdWords campaign that links to it? Is this how it should work:

[A] Campaign --> [A] Landing Page
[A] Campaign --> [B] Landing Page
[B] Campaign --> [A] Landing Page
[B] Campaign --> [B] Landing Page

...right?

Carl Timm

November 5th, 2013

Though a main pillar in Lean is testing and learning, the end goal of the framework is to reduce waste. It's good to keep that in mind when thinking about endless possibilities.

In your case specifically, have you established a baseline yet? If not, it would be wasteful to start testing, because without a baseline you don''t even know for sure where (or if) you have issues in your conversion pipeline and therefore you don't know where to begin testing. And again, that is wasteful.

Dave McClure--through his AARRR framework--advocates you should first spend time on activation and retention before moving on to other metrics, like acquisition. I mention this specifically because I'd argue your landing page is part of your activation metric whereas your AdWords campaign is part of your acquistion metric. So, according to McClure, you'll want to spend time improving/testing your landing page before moving on to improving/testing your AdWords campaign.

That said, I'd argue you'll want a baseline of how well your landing page is converting before investing time building an A/B test. If through establishing your baseline you find your landing page is converting at or above industry standards, you won't even need an A/B test for your landing page. If you happen to find this result, you can avoid the waste of building an unnecessary A/B test....

Just a few thoughts--hope they help! Best of luck!

Mohammad Forouzani CEO at Forecast.net

November 6th, 2013

sorry to be so unspecific, but even adwords traffic varies a lot too.

For example, if you put things like "free xxxx" into your adword text, you might get a click from freebie seekers, but they wont convert well for paid items or other actions such as signups.
Also, the broader the keywords you target, the lower the conversion rate e.g. "travel guides".

Stick to highly targeted (longtail) key phrases such as "short travel guide for Sydney, Australia", and you can expect at least 10% conversion assuming you have everything else right on your landing page (good marketing copy, real value product for the customer etc.)
The problem with longtail is that there wont be much traffic coming to your site.

Of course there is also industry to factor in too - all things being equal, you wont get the same conversion rates for socks as for laptops.

Mohammad Forouzani CEO at Forecast.net

November 5th, 2013

you can if you want to - this is basically multivariate testing. Do some research on it to find out how to apply it.

The only thing to keep in mind is that you will need more data than if you were doing a single test - it's as if you are running multiple tests at once, so to gather statistically significant results you will have to send enough clicks for both tests. Translation: double your budget. 

Morten Jensen Founder at UserChamp; Chief Experimentation Officer at Playpen Labs

November 5th, 2013

What questions are you trying to answer? And which one is most important? I don't know that you're supposed to A/B test, but it is a great method if you have articulated what decision you will be making based on your test. Multivariate is great but gets messy.

Regarding the ads, different keyword lists (different ad groups) help you understand what searches are being performed. Testing different ad texts against each keyword list tells you what concepts and calls to action people doing those searches respond to. Finally, testing conversion rate on your site for each ad text tells you how each group of users respond to the content and call to action on your site.

Matty Sallin System Financial, Inc.

November 5th, 2013

@ Mohammad: Great point about doubling your budget in order to get a statistically significant pool number of clicks, thanks for that.

@ Morten: The questions I'm trying to answer are: "What landing page copy is most effective?" and "What AdWords campaign is most effective?". As I said, I'm just doing the Lean Startup landing page strategy so I'm just trying to gauge interest, so I'm not sure if I'm overthinking this but would appreciate any feedback.

Alex Murray

November 5th, 2013

I wouldn't worry about A/B testing anything until you start bidding on a lot of SEM traffic. For now, just read up on best practices and try to keep it simple. This may help you out: http://unbounce.com/landing-pages/the-complete-guide-infographic/ Best, Alex

Matty Sallin System Financial, Inc.

November 5th, 2013

@ Alex: Thank you!

Morten Jensen Founder at UserChamp; Chief Experimentation Officer at Playpen Labs

November 5th, 2013

With Alex and Carl here on keeping it simple and focusing on one metric/question at a time. Great point about reducing waste. I do occasionally get too many things going trying to "do all the lean things" at once.

Alan McKean President at One Touch, LLC

November 5th, 2013

Nice page, but there are a few typos you should fix: mathematically, not mathmatically insured, not instured FDIC, not FCID

Matty Sallin System Financial, Inc.

November 5th, 2013

@ Carl: That was exactly the kind of answer I was looking for, thank you! However, that has led to a couple more questions:

  1. Um, how do I know when a baseline has been established?
  2. Any opinions on industry standards for landing page conversion? I searched on Quora and found:
5% is bad. 
10% is OK. 
20% is ideal. 
30% is ahead of the curve.  

Do you agree?