We’re running a homepage A/B test right now to decipher best stickiness, CTAs, conversions, copy, etc. We’ve heard and read different things about sufficient sample size for this test. Most say 1,000 visitors is the bare minimum to begin drawing conclusions and making changes based on them. We’ve also seen as high as 3,000 visitors, however. What would you say is the ideal number of homepage visits to begin attaching statistical significance to a homepage A/B test?
I think that would depend on your projection on how many people do you expect to get there.