I'm close to releasing the MVP version of my website, and I've read that stress/load testing is a good idea to make sure it can handle a large amount of traffic (and, of course, the hope is that we'll have a large amount of traffic). My current web developer hasn't done that before, so he suggested hiring someone else. I know there are some free testers out there, but most of our website is behind a log-in screen, so I can't use those.
This seems like a one-time job, so I'd like to pay a fixed fee, but I have no idea whether this is something that takes 1 hour, 10 hours, or 40 hours. Can anyone with experience hiring someone to do this/doing it themselves tell me about how long stress/load testing a website takes?
You don’t need it. Spending money on solving this problem is definitely putting your attention in the wrong place. If this is your MVP, you are 99% more likely to have no traffic rather than too much. Any money you would spend on stress testing the website would be better put into testing customer acquisition strategies.
Whether stress testing is necessary or not really depends on what your website is for. If it's a gaming platform, or a service-centric platform for example, then you definitely want to do SOME stress testing to make sure it can either handle the maximum expected load, or can scale (e.g. via an aws/rackspace service).
You also might want to consider doing a soft launch, which is to launch with a group of "beta" clients who get a discounted rate for the service for a period of time, like a few months. This way, you get real clients and real data, while they understand there may be bugs or hiccups and it doesn't wind up costing you your business' reputation.
Hello Tatyana, I load test cloud based software. Generally speaking, load testing can do 2 things for you:
1. Measure performance and help you do capacity planning for live
2. Find bugs that only manifest under load.
Load testing should be done early in the life cycle, because it can reveal architectural flaws. But even late in the game it's still very important to do.
A review of your functionality and non-tech requirements is in order (2-16 hours). Then a no-BS test plan to be created (2-8 hours). Then scripts created (1-2 days). Then tests executed (1-5 days or longer depending on the test plan). Then results analyzed and reported to you (2-8 hours). (Of course if I find anything particularly scary along the way, I will report it immediately upon confirmation.)
Then the real fun begins. You have to decide what to do with the results -- what you will fix, and what you will ignore. What happens next depends on you, but a retest needs to occur afterwards to validate fixes. This part can drag on, so we need to manage it very closely.
I would need to collaborate closely with your hosting staff, since I would likely need a special load environment to do testing in.
You should keep in mind that site owners often BEGIN looking for performance measurement and capacity planning, and END UP fixing bugs found along the way, causing the entire process to become longer and longer. This is the unfortunate price one must pay for load testing late in the life cycle, but if done with focus could be quite successful for you.
Does this begin to answer your question?
This is definitely not a one-time job. You need someone in your team that is capable of constantly monitoring and comprehending the metrics of your application.
That assumes that you have a real application on your hands. If your website is just a blog, an about page, and a handful of products/services. Indeed it does not really matter. Stress testing is for applications that expect constant user load.
What you really probably want to do is hire someone that is good at "Unit Testing" and have them teach your developer how to run the unit tests when they are done making them. That should take 1-2 weeks for most projects, larger projects can go up from there quite a bit.
- You can temporarily have the session data login an account globally to let "free testers" work for a day or two
- You should really run Kcachegrind with Graphvis to produce a callgraph of your application. This will show you bottlenecks.
- Load balancing and redundancy are the real answers. You want to always be "ahead" of the load that needs balancing. Servers are cheap. Get 3 extra. Load balance at least 1 more with your database, 1 with your application server and the last use with the application or as a replica of your staging server + database just as a spare
There is nothing worse than getting your first customers and then losing them because your platform went down. Small businesses become successful and die from inability to scale fast. All the time.
What Sam wrote!!!!
Thanks, everyone! Johnathan, the website is academicsequitur.com. It's far from a gaming platform, but it's not a blog either. There's an underlying database that users can essentially browse in various forms. I genuinely have no idea what kind of load that creates.
Sam McAfee is right. I took a look at your website and it doesn't look like it'll have any problems. You probably shouldn't worry about scaling unless you find the user experience degrading. Other things are more important in the beginning.
Tatyana - I looked at your website. Looks like you probably focus on aggregation and metrics? Probably stuff you run overnight? I wouldn't worry about stress or load testing.. Your focus would probably be better spent on customer acquistion as Sam McAfee suggests, but make sure you have a system in place that allows your database and search servers to scale. Memory and disk space will be your main issues, assuming you have your search algorithms optimized.
I took a look at the public portion of the web site and sized up the features from the video intro.
You could have bugs hiding in the database concurrency somewhere. You really should do an evaluation of behavior under load and basic capacity planning. You need to know how many virtual users can fit on 1 server, and then together with your operations people we can can do the math to calculate number of servers required to support your expected load levels.
How solid is your infrastructure? Where are you hosting? Do you have load balancing in place?
Thanks, onTy Toom! Right now we're on AWS, using a t2.small server.