David, great question, I'll try to give as unbiased advice as I can.
For your company, I would consider this first question when thinking about whether you should employ a 3rd party or acquire the knowledge in-house. Does this knowledge need to be a core competency of my company? For instance, if the backbone of your company is based on content marketing, then I would highly recommend you make sure you employ that in-house. Possibly seeking support from a 3rd party for consultation or on projects that exceed your staff's understanding or capabilities.
Regarding analyzing ads, placements, and campaigns for effectiveness, I would recommend creating an analytical framework for how you hold advertising accountable for it's ROI. It's easier then it sounds, you'll want to create the right KPI's to push anyone handling your marketing to report to you and be accountable for. There are metrics that are used in the day-to-day of campaigns that will vary by preference and channel, but typically a lot of the KPI's that are at the core of your business are quite similar. Cost per acquired customer, estimated lifetime value of an acquired customer, etc. Once you have a really solid analytical framework in place, you're less likely to fall for a smoke & mirrors type deal, whether it's in-house or agency.
A lot of what you mentioned definitely bridges a few marketing disciplines. Ad creation is typically more creative, digital advertising has it's own disciplines (PPC, SEO, etc), offline advertising follows another specific skill set. You can find good marketers that span a few disciplines, but I'd only trust them with a mix like that if they really got to know your company very well (so, in house or an agency you work very closely with).
Recruiting real professionals in a short time span is always a challenge. Personally, the biggest pit-fall I've found with this is finding people that take credit for ideas that were passed to them from above. Someone has a nice title, has the experience, etc, but didn't really conceive the ideas he's taking credit for. So, to counter this, make sure you get hypothetical and make them brainstorm on their feet in the interview. I would run a LinkedIn paid job posting first before going to a recruiter. I've used smartrecruiters.com to push out job postings in the past and it makes the hunt a little easier. If you're going to use a recruiter, negotiate a good rate beforehand, then start your hunt. If you'd like, I have a couple that I've connected with in my network that I think are decent, I can refer you.
Disclaimer: I do work for an eCommerce agency that provides most of the services you've listed, I run the marketing department. If you'd like to connect & chat further, I'd be open to offering whatever advice I can.