We're not category specific, but geography specific (non-US; mostly Asia & South America), so maybe it's an analogous situation. Most commonly found offers in the US are not applicable for us. We initially spent some time trying to figure out who had the best coverage for our geography.
In the end, our initial research probably wasn't wrong, but it didn't end up mattering: things change, a lot. Good news is the big players spread their offers around - you'll see a lot of overlap for the big brands no matter who you choose. Equally, the barrier to integration can be pretty low if you're willing to take some time to abstract your interface a bit. It took an iteration (we got it pretty wrong the first time), but it wasn't that painful; now it's trivial to add new affiliate partners we had never even heard of and manage the stream of their offers into our app with little overhead. We're doing what I consider to be a lot of revenue given the dedicated headcount.
Anyway, I recognize this isn't actually a direct answer to your question, so apologies if it's off-topic; just a suggestion that you spend the time you have thinking about how to abstract the affiliate connection a little bit in your product rather than optimize for which affiliate might prove best. You don't need volume to sign up with them - most will take nearly anyone as long as you're nice about it. I would pick 2 or 3 that are easy to find, figure out how to abstract your interface so it will work for all of their feeds (& hopefully the next you haven't seen yet), and plow forward.
Even if you did figure out the right answer today, by the time you have volume, the answer might be different. Some of these guys have been around forever and are stable, but as a whole we see a lot of change - hard to know who will have the hot hand in a year, but we found it easy to aggregate anyone ourselves by investing a little time planning for it.
Said another way, I would not want to be an average affiliate network right now, given how low the switching cost seems to be for both advertisers and publishers... You shouldn't need to feel like you need to focus on just a few; it's possible for you to aim to have it all.