Hello, fellow Canadian. I have a trusted cofounder in Ukraine, and our business is registered in Delaware, though I am neither a lawyer nor an accountant.
i. All income that you extract from the business is taxable in Canada, and you can consult with a good tax lawyer here to determine the best way to receive the money, be it wages, dividends, etc. Bottom line, partly because New Zealand is one of many countries around the world with a tax treaty with Canada, you won't have to worry about personal income taxes in New Zealand. And there are no non-sketchy ways of avoiding taxes in Canada on your international income.
ii. Depending on your family situation, age, medical history and appetite for risk, you may want to consider setting up a Canadian subsidiary to pay your salary. This adds some overhead (rather significant at an early stage) but allows you to participate in Employment Insurance -- including disability -- and contribute to CPP. Your first salaried, non-contract employee in Canada will require you to set this up in any case.
iii. An internationally distributed team allows you to take advantage of international tax arbitrage, and play with offshore IP, reinvoicing, out of country sales and the like. There are strategies that progress along a spectrum from conservative to aggressive, and you'll want to engage a reputable international taxation expert that matches your preference. Whether this stuff is worth it or not depends entirely on the prospects of your business. If you are building a $10m business, chances are probably not, but 9 or 10 figures certainly make it worth it. And as tempting as it is to put this off till you're wildly profitable, it requires far fewer legal back flips to get it right at the start than to adjust everything later in the game. Don't forget to survey potential investors on their appetite for this as well, if you'll be going to that well!
If you have more mundane questions about the day to day, happy to chat. I've been working with overseas partners for quite sometime. firstname.lastname@example.org | (647) 297-7126 | skype 'ericrogness'