At a previous agency, I did a lot of things with Mixpanel for event and push tracking. With push notifications (or any event), I tend to divide things into two categories (though I like Devin's layout as well).
- Direct engagement - this is particularly good for global notifications, updates, or other important announcements. Measurements for this are really around interactions (did people read it, did they perform the action requested, etc).
- Attributable engagement - this one is a bit more tricky because it has to be approached a bit differently, but basically you want to attribute having received the notification(s) to an increase in desired behavior. For example, does a notification to inactive users tend to re-engage them and increase retention. Or does your notification(s) lead to an increase in paid purchases, or increase average lifetime value, or a variety of other metrics depending on your goals for the app and that particular user segment.
Aside from general auto-notifications ("you've got mail"), Mixpanel lets you build out segments based on a variety of criteria, which is the really powerful part for influencing specific user behaviors. So you can build segments of people who have (or have not) received specific messages or specific types of messages. Also, you can use other event data to construct specific types of segments based on specific behaviors/non-behaviors (i.e. people who haven't used the app in X days, people who have never commented, people with an incomplete profile, people with X number of friends within the app, people acquired by specific marketing campaigns, etc, etc, etc.). From there, you can also construct messages that are specific to that segment.
For example, if your segment is 'people who have only added a few friends,' perhaps you send them a notification instructing them how to add more friends (and then track whether people who received the "add friends" notification actually added anyone (and how many people they added)). This gives you a marketing-automation-style tool built right into your analytics tool, which is pretty sweet.
The other side of this is that you can do some awesome comparison testing. Most simply, say for example you wanted to know if receiving any push notification increases retention (or is correlated, at least). Or do people who mute/opt-out/opt-in engage with the app differently. Or if Message A is better at increasing [goal] than message B. Or how frequency of messages impacts behavior. All of this requires that you have appropriate event tracking in place, but it is pretty simple to setup.
Mixpanel does require a lot of custom tagging and a bit more technical savvy than something like Flurry, but it is extremely powerful in the right hands.
Overall, Mixpanel was and is by far my favorite for app analytics, although Google Analytics for Mobile has since gotten much better, and there are a variety of other tools as well. But since Mixpanel does a lot more within the system for not only analyzing the data, but also acting on it (though messaging), it is hard to beat.
(Disclaimer: Everything is my own opinion and does not necessarily reflect any opinion, strategy, or recommendation of my current or past employers, or clients)).