Reporting platforms · Business intelligence

Looking for recommendations for Reporting / Dashboard / Analytics / BI technology

Yaron Rosenbaum Software Development Executive

July 15th, 2014

By way of context: WhatsOn is a unique platform that enables broadcasters (TV), format rights owners and brands to enhance engagement via real-time social tracking, big-data analysis and profiling.

As part of our service, we provide our customers analytics, reporting, and online dashboards that give them insights into various aspects of their brand/product/show, and the engagement around it in the social sphere.

This is an area of the product that is constantly expanding and changing, with every customer feedback we get.

Creating the report templates / dashboards manually in HTML doesn't seem like the right approach, because it is time consuming, error prone, and requires Web skill sets that I rather use somewhere else. Plus, a good friend asked me 'why don't you use BI tools?' :-)

There are, it seems, a lot of platforms out there - ranging from reporting infrastructures like BIRT to full-blown BI platforms like SpagoBI; there's a full spectrum of pricing models behind these products, from free open source to proprietary with yearly license at tens of thousands of $ per year.

As a small startup, I am automatically drawn to the free, open-source edge of the spectrum, but I know that there may be hidden costs. Some open source projects are just too complex to really drive on your own, or lack the level of community engagement that could improve the learning curve; Some are just not 'enterprise grade' and some are great - easy to use, quick to learn, high quality etc.

I wanted to ask the community here about your experience with such platforms, hear your recommendations, and possibly ask you to share some numbers (cost (license? support? hosting? consultancy?), time, quality, etc)

Converting the information in google into knowledge is a challenge - I would like to tap into the collective knowledge of this group if possible.


Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

If you're providing "analytics" and "insights" currently, then you're doing more than hacking up HTML pages. Where is your data stored? How are you currently analyzing it to create the HTML reports you mention in your question?

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Shadakshari Swamy Healthcare IT, Product Guy ,Blogger, Mentor

July 16th, 2014

Hi Yaron

You may try tools like Qlikview, it offers intuitiveness and flexibility to do lot of self serving BI stuff. But comes with a cost :)., as i remember they have different packages such as small, enterprise and even a cloud based!.
 

David Crooke Serial entrepreneur and CTO

July 16th, 2014

I recently looked into a bunch of these tools for a client, who will be reselling with dashboards and intelligence to their customers. There is a wide range of features and you'll need to decide what you need. I found BIRT and it's kin to be very overpriced, the low end version of MicroStrategy just didn't work. I ended up going with a vendor called ChartIO - their stuff is dashboards not BI, but it's visually slick, it works well including for power users, they push new features very rapidly and support is very responsive. I am impressed.

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

Check out the cloud-based analytics packages offered on Amazon Web Services: https://aws.amazon.com/marketplace/search/results/ref=mkt_ste_l2_analytics_category?page=1&searchTerms=analytics

This should provide a good starting list of packages to review, and test drive, which is one of the benefits of AWS. That's actually bad step 0 advice. Your step 0 should be to sit down with your team and enumerate your requirements in writing. Shopping features first is a sure fire way to not get what you really need.

Yaron Rosenbaum Software Development Executive

July 16th, 2014

@Alan thanks for the quick response!

I have quite a bit of experience with processing large amounts of data, but form another domain. I was the chief architect of Mercury (HP) Topaz (Business Availability Manager) - a monitoring platform. 

So I had the advantage of knowing this is something that requires some thinking and planning in advance. Specifically, I made a bunch of very useful decisions around what data to keep, how to aggregate it, and how to store it so that it will be easily accessible later. But this is of course, limited to a degree, by the types of reports I could think of at the time.

Long story short, the data used for the _current_ reports, is stored in a SQL DB, in specific tables, and very easy to access.

In the future, as we go forward, this may not be the case.
So I'm looking for a 'start small & grow' approach, where the cost of initial implementation will be limited, but I will be able to expand on the same infrastructure as we go forward 

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Mohamed GHAZAL BI & WFM UCCE Engineer chez Expertflow

July 16th, 2014

As i have experience on this platform, there is different kind of platforms (open and paid). For open platform, you can work with : Penatho. For paid platform you can work with : MS SQL Server, Cognos, Clikview.. 

Sandy Fischler Experiential Marketing Director | Event Producer | Event Management | Entrepreneur

July 16th, 2014

You might want to check out Mixpanel - we're integrating that into a new project. The one thing I can say about them so far is that their onboarding team is awesome.