Reporting platforms · Business intelligence

Looking for recommendations for Reporting / Dashboard / Analytics / BI technology

Yaron Rosenbaum Software Development Executive

July 15th, 2014

By way of context: WhatsOn is a unique platform that enables broadcasters (TV), format rights owners and brands to enhance engagement via real-time social tracking, big-data analysis and profiling.

As part of our service, we provide our customers analytics, reporting, and online dashboards that give them insights into various aspects of their brand/product/show, and the engagement around it in the social sphere.

This is an area of the product that is constantly expanding and changing, with every customer feedback we get.

Creating the report templates / dashboards manually in HTML doesn't seem like the right approach, because it is time consuming, error prone, and requires Web skill sets that I rather use somewhere else. Plus, a good friend asked me 'why don't you use BI tools?' :-)

There are, it seems, a lot of platforms out there - ranging from reporting infrastructures like BIRT to full-blown BI platforms like SpagoBI; there's a full spectrum of pricing models behind these products, from free open source to proprietary with yearly license at tens of thousands of $ per year.

As a small startup, I am automatically drawn to the free, open-source edge of the spectrum, but I know that there may be hidden costs. Some open source projects are just too complex to really drive on your own, or lack the level of community engagement that could improve the learning curve; Some are just not 'enterprise grade' and some are great - easy to use, quick to learn, high quality etc.

I wanted to ask the community here about your experience with such platforms, hear your recommendations, and possibly ask you to share some numbers (cost (license? support? hosting? consultancy?), time, quality, etc)

Converting the information in google into knowledge is a challenge - I would like to tap into the collective knowledge of this group if possible.


David Crooke Serial entrepreneur and CTO

July 16th, 2014

I recently looked into a bunch of these tools for a client, who will be reselling with dashboards and intelligence to their customers. There is a wide range of features and you'll need to decide what you need. I found BIRT and it's kin to be very overpriced, the low end version of MicroStrategy just didn't work. I ended up going with a vendor called ChartIO - their stuff is dashboards not BI, but it's visually slick, it works well including for power users, they push new features very rapidly and support is very responsive. I am impressed.

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

If you're providing "analytics" and "insights" currently, then you're doing more than hacking up HTML pages. Where is your data stored? How are you currently analyzing it to create the HTML reports you mention in your question?

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Shadakshari Swamy Healthcare IT, Product Guy ,Blogger, Mentor

July 16th, 2014

Hi Yaron

You may try tools like Qlikview, it offers intuitiveness and flexibility to do lot of self serving BI stuff. But comes with a cost :)., as i remember they have different packages such as small, enterprise and even a cloud based!.
 

Paul Smullen CTO/Founder focused on data and integration

October 9th, 2020

Hi Yaron. If you presenting reports on a web site or App at high volumes of users then your best bet is to use a JavaScript library in whichever tech you are using e.g. React. However building the ability to slice and dice data using lots of complex filtering etc. will need a proper tool. I always suggest to developers to use both and give the simple reports with some basic search/filtering to all and then give something more elaborate to the few that need real Analytics. For the latter we use Microsoft Power BI. Which is free as a Desktop App but can be expensive as a licensed embedded application. 'Expensive' is always relative of course. Most competitors will be similarly priced with pros and cons to each.


I would also suggest that you do most of the heavy lifting on the back end. Get some data pipelines setup and cleanse and pre-crunch your data into a solid data model that easy to consume by any reporting/BI tool and reasonably easy to understand by a non-developer analyst. Never point a BI tool at your core transactional operational database (for lots of good reasons) - always replicate the data to a reporting database or data warehouse. This is the harder bit to get right and I would follow a standard data warehouse design patern for this part - although I would not recommend enterprise data warehouse vendors unless you have a massive volume of data and large budget. We have achieved solid solutions using low end SQL databases with Power BI on top.


I personally lean towards Microsoft/Azure as an end to end solution for what we do but the architecture and approach above is agnostic to what vendor/platform you choose.


Hope this helps.


Paul





Peter Verrykt Cofounder, People focussed entrepreneur, Strategic Account Management, Customer Satisfaction Manager

October 7th, 2020

I've been using a lot of these BI tools. Qlikview has a very strong front-end but like said before, comes with a cost, especially if you grow the usage it can become expensive. I would suggest to take a peek into PowerBI, there is a free version available and it connects natively to your SQL DB.

Sandy Fischler Experiential Marketing Director | Event Producer | Event Management | Entrepreneur

July 16th, 2014

You might want to check out Mixpanel - we're integrating that into a new project. The one thing I can say about them so far is that their onboarding team is awesome. 

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Alan Schunemann CTO and Co-founder of eTelemetry

July 16th, 2014

That sounds like the pitch for AWS hosted software... "start small & grow". Their (or similar service) rates are reasonable to "store everything", so you can sift through it later. It sounds like you were an expert in your previous domain, which gave you an edge when it came time to figure out what to keep and what to toss. I know Mercury well. We used it at USi (USinternetworking...) now part of AT&T. You can improvise, adapt, and overcome, as new data comes down the pike. As long as you have good ETL skills and tools available, you'll be ok. (specifics will be in your list of requirements) The flexibility necessary to adapt to an unknown data source/type likely won't be handled by a magical package you pick today. It's interesting to note that the discussion has shifted from HTML reports to the data. To gain "insight", as you highlighted in your original question, a good set of data visualization tools is a must. (again, an explicit list of requirements is a must, not a "good set" and "tools") With your background, I'm sure you're aware of the shop by feature black hole. A discussion like this, properly distilled into requirements, is a necessary first step. (before looking for "packages") Your company's product road map is critical for generating placeholder future requirements. 1. Storage 1. Currently 2. growth curve points to ? 2. DB 1. SQL Today 2. NoSQL Tomorrow? (likely both) 3. ETL 1. List of current ETL functions required 2. Inventory competitors / your road map / and market for likely needs 4. Analytics (software & hardware/infrastructure) 1. What's the nature of today's number crunching? 2. What does road map indicate about possible future? 5. Reporting/visualization 1. Make list of existing HTML reports and categorize them 2. Road map - again

Yaron Rosenbaum Software Development Executive

July 17th, 2014

@David Crooke

Thanks.
I checked ChartIO and I am impressed as well (I don't know if that's indicative of the product, or just their marketing :-) )

Can you share how their pricing works ?