Productivity · Developers

When Evaluating Tools, What Factors Do You Consider?

Owen Rubel Creator of API Chaining, IO State, API I/O Abstraction and modern API Automation

September 24th, 2014

I do alot of enterprise api development and find that most testing tools I need have to span the entire I/O flow and not just the application. I also am building my application as a microservice and am moving to spring-boot which is a micro service framework and eventually to Grails 3.0 which will be based on spring-boot.

When you are building your applications and determining what you need, what factors influence your decisions? Support, community, plugins? Or is hype and a friends recommendation enough to sway your head (I know it has done it for me)...

Simone Brunozzi Vice President and Chief Technologist, Hybrid Cloud at VMware

September 25th, 2014

Owen,
I agree that most of the times we're influenced by what we know or hear from close colleagues and friends.

Having said that, you should take a look at Sysdig (by Draios): http://www.sysdig.org/
These are the same guys that built Wireshark years ago.
The tool is designed to help you troubleshoot your entire fleet. I'm pretty sure that they will also release a paid version with enterprise-grade service.

Heather Wilde

September 28th, 2014

I start out by asking around, certainly. 

I also read, read, read. Everything changes so frequently something new may be the best thing, and you will only find out about it by doing your research.

Once you identify two or three possibilities, you need to put them to a test phase. Never simply deploy a solution without running them through your own test cycle.

There's no guarantee that what works for my company will work for yours.

Troy Gardner Chief Technology Officer, Chief Brewing Officer at Cloud9 Brewing Systems

September 28th, 2014

It depends on the nature of the app/product service.

Much of it comes down to goals and architecture decisions, many tech stacks could be used to implement.

With the CTO hat on, i look for standardization of syntaxes (e.g. settling on ecmascript, or javascript) across tiers wherever possible, combined with a vibrant community of developers (as evidenced by conferences, meetups, github)

For example, React.js (by facebook, instagram) is component oriented in plain javascript so can be tested without needing fullblown browsers (e.g. Selenium) and similarly rendered serverside for those clients who don't support javascript.(e.g. google bot for seo) if you're using Node.js or similar for the webserver.

Language wise I'm a fan of Haxe, which allows targeting multiple language targets from a typed ecmascript syntax, allowing capturing of some issues before testing in the app/browser.

DB wise I'm a fan of Redis as it's in memory (backed up by database) structures like queues and pub/subs are good backbone to most apps aiming for real time. As well many of it's structures (like record expiring) aren't things that developers have to implement, thus less overhead to test.

Owen Rubel Creator of API Chaining, IO State, API I/O Abstraction and modern API Automation

September 28th, 2014

Well api done in application is an interface to a separation of concern, API flow however extends out to architecture and thus is architectural cross cutting concern (or 'shared concern'). Thus a contradictory state is created across the I/O of a request/response in the architecture creating duplication and entanglement.

You also can't just test locally at the developer since the I/O flow can be recursive (ie I/O monads) and follows the architectural concerns and api flow.

Troy Gardner Chief Technology Officer, Chief Brewing Officer at Cloud9 Brewing Systems

September 28th, 2014

As we both said...it depends.

Consider it this way, the percentage of your application that can be tested, relative to the proximity to the developer working on it. The more the work is custom components, the better it is to have this tested as close to developer at development time in the same language as, rather than throwing it over the wall and hoping some QA person will get to all possible edge cases.

I not following you on API cross curing/cutting concerns.  I work in websockets/long polling asynch things, both between clients and server and server to server.  Clients make requests and sometime later they are called back based on whatever they are subscribed to.

Async is inevitable. Any scalable solution will eventually get into message passing, queues, sharding, load balancing, and the like.

For ad-hoc reporting yeah I can dump it into RDBMS as needed, but generally I think that most abuse RDBMS for areas like real time apps and analytics/games where I work in.

Owen Rubel Creator of API Chaining, IO State, API I/O Abstraction and modern API Automation

September 28th, 2014

Well that's not necessarily true. depending on what you are doing in React, you still have to test in browser as you can serve additional JS and JS scripts. Assuming you dont have to will definitely cause you to have issues because React does not take into consideration all needs and custom needs.

And React/Node does not take into consideration architectural cross curing concerns in the api when scaling thus creates redundancy and duplication.. It is also impossible to to forwarding within the same request and have to leave and come back in via a secondary request creating additional overhead; I/O monad redirect handles this better.

EhCache/NoSql storage in conjunction with RDBMS for many-tomany index linkage is always a good idea but should be based on application needs.

Owen Rubel Creator of API Chaining, IO State, API I/O Abstraction and modern API Automation

September 28th, 2014

@Simone,

Sysdig is an Apache only tool which is great for scripting languages and small to middle tier businesses.

I deal mostly with Tomcat, Java, Spring, Groovy/Grails and have tools already for that kind of thing but thanks.