Open Source Microservices Monitoring is a Terrible Idea

The Methodology Behind the APM Experts Research and Whitepaper

Bernd Harzog is the CEO of APM Experts and the primary author of the recently published whitepaper: Open Source Microservices Monitoring is a Terrible Idea

When we began researching the full cost implications of manual monitoring methodologies using Open Source technology, I wasn’t sure what I would find. Sure, everyone knows that using open source, especially in IT shops, has costs that go beyond the “free” licensing. Analysts have been writing about that for years. What surprised me was just how much ongoing manpower costs open source monitoring creates. Depending on the size and number of entities being monitored, it could be well into 6 figures — ANNUALLY!

Methodology

It’s important to understand the methodology used when creating research, especially operational research. Here’s a summary of our research methodology:

User interviews

We spoke with DevOps teams form companies of various sizes – from small private companies to some of the largest corporations in the world. Each group had experience, if not active monitoring, with both Open Source manual technology, as well as new automated tools

Costing Model

Using $100 / hour on average for the cost of a fully loaded developer’s time, and based on the information gleaned from the interviews, a costing model was built. Regardless of team size, the model indicated an interesting correlation between the total cost and the number of microservices being managed.
Second interviews
After developing the model, we went back to the DevOps teams, plus a few more, to confirm the assumptions, the hour estimates, the correlation to counts, and the overall time / manpower costs. All confirmed that the model matched the reality of their situation.

Conclusions

Ultimately, the conclusion ended up being the title of our whitepaper: Open Source Microservice Monitoring is a Terrible Idea. Not only does it have up-front manpower cost implications, but over time, those costs escalate. You can read the full whitepaper to see the whole set of research.

P.S. I’m also doing a webinar with Instana on December 5th titled Yes Virginia, Automatic Monitoring Does Exist!. If you attend the live event, you can ask directly how some of this research might impact your team.