Blazing fast Ad-Hoc, Multi-Dimensional Aggregations at your Fingertips

May 19, 2021

Running a complex system landscape comes with its own set of challenges. With microservices architectures this problem becomes more prevalent though. To understanding the behavior of a specific technology, like a database, or the JVM, across all microservices is a common issue to quickly find issues, foresee scalability bottlenecks before they happen, or to plan a technology update and get a quick overview if the new version behaves or performs better, worse or the same as before.

Using Custom Dashboards for Infrastructure Metrics and their composable aggregations, Instana provides a way to build out dashboards to answer those kinds of questions. Furthermore, multiple such aggregations can be combined into a single dashboard, as well as any kind of technology or application specific metrics (such as error rate).

Technical Infrastructure Metrics

At Instana we’re always talking about the connection between independent chunks of data, the context. While the data contain the actual elements of evidence to find issues and understand behavior, only the context between such elements brings out the meaning. It’s like being a detective trying to find the driving force behind a case. For that reason, Instana always is hard at work to bring connections, more context, to the table and increase the understanding of all the components in a system and how they work together.

With the Infrastructure Metrics data storage, Instana stores technical system metrics of your infrastructure, operating system and similar components, in a highly optimized data store and connects it to all other components in Instana’s Dynamic Graph. Internally, the store automatically generates and optimizes a variable set of exponential histograms to provide blazing fast ad-hoc statistical aggregations over multiple, user-composable data dimensions.

Instana Custom Dashboard with Docker Metric PercentilesPicture 1: Technical metrics with multiple percentiles

While the usage of histograms is not new, and users already build that kind of data stores already, the configuration and setup is tedious and complicated. Instana automatically collects the necessary data, analyzes it and configures the histogram based data storage. It also updates it to always provide the best architectural basis for aggregations such as percentiles as an unseen speed.

Custom Dashboards for Infrastructure Metrics

Built upon the optimized histogram data storage, Instana offers the possibility to build customized dashboards to answer questions on a broader technical level. While most people look for metrics on very detailed, very specific components of the architecture, sometimes we want to answer questions from the bird’s eye view. I mean, if we have a system with 1000s of JVMs, we are not interested in a single outlier, but in the high percentiles.

These broader questions include information about the general update or upgrade level of operating systems (and images), but also the performance level of a specific technology, such as the JVM. Is one version better or worse than another one? Is the garbage collector running more often in staging where we test a new JVM deployment. Many such questions have to be answered across all of the application landscape.

Instana Percentile Graph on JVM GC by JVM VersionPicture 2: JVM GC times by JVM Version correlated with Docker CPU Throttling

With Custom Dashboards for Infrastructure Metrics, users can compose their own set of statistically relevant aggregations and on any dimension and slice of components they need. On top of that, those broader aggregations can be combined with application specific metrics such as error rates or response times, giving an in-depth understanding of how things like increasing / decreasing GC times or physical memory usage may influence the performance of applications as a whole.

Compose your own Dashboard Aggregations with the Widget WizardPicture 3: Compose your own dashboard aggregations

Since Custom Dashboards can be edited, imported and exported as JSON documents, my colleague, Matthias Luebken, created a set of ready to use dashboard templates, and made them available on his Github account. It is just important to mention that this is his private project and not officially supported by Instana, however, I bet he’d be happy to see one or two pull requests 🙂

Context is Key

As mentioned above, the importance of context between the otherwise disconnected information pieces is often underestimated, while the opposite is true. Context increases the speed to understand a situation, how elements influence or depend on each other. Context is key. The level of components we want to put into context depends on our question though, and with the level of component granularity changes, too. Sometimes we need a more specific view, sometimes we need a broader one.

For that reason, there is one more important element, which will make everyone happy, who’d fallen into the trap before: Instana will not forget about components or entities after they disappear. If you select a timespan, Instana takes all elements into account that have been alive at any point in time during this time.

Answering the CPU usage across all clustersPicture 4: Answering the CPU usage across all clusters

For further information on the Custom Dashboards, see the Instana documentation.

Anyway, whatever your question is, Instana is here to deliver the context at the level you need. Start to use it today. If you don’t have an Instana account yet, sign up for an Instana Trial right now and get the level of visibility and contextual information you need to solve incidents fast.

Play with Instana’s APM Observability Sandbox

Customer Stories, Featured, Thought Leadership
A few short years ago, contactless car rentals were the realm of science fiction. If you wanted a car, you had to fill out and sign half a dozen forms and wait...
Announcement, Developer, Product, Thought Leadership
To be successful in Observability, you must have the ability to observe the behavior of a system and derive its health state from it. But deriving the health state of any given...
Customer Stories
Innovation is embedded in TIH Investment Holdings’ DNA. From launching Auto & General Insurance in 1985, to launching an initiative to assist Black-owned Exempted Micro Enterprises in taking their business to the...

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit