Instana’s Holistic Data Model

September 3, 2020

Instana's Holistic Data Model

The Challenge with Managing Ephemeral Application Infrastructure

If a monitoring solution cannot accurately model an environment, how can that solution monitor it effectively?

Application environments have become increasingly more complex as they have evolved from monolithic through Service Oriented Architecture (SOA) to containerised microservices. Along with spiralling complexity, the rate of change has also dramatically increased from one or two releases per year to several per day. The ever escalating complexity coupled with the rapid rate of change presents a significant challenge for monitoring solutions to track and model.

Legacy monolithic applications were directly deployed on to a host and did not change often. A typical data model could be Application – Host.

Service Oriented Architecture (SOA) started to break up monolithic applications into smaller services, it was the start of the journey to microservices. With SOA there are more applications or services deployed on more hosts. Due to the independent nature of the services the rate of change could increase to a release every month. A typical data model for SOA could be Application – Tier – Host.

The move from monolithic to SOA was an evolution, the jump to microservices is more of a revolution. Microservice architecture applications have many more smaller services and are typically deployed inside containers managed by an orchestration tool such as Kubernetes. Along with the move to microservices, application technologies have become more diverse. Legacy applications were primarily Java J2EE and Oracle RDBMS, with microservices there are newer technologies in the mix such as: Node.js, Golang, Python, MongoDB, Redis, .Net Core, etc. Kubernetes also brings its own set of abstractions to be considered: Deployment, Service, StatefulSet, ConfigurationMap, ReplicaSet, etc. A service is no longer tied to a specific host, Kubernetes will run the container on whichever node has capacity at the time. The hierarchy now looks something like:

This is for just one service, typically there are hundreds of services that make up a complete application. To finally twist the knife, Deployments can have multiple replicas, running multiple containers, Services can map to multiple Pods from multiple Deployments. The straw to break the camel’s back is when a service mesh such as Istio is then added on the top, providing another layer of abstraction. Now we’re a long way from Application – Tier – Host.

Just when you think you’ve got a handle on all of this, along comes Serverless. Now there’s just an endpoint running in the cloud, mind blown not to mention the data model.

Connecting Disparate Application Components

With such a diverse set of architectures to model, what would the schema look like that can accurately model them all? If you boil it right down, what you have is a collection of entities that have relationships with each other that change over time. In other words a graph database with a time axis. That’s effectively what we built at Instana and we call it the Dynamic Graph and it’s at the heart of how Instana operates. It ensures that for any point in time Instana knows what is talking to what and where it is running.

The Dynamic Graph can accurately model any application architecture from monolithic, through SOA to serverless and beyond. Because it makes no assumptions about the structure of the application environment it can handle your legacy and current applications as well as being future proof.

Data from the Dynamic Graph brings value to Instana in numerous places:

  • Flow maps
  • Automatic root cause analysis
  • Context guide
  • Just about anywhere there is a drill down

Automatic Microservice Observability and Visibility

Having a precise model of the environment being monitored guarantees error free automatic root cause analysis by only considering those entities with direct relationships for analysis. This completely eliminates any false positives that can happen when using only time based analysis. The context guide provides DevOps engineers with unparalleled visibility of the upstream and downstream dependencies allowing engineers to fully understand the context in which their services are operating.

Play with Instana’s APM Observability Sandbox

Developer, Thought Leadership
In my last blog on preparing for Black Friday, I teased how easy it is to gain insight into the state of your eCommerce retail applications and all its components with an...
Instana, an IBM company, has been recognized as a Customers’ Choice in the 2021 Gartner Peer Insights ‘Voice of the Customer’: Application Performance Monitoring, and we are thrilled. Our team takes great...
Developer, Product, Thought Leadership
In my last blog post, I talked about the history and rapid adoption of the OpenTelemetry project. Today we discuss the progress made in terms of end-to-end compatibility between distributed tracing with...

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit