Instana’s Holistic Data Model

September 3, 2020

Instana's Holistic Data Model

The Challenge with Managing Ephemeral Application Infrastructure

If a monitoring solution cannot accurately model an environment, how can that solution monitor it effectively?

Application environments have become increasingly more complex as they have evolved from monolithic through Service Oriented Architecture (SOA) to containerised microservices. Along with spiralling complexity, the rate of change has also dramatically increased from one or two releases per year to several per day. The ever escalating complexity coupled with the rapid rate of change presents a significant challenge for monitoring solutions to track and model.

Legacy monolithic applications were directly deployed on to a host and did not change often. A typical data model could be Application – Host.

Service Oriented Architecture (SOA) started to break up monolithic applications into smaller services, it was the start of the journey to microservices. With SOA there are more applications or services deployed on more hosts. Due to the independent nature of the services the rate of change could increase to a release every month. A typical data model for SOA could be Application – Tier – Host.

The move from monolithic to SOA was an evolution, the jump to microservices is more of a revolution. Microservice architecture applications have many more smaller services and are typically deployed inside containers managed by an orchestration tool such as Kubernetes. Along with the move to microservices, application technologies have become more diverse. Legacy applications were primarily Java J2EE and Oracle RDBMS, with microservices there are newer technologies in the mix such as: Node.js, Golang, Python, MongoDB, Redis, .Net Core, etc. Kubernetes also brings its own set of abstractions to be considered: Deployment, Service, StatefulSet, ConfigurationMap, ReplicaSet, etc. A service is no longer tied to a specific host, Kubernetes will run the container on whichever node has capacity at the time. The hierarchy now looks something like:

This is for just one service, typically there are hundreds of services that make up a complete application. To finally twist the knife, Deployments can have multiple replicas, running multiple containers, Services can map to multiple Pods from multiple Deployments. The straw to break the camel’s back is when a service mesh such as Istio is then added on the top, providing another layer of abstraction. Now we’re a long way from Application – Tier – Host.

Just when you think you’ve got a handle on all of this, along comes Serverless. Now there’s just an endpoint running in the cloud, mind blown not to mention the data model.

Connecting Disparate Application Components

With such a diverse set of architectures to model, what would the schema look like that can accurately model them all? If you boil it right down, what you have is a collection of entities that have relationships with each other that change over time. In other words a graph database with a time axis. That’s effectively what we built at Instana and we call it the Dynamic Graph and it’s at the heart of how Instana operates. It ensures that for any point in time Instana knows what is talking to what and where it is running.

The Dynamic Graph can accurately model any application architecture from monolithic, through SOA to serverless and beyond. Because it makes no assumptions about the structure of the application environment it can handle your legacy and current applications as well as being future proof.

Data from the Dynamic Graph brings value to Instana in numerous places:

  • Flow maps
  • Automatic root cause analysis
  • Context guide
  • Just about anywhere there is a drill down

Automatic Microservice Observability and Visibility

Having a precise model of the environment being monitored guarantees error free automatic root cause analysis by only considering those entities with direct relationships for analysis. This completely eliminates any false positives that can happen when using only time based analysis. The context guide provides DevOps engineers with unparalleled visibility of the upstream and downstream dependencies allowing engineers to fully understand the context in which their services are operating.

Play with Instana’s APM Observability Sandbox

Featured, Thought Leadership
The Muddy Messaging of Observability and Application Performance Management Here's a question I get asked quite a bit: “How is Enterprise Observability different from APM and/or just plain Observability?” It’s a reasonable...
Featured, Product, Thought Leadership
Instana prides itself in being the first Observability tool to launch support of Google Cloud Run via a Cloud Native Buildpack. The Instana Cloud Native Buildpack for Cloud Run makes adding Instana...
CircleCI is a CI/CD platform that lets teams build and deliver software quickly and at scale. They make delivering great software simple in both the cloud and on your own infrastructure. New...

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit