Instana’s Holistic Data Model

September 3, 2020


The Challenge with Managing Ephemeral Application Infrastructure

If a monitoring solution cannot accurately model an environment, how can that solution monitor it effectively?

Application environments have become increasingly more complex as they have evolved from monolithic through Service Oriented Architecture (SOA) to containerised microservices. Along with spiralling complexity, the rate of change has also dramatically increased from one or two releases per year to several per day. The ever escalating complexity coupled with the rapid rate of change presents a significant challenge for monitoring solutions to track and model.

Legacy monolithic applications were directly deployed on to a host and did not change often. A typical data model could be Application – Host.

Service Oriented Architecture (SOA) started to break up monolithic applications into smaller services, it was the start of the journey to microservices. With SOA there are more applications or services deployed on more hosts. Due to the independent nature of the services the rate of change could increase to a release every month. A typical data model for SOA could be Application – Tier – Host.

The move from monolithic to SOA was an evolution, the jump to microservices is more of a revolution. Microservice architecture applications have many more smaller services and are typically deployed inside containers managed by an orchestration tool such as Kubernetes. Along with the move to microservices, application technologies have become more diverse. Legacy applications were primarily Java J2EE and Oracle RDBMS, with microservices there are newer technologies in the mix such as: Node.js, Golang, Python, MongoDB, Redis, .Net Core, etc. Kubernetes also brings its own set of abstractions to be considered: Deployment, Service, StatefulSet, ConfigurationMap, ReplicaSet, etc. A service is no longer tied to a specific host, Kubernetes will run the container on whichever node has capacity at the time. The hierarchy now looks something like:

This is for just one service, typically there are hundreds of services that make up a complete application. To finally twist the knife, Deployments can have multiple replicas, running multiple containers, Services can map to multiple Pods from multiple Deployments. The straw to break the camel’s back is when a service mesh such as Istio is then added on the top, providing another layer of abstraction. Now we’re a long way from Application – Tier – Host.

Just when you think you’ve got a handle on all of this, along comes Serverless. Now there’s just an endpoint running in the cloud, mind blown not to mention the data model.

Connecting Disparate Application Components

With such a diverse set of architectures to model, what would the schema look like that can accurately model them all? If you boil it right down, what you have is a collection of entities that have relationships with each other that change over time. In other words a graph database with a time axis. That’s effectively what we built at Instana and we call it the Dynamic Graph and it’s at the heart of how Instana operates. It ensures that for any point in time Instana knows what is talking to what and where it is running.

The Dynamic Graph can accurately model any application architecture from monolithic, through SOA to serverless and beyond. Because it makes no assumptions about the structure of the application environment it can handle your legacy and current applications as well as being future proof.

Data from the Dynamic Graph brings value to Instana in numerous places:

  • Flow maps
  • Automatic root cause analysis
  • Context guide
  • Just about anywhere there is a drill down

Automatic Microservice Observability and Visibility

Having a precise model of the environment being monitored guarantees error free automatic root cause analysis by only considering those entities with direct relationships for analysis. This completely eliminates any false positives that can happen when using only time based analysis. The context guide provides DevOps engineers with unparalleled visibility of the upstream and downstream dependencies allowing engineers to fully understand the context in which their services are operating.

Play with Instana’s APM Observability Sandbox

Announcement, Developer, Product, Thought Leadership
At Instana, we recently improved the installation process for our self-hosted customers. Instana’s self-hosted platform now utilizes a fully Docker based installation process. In a previous blog post, Lessons Learned From Dockerizing...
An ever-increasing number of System architectures and deployment strategies depend on Kubernetes-based environments. Kubernetes (also known as k8s) is an orchestration platform and abstract layer for containerized applications and services. As such,...
Announcement, Developer, Product, Thought Leadership
To be successful in Observability, you must have the ability to observe the behavior of a system and derive its health state from it. But deriving the health state of any given...

Start your FREE TRIAL today!

As the leading provider of Automatic Application Performance Monitoring (APM) solutions for microservices, Instana has developed the automatic monitoring and AI-based analysis DevOps needs to manage the performance of modern applications. Instana is the only APM solution that automatically discovers, maps and visualizes microservice applications without continuous additional engineering. Customers using Instana achieve operational excellence and deliver better software faster. Visit to learn more.