Instana AutoTrace: Fully Embracing OpenTelemetry

Instana AutoTrace: Fully Embracing OpenTelemetry

In my last blog post, I talked about the history and rapid adoption of the OpenTelemetry project. Today we discuss the progress made in terms of end-to-end compatibility between distributed tracing with OpenTelemetry and Instana AutoTrace, and how we are embracing OpenTelemetry as a first-class citizen in Instana.


Despite the foundation provided by W3C Trace Context, end-to-end interoperability is actually much harder than it sounds.


Normally, vendors have to integrate code upstream, inside the OpenTelemetry SDKs you deploy in your code and infrastructure, in order to receive data in a format they can process. This creates all types of headaches because you may need to rebuild and redeploy your applications to add the missing libraries. This task might not be a “biggie” for a PoC, but it becomes daunting at scale when you need to wrangle tens of teams and hundreds of code bases.

At Instana, we really do not want to give you libraries to integrate in your code. One of the fundamental product philosophies of Instana is: “Deploy the Instana agent; it does the work for you.” And we wanted to achieve the same refreshing simplicity with OpenTelemetry, which is why we came up with a unique, super cool way of getting the full value of both Instana and OpenTelemetry without changing any code!


In Instana you will see traces associated with the infrastructure powering those applications, which is one of the killer features of AutoTrace and Unbounded Analytics.


Instana’s support for OpenTelemetry is predicated on:

  • Native support within the Instana agent for OpenTelemetry’s otlp protocol that OpenTelemetry uses to transmit data
  • Seamless integration with OpenTelemetry applications in Instana’s Dynamic Graph

The support for otlp is straightforward: you point the OpenTelemetry SDK, typically by means of one environment variable, to your local Instana agent, and let the telemetry be delivered to your Instana backend. Today we support collecting traces; metrics support will come in the next few releases.

The Instana service dependency map.
The Instana service dependency map.

End-to-end distributed tracing with OpenTelemetry and Instana’s AutoTrace™

Instana AutoTrace and OpenTelemetry support out-of-the-box W3C Trace Context, which is the foundation for end-to-end traces mixing spans (the building blocks of traces in both AutoTrace and OpenTelemetry) generated by the two different tracing systems into one trace.

However, despite the foundation provided by W3C Trace Context, end-to-end interoperability is actually much harder than it sounds. There are a lot of corner cases to deal with in terms of trace gaps, e.g., if one or more applications is not reporting tracing data. For example, we can know in most cases whether we have a gap, but we do not know how large it is (one span or more?) or if the data will come in late(r). Additionally, we had some assumptions to clear from our instrumentation and trace data processing about how trace identifiers are created.

Integrating OpenTelemetry in Instana’s Dynamic Graph

Instana supports hundreds of technologies, and the rich telemetry it collects is correlated by means of semantic relationships that describe how the different entities in your infrastructure — like hosts, zones, containers, namespaces, clusters and all the rest — fit with one another. All this information is available at your fingertips, for example, via the Context Guide. The data powering the Dynamic Graph is collected by specific sensors in the Instana agent, and we meld this data with the data sent to Instana by OpenTelemetry in a way that is unique and novel. When receiving otlp data, the Instana agent will find out which process on the same host is sending the data, and send to the Instana backend data about the process, the container it runs in, etc. In this way, in Instana you will see traces associated with the infrastructure powering those applications, which is one of the killer features of AutoTrace and Unbounded Analytics.

Automatic discovery by the Instana Dynamic Graph.
Automatic discovery by the Instana Dynamic Graph.

Next steps

There are a few areas in which we definitely plan to further invest with regards to OpenTelemetry:

  • Trace Semantic conventions: Similar to OpenTracing and Instana’s own AutoTrace tagging best practices, OpenTelemetry issues guidance on how to tag particular pieces of metadata on spans. We intend to create mapping inside Instana from those semantic conventions to AutoTrace’s. When compliant with the semantic conventions, your OpenTelemetry tracing data will look their best in Instana!
  • Resources: Resources are a concept in OpenTelemetry akin to Instana’s own entities. You can specify key-value pairs describing different aspects of the application emitting telemetry, from host and process data to cloud provider attributes. There is a certain amount of overlap between the data exposed by OpenTelemetry over resources and what Instana’s Dynamic Graph entails, and we are interested in exploring how the two concepts can work and help one another.
  • Metrics: OpenTelemetry not only provides APIs and implementations to collect trace data, but also metrics (and logs, see next point). Metrics is getting stable enough to start being adopted. While ecosystems like the Java Virtual Machines have widely adopted libraries to express metrics (micrometer.io and Dropwizard above all others, both supported natively by Instana), we think there is significant potential for adoption in other runtimes. Just as we support pushing OpenTelemetry traces from your app to the Instana agent, we intend to do the same with OpenTelemetry metrics, and offer you the same seamless, delightful experience that you already experience with Instana’s micrometer.io and Dropwizard integrations across all runtimes.
  • Logs: The same thing we said about metrics may be applicable to logs, although the Logs part of OpenTelemetry is really far, far away from being ready for adoption. Also, while many runtimes do not have strong contenders in terms of which API to use for logs, every single app out there already has logging enabled with existing libraries. It remains to be seen how much traction the logging portion of OpenTelemetry can gain.

Conclusions

It has never been easier to realize the full potential of your investments in OSS. With Instana’s seamless support for OpenTelemetry, users now have the same level of visibility and correlation that they get with Insana’s AutoTrace technology. If you’re an Instana customer, you can see the power of Instana’s OpenTelemetry support, now in Tech Preview, simply by asking your Customer Success Manager for access. If you don’t already have an Instana instance, you can try Instana for free with a no strings attached free trial today. OpenTelemetry support will be coming to the trial and every Instana instance in the near future.

Play with Instana’s APM Observability Sandbox

Developer, Thought Leadership
In the past few years, distributed tracing has emerged in the global DevOps consciousness as an indispensable tool in the microservices arsenal. In April 2019, the open source observability community rose to...
|
Announcement, Conceptual, Developer, Engineering, Product
According to AWS: “[Graviton2 is] custom built by Amazon Web Services using 64-bit Arm Neoverse cores to deliver the best price performance for your cloud workloads running in Amazon EC2” At Instana...
|
Announcement, Developer, Product, Thought Leadership
AWS Lambda supports development and deployment of functions in a variety of programming languages including Node.js, Go, Java, and Python. Instana has previously announced automated tracing of AWS Lambda Functions for Node.js and...
|

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit instana.com.