Instana’s Automatic Lambda Tracing: Why Instana Doesn’t Use the New Lambda Extensions

Observability Best Practices for AWS Lambda Functions


UPDATE: We made some corrections to the “What are Lambda extensions” section based on feedback suggested by AWS.

Earlier this month, AWS launched the preview of the Lambda extensions framework. Some players in the Observability space announced their own extensions. In contrast, Instana did not make such a move.

I thought I would explain why we didn’t launch our own extension, why we currently have no plans to do so (despite being kindly offered the opportunity) and — most importantly — why Instana users will miss nothing by NOT having an Instana Lambda extension. In fact, here’s a post describing what Instana delivers for Lambda tracing.

Disclaimer: in this blog we go technical, because we believe the topic deserves it, and that we should really try our hardest to give you the required clarity and depth here.

What are Lambda extensions

According to AWS, Lambda extensions are:

“[Extensions are] a new way for tools to integrate deeply into the Lambda environment. There is no complex installation or configuration, and this simplified experience makes it easier for you to use your preferred tools across your application portfolio today.”

That is, Lambda extensions are fundamentally a set of facilities for third-party providers that couldn’t figure out how to easily insert their tracing to better integrate inside the Lambda runtimes.
The new mechanics of Lambda extensions are:

  • External extensions are fundamentally companion processes (that is, without sandboxing!, and we’ll get to this later). That is, external extensions are other processes running inside the same execution environment as your function, and that can listen to lifecycle events like INVOKE and SHUTDOWN. You add the code to be run as an external extension via a Lambda layer.
  • Internal extensions reside inside your Lambda function, and can listen to INVOKE events. To leverage an internal extension, you usually have to either modify the code of your function, or opt-in via configuration by adding, for example, a package to your Node.js function using the NODE_OPTIONS environment variable or launching a javaagent via JAVA_TOOL_OPTIONS.
  • Wrapper scripts are a way of changing the startup parameters of the code running your Lambda function by controlling the initialization of the runtime. Similarly to internal extensions, they are opt-in, and you need to activate them via the AWS_LAMBDA_EXEC_WRAPPER environment variable.

From a technology perspective, Lambda Extensions are pretty cool and provide plug-and-play value in use-cases in which a companion process provides value. We do have reservations about the lack of sandboxing of external extensions, as those will compete with your function for memory, and you will have to pretty much rely on the vendor not to cause Out of Memory errors. But all in all, the bar of configuring tracing with Instana is already so low that none of the Lambda extension facilities listed above can improve it.

Why Instana does not need a Lambda extension

When AWS reached out to us showing the Lambda extension, we looked at it with interest. We are always on the lookout for new, easier ways to use and roll out Instana. But, for AWS Lambda, we have already created and delivered innovative ways to make it super simple:

  1. No code changes are required, at all
  2. The instrumentation is delivered over a Lambda layer, the same way it would be via an Extension
  3. All you need to set up is configuration, which Instana handily provides in the dashboard:


Word Image 302

In the image above you see a screenshot of the Instana Agent Onboarding screen, providing you at a glance, with copy & paste simplicity, all you need to do to trace your Lambda functions via Instana, including how to configure end-to-end a Lambda function via the aws command line tool.

We thought long and hard how we could leverage the Lambda extension to further improve this already streamlined user experience and, frankly, we came up empty.

If we could either activate an internal extension or a wrapper script automatically by virtue of you including the Instana Layer in your Lambda function, we could automatically trigger our function wrappers. That would allow you to avoid two arguments of the aws command shown in the screenshot above, or two steps in the manual setup using the AWS console. We’d really love to be able to do that, as our goal, from the very beginning of working on our Lambda support, has been “Add the layer, tell where to find Instana, and that’s it!”.
[1]
Alas, this is not something that Lambda extensions supports yet, so there is currently no value for Instana to reap from a Lambda extension.

Summary

Lambda extensions are a new facility in AWS to streamline how 3rd-party providers can deliver tracing and other types of instrumentation into Lambda functions. However, Instana’s methodology, designed with AWS Lambda in mind, is already so simple that Lambda extensions provide no extra capabilities or functionality to further simplify installation / configuration.

Of course we remain on the lookout for ways of making Instana Lambda tracing better and even simpler to install and run. We look forward to continuing to monitor (see what I did there?) how Lambda extensions develop. If we find a “killer application” for Instana + Lambda extensions, you can be sure we will build it quickly, of course.

  1. Rumor has it that Michele, our Product Management for all things tracing, had gotten rather obsessed for a while trying to find ways of achieving the injection of environment variables from the Lambda layer, much to the neverending joy of our engineers (above all, Bastian Krol). Michele is known to occasionally still fall down that particular rabbit hole, which made overall Lambda Extensions a rather cruel thing to show him, as Lambda extensions make easier solving problems we had already solved.

Play with Instana’s APM Observability Sandbox

Engineering, Product
As of the latest release, Instana supports the monitoring of Ruby applications running on Fargate, a serverless container orchestrator managed by Amazon Web Services. This enables Ruby teams to take advantage of...
|
Featured, Product, Thought Leadership
Instana prides itself in being the first Observability tool to launch support of Google Cloud Run via a Cloud Native Buildpack. The Instana Cloud Native Buildpack for Cloud Run makes adding Instana...
|
Announcement, Conceptual, Developer, Engineering, Product
According to AWS: “[Graviton2 is] custom built by Amazon Web Services using 64-bit Arm Neoverse cores to deliver the best price performance for your cloud workloads running in Amazon EC2” At Instana...
|

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit instana.com.