UPDATE: We made some corrections to the “What are Lambda extensions” section based on feedback suggested by AWS.
Earlier this month, AWS launched the preview of the Lambda extensions framework. Some players in the Observability space announced their own extensions. In contrast, Instana did not make such a move.
I thought I would explain why we didn’t launch our own extension, why we currently have no plans to do so (despite being kindly offered the opportunity) and — most importantly — why Instana users will miss nothing by NOT having an Instana Lambda extension. In fact, here’s a post describing what Instana delivers for Lambda tracing.
Disclaimer: in this blog we go technical, because we believe the topic deserves it, and that we should really try our hardest to give you the required clarity and depth here.
What are Lambda extensions
According to AWS, Lambda extensions are:
“[Extensions are] a new way for tools to integrate deeply into the Lambda environment. There is no complex installation or configuration, and this simplified experience makes it easier for you to use your preferred tools across your application portfolio today.”
That is, Lambda extensions are fundamentally a set of facilities for third-party providers that couldn’t figure out how to easily insert their tracing to better integrate inside the Lambda runtimes.
The new mechanics of Lambda extensions are:
- External extensions are fundamentally companion processes (that is, without sandboxing!, and we’ll get to this later). That is, external extensions are other processes running inside the same execution environment as your function, and that can listen to lifecycle events like
SHUTDOWN. You add the code to be run as an external extension via a Lambda layer.
- Internal extensions reside inside your Lambda function, and can listen to
INVOKEevents. To leverage an internal extension, you usually have to either modify the code of your function, or opt-in via configuration by adding, for example, a package to your Node.js function using the
NODE_OPTIONSenvironment variable or launching a
- Wrapper scripts are a way of changing the startup parameters of the code running your Lambda function by controlling the initialization of the runtime. Similarly to internal extensions, they are opt-in, and you need to activate them via the AWS_LAMBDA_EXEC_WRAPPER environment variable.
From a technology perspective, Lambda Extensions are pretty cool and provide plug-and-play value in use-cases in which a companion process provides value. We do have reservations about the lack of sandboxing of external extensions, as those will compete with your function for memory, and you will have to pretty much rely on the vendor not to cause Out of Memory errors. But all in all, the bar of configuring tracing with Instana is already so low that none of the Lambda extension facilities listed above can improve it.
Why Instana does not need a Lambda extension
When AWS reached out to us showing the Lambda extension, we looked at it with interest. We are always on the lookout for new, easier ways to use and roll out Instana. But, for AWS Lambda, we have already created and delivered innovative ways to make it super simple:
- No code changes are required, at all
- The instrumentation is delivered over a Lambda layer, the same way it would be via an Extension
- All you need to set up is configuration, which Instana handily provides in the dashboard:
In the image above you see a screenshot of the Instana Agent Onboarding screen, providing you at a glance, with copy & paste simplicity, all you need to do to trace your Lambda functions via Instana, including how to configure end-to-end a Lambda function via the aws command line tool.
We thought long and hard how we could leverage the Lambda extension to further improve this already streamlined user experience and, frankly, we came up empty.
If we could either activate an internal extension or a wrapper script automatically by virtue of you including the Instana Layer in your Lambda function, we could automatically trigger our function wrappers. That would allow you to avoid two arguments of the aws command shown in the screenshot above, or two steps in the manual setup using the AWS console. We’d really love to be able to do that, as our goal, from the very beginning of working on our Lambda support, has been “Add the layer, tell where to find Instana, and that’s it!”.
Alas, this is not something that Lambda extensions supports yet, so there is currently no value for Instana to reap from a Lambda extension.
Lambda extensions are a new facility in AWS to streamline how 3rd-party providers can deliver tracing and other types of instrumentation into Lambda functions. However, Instana’s methodology, designed with AWS Lambda in mind, is already so simple that Lambda extensions provide no extra capabilities or functionality to further simplify installation / configuration.
Of course we remain on the lookout for ways of making Instana Lambda tracing better and even simpler to install and run. We look forward to continuing to monitor (see what I did there?) how Lambda extensions develop. If we find a “killer application” for Instana + Lambda extensions, you can be sure we will build it quickly, of course.
Rumor has it that Michele, our Product Management for all things tracing, had gotten rather obsessed for a while trying to find ways of achieving the injection of environment variables from the Lambda layer, much to the neverending joy of our engineers (above all, Bastian Krol). Michele is known to occasionally still fall down that particular rabbit hole, which made overall Lambda Extensions a rather cruel thing to show him, as Lambda extensions make easier solving problems we had already solved.