Continuous Application Profiling in Production


Profiling is an essential part of application development, where optimizing performance and resource efficiency are important. It is also useful for troubleshooting performance and crash issues. Profilers provide details about code execution, which are otherwise not available through logging and code instrumentation. A typical profile provides a statistic, e.g. CPU usage or allocated memory, for every line of code highlighting the hot spots. This allows developers to directly work on optimizing the hot spots. Without such code-level statistics it would be extremely hard to locate hot code.

Profiling in Development Environment

Traditionally profiling was done on the developer’s machine. This makes sense for initial program optimization. Mostly, these optimizations are related to algorithm efficiency. More advanced uses include profiling applications with synthetic data in order to get closer to the production environment where the application will live after deployment. The problem with local profiling is that it can cover only a subset of use cases, which may arise in production. In case of performance issues or optimization requirements originating in production, the difference between environments may cause much longer MTTR due to inability of a developer to reproduce the issue locally. Another problem is that many applications are partially or fully developed in the cloud and profiling them locally is not even an option.

This doesn’t mean that profiling is not usable in modern applications. On the contrary, with the right methodology profiling can be used in highly scalable, on-prem and cloud applications. Those approaches are discussed next.

On-Demand Profiling in Production

Some profilers offer the possibility to attach to production processes for profiling. There are a few requirements to be satisfied before a profiler can attach.

  1. An application process should be locatable, which might not be easily achievable in container-based, serverless and/or autoscaled cloud environments.
  2. An application process must be accessible via specific port, which could be a technical as well as security challenge.
  3. If a profiler is not designed for product environments, e.g. has high overhead, it may have side effects and damage the already failing process it is trying to profile, which is ironic.
  4. Issues such as memory leaks require analyzing historical profiles to understand the trends in resource consumption by code. In this case periodic profile collection over some period of time is necessary, and if not automated, it is time consuming and error prone with on-demand profilers.

It is worth noting that there are on demand profilers that operate in push vs traditional pull mode. This makes them more suitable for production, but not fully.

Continuous Profiling in Production

Continuous profiling addresses the above mentioned challenges by being designed for production from the ground up. By continuously collecting profiles from all application processes, we now have historical and current profiles accessible anytime by anyone in the team.

Another advantage of continuous profiling is the fact that it is constantly activated throughout the lifecycle of the process, and the risk that it may cause additional problems compared to on-demand profiler attaching to the process first time is greatly minimized.

The implementations of continuous profilers differ for every language, however the concepts are the same; the profiler collects profiles automatically and periodically making sure the overhead stays unnoticeable. It also makes sure there is some randomness to avoid periodicity bias, which may happen for example when a profiling interval matches some internal application job.

Production profilers are not easy to implement due to very low overhead constraints and multiple deployment possibilities. That’s why, when delivered as an integrated tool inside of an automatic APM product, profiling gets its production super power.

Instana AutoProfile™

Instana AutoProfile™ takes continuous profiling to the next level by making it intuitively interoperate with its full APM suite. The profiles are one click away from the process dashboards and transaction traces, which may need a deep dive into code level in case of increased resource consumption.

Word Image 263

Last but not least, enabling AutoProfile in Instana is just one configuration option. See Instana’s profiling documentation for details on setting up continuous profiling. There are also tips on analyzing profiles.

Oh – AND not only does continuous profiling come with Instana’s APM solution – FOR FREE – it is also completely available to Instana Trial  users.

Play with Instana’s APM Observability Sandbox

There are multiple reasons why a program will consume more CPU resources than excepted. In the case of a high computational complexity of an algorithm, the amount of data it operates on...
Announcement, Developer, Product
PHP profiling is used by developers to identify performance issues or bottlenecks in their code. Profilers enable developers to drill into individual lines of code to ascertain which ones are running slow...
Announcement, Conceptual, Customer Stories
This guest blog post was written by Dean Record, Engineer at Goji Investments. Goji Investments launched in 2016. Our platform democratises access to real estate, business lending, renewables, and other alternative investments....

Start your FREE TRIAL today!

As the leading provider of Automatic Application Performance Monitoring (APM) solutions for microservices, Instana has developed the automatic monitoring and AI-based analysis DevOps needs to manage the performance of modern applications. Instana is the only APM solution that automatically discovers, maps and visualizes microservice applications without continuous additional engineering. Customers using Instana achieve operational excellence and deliver better software faster. Visit to learn more.