It is sometimes easy to lose sight of the real challenges the application monitoring industry is facing when discussing particular observability techniques, technologies, and tooling. Here I take a step back and list some of the major concerns that we as a solution provider consider.
The system of services we are building and running today are increasingly considered as Complex Systems. Such systems are composed of a large number of simple parts. There are a significant amount of interconnections and interactions between parts and at different space and time scales. Whole sections of systems are becoming adaptive, in responding to changes in the environment as well as the system itself, so as to optimize and regulate functioning. With change resulting in more change, we must deal with dynamic states of availability and health.
A key driver for much of what we see today in the industry concerning platforms, frameworks, architectures, and services is Change. When companies talk about agility, they are talking about change. They are effectively readying a company to be far more responsive to internally and externally generated change – small and more frequently occurring. When they talk about DevOps, they are talking about managing the process of change to systems that support organizational and business process change. The software industry has gone from the deployment of new and enhanced applications and services every few months to weeks and days, with the target being continuous and for the most part automatic. There is a growing acceptance that with change comes risk and possible failure but smaller in scope and duration.
A critical requirement for operational staff responsible for complex and ever-changing systems is situational awareness – being able to assess, reconstruct, and understand past and present contexts exceptionally quickly. Context is the circumstances that form the setting for an event. Events shape the context, but they are not the context alone. A context includes the environment of a system and the set of system identified entities existing during a period in time. The context also includes named flows of control and communication and the many user-defined boundaries they cross – such as Instana’s powerful Application Perspective groupings.
Why do we observe our environment and the entities that exist within it? For animals, man, and now machines, that would be to monitor for signals. Signals, in particular, repeating patterns of signals, allows us to infer the state of our environment and the various entities present. We generally monitor signals to indirectly control states via (re)actions and adaptions to the recognition of patterns that, in the past, lead to specific outcomes of concern. Here is where application monitoring vendors like Instana look to augment the intelligence of individuals and teams so that they do not collapse under the stress caused by ever-increasing complexity and rapidly changing contexts. Companies need help regulating change via control feedback loops.
Instant and dynamic connecting of events and contextualizing of an environment and its entities is a crucial requirement for effective monitoring and management of modern applications and services. The speed of awareness of and reaction to change, direct or indirect, intended or inadvertent, is paramount to the success of today’s modern systems – faster and better.