This is the fourth post in a series of posts alphabetically listing and defining important Observability terms. If you have just jumped in then please consider starting from the beginning to get a better understanding of my selection.
Everything in life is memory, save for the present – a fleeting moment. Without memory, there can be no recognition, identification, and change detection. For the human mind, the events remembered have importance. Observability tooling should operate likewise, at the very least, in terms of the organization, recollection, and display of memories consisting of low-level data such as traces, metrics, events, logs, as well as high-value information such as signals and states. Meaningful organization of memory leads to enhanced performance in searching through data and, more importantly, in the suggestion of what to attend too. Memory is one of the three essential components in human and machine cognition, the other two being attention and perception. Usually, we think of memory as being retrospective, looking behind. Still, modern application monitoring tooling must take on a prospective perspective, such as when a new software release is deployed, and there is an expectation of future events reflecting system state transitions and eventually semi-permanent behavioral change. Humans and machines need to project their cognition back and forth along a timeline with ease. Today memory pertains principally to sensory data collected; in the future, there will be a shift to the memory of the human-and-machine interactions, communications, and learnings. Without memory, there can be no adapting, planning, or predicting.
Alternatives: Monitoring, Management, Metrics, Microservices, Model, Metering, Monitor
Much of the Observability data collected today is not curated with regard to its significance leading to dashboards being over polluted with data noise. Less is more when it comes to active monitoring and management of systems. Being effective requires a focus on meaningful signals that indicate, at least reliably infer, a change in behavior and state. Unfortunately, extracting signals from a vast sea of unqualified unclassified data is a near-impossible task that will invariably overwhelm most engineering teams. Machine learning is not a magic wand here that will clear away the fog created by noisy data in the form of metrics, traces, and logs. This is further complicated by growing complexity and, more troubling, continuous change. Today most tooling does a poor job of distinguishing a change in a measure because of dynamics or environmental factors as opposed to an actual change in the underlying code or configuration of a component. Intelligent contextual filtering, augmentation, and classification, as opposed to (simple) sampling, is required at all stages in an Observability pipeline along with the synthesis of higher-order measures and extraction of predictive signals and inference of states. The model is not the data, change is.
Alternatives: Network, Naming, Node, Navigation
A significant factor in the success of an Observability initiative at scale comes down to organizing. Organizations need to arrange and structure not just the machine memories but the flow of communication and control across systems, services, and organizational team structures. Organizing leads to greater efficiency and simplicity in operational work, reflecting existing service and social structures, processes, functions, forms, and groupings. Observability solutions need to organize displays and interactions around common patterns of perception and processing, including temporal, sequential, spatial, similarity, cause-and-effect, depth, topics, tags, and markers. Information can shape an organization but at the same time, it must be organized to do so more effectively.
Alternatives: Orientation, Optimization, Organization, Outage, OpenCensus, OpenTracing, OpenTelemetry, OSS
An essential tool in life is perspective – being able to observe, understand, and reason about an event or situation from a different point of view or framing. Having the ability to observe systems from the many different and diverse perspectives is crucial in developing critical thinking skills within an organization – it is a far more vital and productive technique than high-dimensional data or deep systems exploration. Multiple perspective-taking is an integral component of conflict resolution and dramatically improves problem-solving, especially in complex and changing systems with much uncertainty – it builds on the ability to integrate multiple channels and types of sensory data from different sources. Instead of scanning through datasets, switch between perspectives.
Alternatives: Profiling, Performance, Probe, Profiles, Path, Problem, Prediction, Prognosis, Pipeline, Pod, Platform, Playback