U is for usability, V is for visualization, W is for workflow, and finally, X is for xerox. Welcome to the sixth installment in a series of posts alphabetically listing and defining important Observability terms. If you have just jumped in, then please consider starting from the beginning to get a better understanding of my selection.
While we can expect machine learning and predictive machines to take on even more of the number crunching, it will be imperative that the interface between humans and machines is streamlined and maximized with efficient and effective communication and learning in mind. At the very heart of usability lies the ease of which one can use and learn of and from a tool. Usability is not achieved by merely layering an intuitive interface on top of data – sources, streams, and sinks. I admit there is some satisfaction to be gained in making interfaces pleasant and pretty, but this is, in general, transient and higher in the initial stages of adoption and use. Longer lasting enjoyment, growth, and productivity are gained in bringing newer higher-order models of system and process introspection, interfacing, and intervention that more immeasurable reflect capacities of communication and cognition while aligning and adapting to context, confidence, concern, and control. We grow and change minds with better models.
Humans are so highly dependent on the vision of the physical world, and now too the virtual and simulated worlds. All sensory data gathered is transformed into representations, symbols, and signals deep within our consciousness. Our minds mentalize information into models that usually take the form of conceptual imagery and visual metaphors. An Observability solution must play to this human strength to most effectively and efficiently convey information, in particular the significance of a past event or an unfolding situation. The primary goal of information visualization is to bring a comprehensive understanding to even the most complex of systems (of services) in the context of change and control (steering) – for the eyes to see and the mind to attentively perceive patterns of behaviors and states of affairs across vast space and time observational spaces. The manifestation of good design in communication, understanding, and learning is never more apparent than in the area of visualization. Today’s Observability tooling is still far too rudimentary in the display of data dumped on custom dashboards; tomorrow’s tooling will need to refocus on knowledge and wisdom on the motion and dynamics of systems and services as a result of accelerating change everywhere and anywhere. We should not be adhocly inspecting or pivoting data but instead seeing motion, and the information it conveys, of processes at various scales and adjusting frames of contexts. Our unhealthy over pre-occupation with data and its distribution as opposed to more essential signals, states, situations, structures, and systems must come to an end for there to be continued progress in the future.
Alternatives: Valve, Virtualization, View
Tracing, whether distributed or local, is a poor man’s attempt at capturing and modeling what effectively is organizational workflow – orchestrated and repeatable patterns of activity, directed by the systematic organization and management of resources into transformative processes of value creation. The nature of work, for both humans and machines, has changed significantly over the last few years. Yet, we still employ old, and now very dated, techniques of observation and measurement to glean from highly technical low-level forms of data flow the business enclosing intent, objective, and workflow context. Even at the technical level, our techniques for runtime introspection is, for the most part, ignorant of resilience mechanisms that have a significant bearing on the assessment of service quality. The overriding mindset of closure and sequential sequencing of requests-response closures does not reflect the reality of work that now spans many loosely coupled, in time and space, systems of services. It is now apparent that tracing, as defined today, is insufficient for the proper representation and understanding of organizational workflow processes that potentially consume multiple different and diverse resources – technical and social. Time is just one dimension, one resource, in a highly interconnected and interdependent system of processes, flows, activities, and tasks that needs to be monitored and managed. Instead of spans, those instrumenting software should be concerned with schedules, stages, states, signals, and synchs.
Alternatives: Workload, Web, Waits
On one of the two predicted timelines for Observability, the end game is the near-complete reconstruction of an episodic memory of past software execution and system dynamics. Today it is common to replicate data across machine boundaries, but what of execution behavior? While remote procedure call (RPC) middleware has allowed us to move performance across process and machine boundaries, at a very coarse granularity, these calls do not necessarily represent the replication of software behavior, but merely a form of service delegation. The mirroring I am referring to here is the simulated playback, online or offline, of software’s execution behavior in which a thread performs a local function or procedure call that is near-simultaneously mirrored in one or more “paired” runtimes. Within such runtimes, a paired thread is created, if not already done so, to mirror the real application thread and in tandem, when online, perform pushing and popping of stack frames as they occur within the application. Future Observability will record and playback, online or offline, the observable behavior and state changes of systems and services, much like a copy of the machine universe but within a single observable space used introspect work, flow, consumption, cost, and state safely. The past, present, and future of machine memory will be simulated, eventually.