Applying Pipeline Feedback Scope to Uncover Hidden Dependencies

Increased Agility with Pipeline Feedback Scoping

In a previous blog, Increasing Agility with Pipeline Feedback Scoping, we discussed the different types of pipeline feedback scope. In this follow up post, we’ll look at how to best apply pipeline feedback scope. To keep things simple and consistent, the same monitoring example from the first blog is used for illustration.

Cross Cutting Services

In most modern applications, services cut across multiple applications, creating cross-cutting, unexpected dependencies. To ensure full visibility, Instana allows for the service and application markers to be combined for these cross cutting services. This is shown in the figure below where the Shopping application has been expanded (i.e., the dashed box is now bigger) to include the cross cutting services and, unsurprisingly, is now called the ExpandedShopping application. There are three release markers shown:

  • SM #11 is the service marker assigned to the User and History services;
  • SM #12 is the service marker assigned to the warehouse service; and
  • Application PerspectiveM #10 is the application marker assigned to the entire ExpandedShopping application.

The six related dashboards would show the following markers (endpoint dashboards are hidden for clarity):

  • The Discount and Inventory service are only part of the application so they would show the Application PerspectiveM #10 marker only when the ExpandedShopping application context is there.
  • The User and History service show the Application PerspectiveM #10 application marker because they are part of the application. They also show the service marker SM #11 because that was assigned to those services.
  • Similarly the Warehouse service shows Application PerspectiveM #10 and SM #12.
  • Finally, the ExpandedShopping application dashboard shows the following three markers Application PerspectiveM #10, SM #11, and SM #12. It rolls-up the service markers for those services it is part of, as well as its own Application Perspective markers.


Word Image 353

The CURL commands to create these markers is shown below.

curl --location --request POST https://<YOUR_INSTANA_HOST>/api/releases \
--header 'Content-Type: application/json' \
--header 'Authorization: apiToken <YOUR_Application API_TOKEN>' \
--data "{
  \"start\": $(date +%s000),
  \"name\": \"application-marker-#10\",
  \"applications\": [
    {
      \"name\": \"ExpandedShopping application\"
    }
  ]
}"
curl --location --request POST https://<YOUR_INSTANA_HOST>/api/releases \
--header 'Content-Type: application/json' \
--header 'Authorization: apiToken <YOUR_Application API_TOKEN>' \
--data "{
  \"start\": $(date +%s000),
  \"name\": \"service-marker-#11\",
  \"services\": [
    {
      \"name\": \"user-service\"
    },
    {
      \"name\": \"history-service\"
    }
  ]
}"
curl --location --request POST https://<YOUR_INSTANA_HOST>/api/releases \
--header 'Content-Type: application/json' \
--header 'Authorization: apiToken <YOUR_Application API_TOKEN>' \
--data "{
  \"start\": $(date +%s000),
  \"name\": \"service-marker-#12\",
  \"services\": [
    {
      \"name\": \"warehouse-service\"
    }
  ]
}"

A Hotfix for a Service

It is not uncommon to find that a service needs a hot fix in production for an important customer. When this happens, the entire application does not always, or even usually, need to be redeployed. In these scenarios, it is important to communicate these one off situations where the service may have a different marker than the application.

Cross Cutting Hotfix

The simplest hotfix is when it is a cross cutting service. This can be handled just like a normal service release by assigning a new service marker for the hotfix.

Service Hotfix in an Application Perspective

A hot fix for a service can be recorded globally using a service marker but it is also possible to scope a hotfix using an application marker. The figure below shows this situation where the Inventory service needs a hot fix but this is only relevant for the Shopping application so a service marker or global marker is not warranted. In this situation, a service marker is assigned but it is within the application context only and not globally. SM #8 is such a service marker that is assigned in the shopping application for the Inventory service. Since it is scoped to the Shopping application that service marker is shown in the Shopping application dashboard. The Inventory dashboard would show both the Application PerspectiveM #5 and SM #8 markers if it was in the Shopping application context. The Inventory service would only show the Application PerspectiveM #7 marker with the ReturnItem application context.

Word Image 354

If there was no application context because the user went to the Inventory service dashboard directly, then no markers would be shown because the assigned service markers depend upon an application context to be visible. Similarly for the associated endpoint dashboards. This is shown below.

Word Image 355

The code example below shows how a service marker can be scoped to an application by adding the scopedTo attribute in the services section of the JSON structure.

curl --location --request POST https://<YOUR_INSTANA_HOST>/api/releases \
--header 'Content-Type: application/json' \
--header 'Authorization: apiToken <YOUR_Application API_TOKEN>' \
--data "{
  \"start\": $(date +%s000),
  \"name\": \"service-marker-#8\",
  \"services\": [
    {
      \"name\": \"inventory-service\",
      \"scopedTo\": {
    \"applications\": [
      {
        \"name\": \"Shopping application\"
      }
    ]
     }
   }
  ]
}"

Deployment Across Geography or Zones

A large enterprise can have multiple data centers or cloud regions. Sophisticated deployment mechanisms are used here: begin by deploying the application in one location, monitor it for a period of time to make sure it is working properly, and then deploy the application to another location, etc. In the event of a problem the application is rolled back to the prior working release. What is novel about these types of mechanisms is that there is a time gap between the similarly versioned services running at the different locations. The example below shows a strategy for using Pipeline Feedback scope for deploying across geographies or zones.

In this example the Shopping application is deployed first in the EU zone and then, later, in the US zone. The SRE team has three different Application Perspectives: one for the EU zone (using the zone=”EU” tag), one for the US zone (using the zone=”US” tag), and one that is the global application view for all the zones. As shown below, when the EU rollout finishes at 2:00, an application marker is set for the “Shopping EU application” which includes the services in the EU zone. Later, around 10:00, the application is deployed to the US zone and an application release marker for that Application Perspective is set. Then another application marker is set on the “Shopping global application” when both the EU and US zones are running the same service version. When all data centers are running well, then the global application can be used as the central monitoring dashboard.

Word Image 356

This strategy allows each zone to be monitored independently of the other zone, helping to identify if there is an issue specific to a zone. If there is a problem during any stage of the rollout at a particular zone then that Application Perspective can be used to triage the problem within that zone. If there is a hotfix to be deployed then it can be specific to that zone by scoping the hot fixed updated service to the appropriate Application Perspective.

Word Image 357

Communicating Software or Platform Version Changes

Whenever a technology, database, software or hardware upgrade has occurred it is a challenge to communicate this to everyone who is interested. An upgrade of this type can be viewed as a release change, at some level, and so the pipeline feedback toolbox can be useful for communicating these types of changes. Let’s consider these different situations.

A database is modelled in Instanta as a service so a service marker can be used to mark a database upgrade. The service marker name just needs to reflect the type of change and the marker’s timestamp set appropriately. A similar approach can be used for a message broker server upgrade, hardware upgrade, etc. since a message broker (e.g., Kafka) is also a service.

If such an upgrade is not of global interest then an Application Perspective marker may be more appropriate. For example, if an Application Perspective exists for a team of developers then an Application Perspective marker can identify major architectural changes as a reminder to the team that the prior measurements may not be applicable any longer.

In some cases, setting a service marker in an Application Perspective may be helpful. For example, if a database schema undergoes a significant change then setting a service marker in an Application Perspective would mark this change.

Summary

The first blog in the series reviewed the new types of scoping for Pipeline Feedback. This blog has shown several applications of Pipeline Feedback scope markers. The last blog in this series will explore the API for Pipeline Feedback scope which will complete the blog series.

Play with Instana’s APM Observability Sandbox

Developer, Thought Leadership
In my last blog on preparing for Black Friday, I teased how easy it is to gain insight into the state of your eCommerce retail applications and all its components with an...
|
Announcement
Instana, an IBM company, has been recognized as a Customers’ Choice in the 2021 Gartner Peer Insights ‘Voice of the Customer’: Application Performance Monitoring, and we are thrilled. Our team takes great...
|
Developer, Thought Leadership
In the past few years, distributed tracing has emerged in the global DevOps consciousness as an indispensable tool in the microservices arsenal. In April 2019, the open source observability community rose to...
|

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit instana.com.