How to Monitor Google Cloud Pub/Sub

How to Monitor Google Cloud Pub/Sub

Google Cloud Pub/Sub is a messaging middleware cloud service that allows collaboration of various loosely coupled applications and services by providing message exchange among them. According to the Google Cloud site “You can use Pub/Sub as messaging-oriented middleware or event ingestion and delivery for streaming analytics pipelines. Pub/Sub offers durable message storage and real-time message delivery with high availability and consistent performance at scale. Pub/Sub servers run in all Google Cloud regions around the world.”

Word Image 345
^Pub/Sub use cases example

Using Google Cloud Pub/Sub

The Google Cloud Pub/Sub service is based on the publisher-subscriber paradigm and it is a central point for the data exchange between different applications and services. Publisher is an application responsible for creating a topic and sending messages to the topic while Subscriber applications consume these messages. In order to receive messages from a topic, Subscriber has to create a Subscription to the topic. The message is retained in the service until each subscriber subscribed to the topic receives the message and sends an acknowledgment. The message contains a payload and it can optionally contain additional meta-data in the form of attributes. Subscriber receives messages for service via pull and push mechanisms. Messages are sent and received asynchronously which guarantees that each message will be consumed and processed even in the event a subscriber is offline for some time.

According to Google, some of the common use cases for Pub/Sub service are:

  • Balancing workloads in network clusters.
  • Implementing asynchronous workflows
  • Distributing event notifications
  • Logging to multiple systems

Monitoring Google Cloud Pub/Sub with Instana

Instana has created a new sensor to monitor Google Cloud Pub/Sub. By installing a lightweight Instana agent on Google Compute Engine and providing credentials, the sensor is automatically activated and starts collecting metrics for Pub/Sub. Instana’s Pub/Sub sensor provides a comprehensive set of metrics, KPIs, distributed tracing, and built-in and custom events.

Pub/Sub services can contain labels and, although not required to monitor Pub/Sub, you can define which entities you want to include and which you want to exclude from your monitoring based on those labels (tags in Instana). To define what to monitor, simply fill in the include and exclude list in the configuration file. Additionally, you can set the level of granularity based on topic and subscription level for Pub/Sub down to 1 minute.

In terms of distributed tracing, Instana provides comprehensive support for the Pub/Sub SDKs that Google has published for a variety of languages, including but not limited to Java, .NET Core, Node.js, and Go. Instana traces every single request, message sent to Pub/Sub or message consumed from Pub/Sub, whether by polling Pub/Sub or by consuming HTTP-based push notifications. Instana also added out-of-the-box support for Spring Cloud Google Pub/Sub to provide maximum ease of setup.

Through its no-sampling, real-time distributed tracing, Instana provides the means to observe your application performance, enabling you to ensure that each message reaches its destination and that the message is delivered on time. If latency suddenly increases, Instana will send an alert indicating that a service is slowing down and/or some messages won’t be delivered on time. Additionally, Instana will raise an event or alert if the oldest message on the topic is older than the threshold value.

Word Image 346
^Google Cloud Pub/Sub main dashboard

The main Pub/sub dashboard contains important KPIs, metrics, and list of topics and subscriptions. By clicking on the topic or subscription in the list you can navigate to the dashboard for that particular entity. Giving you immediate access to the information needed to troubleshoot any issue.

Word Image 347
^Google Cloud Pub/Sub tracing

Instana traces every request flowing through GCE ensuring that you always have a complete distributed trace to get to the root cause of any issue. Instana tracks all dependencies, service, and infrastructure, by creating a logical model referred to as the Dynamic Graph. This model contains all of the contextual information needed to automatically determine the root cause of problems in applications that use Pub/Sub. An example of a distributed trace is shown in the screenshot above.

Instana’s Google Cloud Pub/Sub monitoring includes automatic discovery, metric monitoring, dependency mapping, anomaly detection, and analytics across the complete trace data set. Combined together, these features give you a holistic insight into the state of your Google services.

If you want to try Google Cloud Pub/Sub monitoring and experience its full power, you can sign up for a free trial of Instana today.

Play with Instana’s APM Observability Sandbox

Developer, Product, Thought Leadership
In my last blog post, I talked about the history and rapid adoption of the OpenTelemetry project. Today we discuss the progress made in terms of end-to-end compatibility between distributed tracing with...
|
Developer, Thought Leadership
In the past few years, distributed tracing has emerged in the global DevOps consciousness as an indispensable tool in the microservices arsenal. In April 2019, the open source observability community rose to...
|
Announcement, Product
This blog post was originally published by IBM What is IBM Observability by Instana? IBM Observability by Instana provides businesses with advanced application performance monitoring and observability capabilities, manages the performance of...
|

Start your FREE TRIAL today!

Instana, an IBM company, provides an Enterprise Observability Platform with automated application monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside – on-premises or in public and private clouds, including mobile devices or IBM Z.

Control hybrid modern applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also gives visibility into development pipelines to help enable closed-loop DevOps automation.

This provides actionable feedback needed for clients as they to optimize application performance, enable innovation and mitigate risk, helping Dev+Ops add value and efficiency to software delivery pipelines while meeting their service and business level objectives.

For further information, please visit instana.com.