How to Monitor Google Cloud Pub/Sub

Post

Google Cloud Pub/Sub is a messaging middleware cloud service that allows collaboration of various loosely coupled applications and services by providing message exchange among them. According to the Google Cloud site “You can use Pub/Sub as messaging-oriented middleware or event ingestion and delivery for streaming analytics pipelines. Pub/Sub offers durable message storage and real-time message delivery with high availability and consistent performance at scale. Pub/Sub servers run in all Google Cloud regions around the world.”

Word Image 345
^Pub/Sub use cases example

Using Google Cloud Pub/Sub

The Google Cloud Pub/Sub service is based on the publisher-subscriber paradigm and it is a central point for the data exchange between different applications and services. Publisher is an application responsible for creating a topic and sending messages to the topic while Subscriber applications consume these messages. In order to receive messages from a topic, Subscriber has to create a Subscription to the topic. The message is retained in the service until each subscriber subscribed to the topic receives the message and sends an acknowledgment. The message contains a payload and it can optionally contain additional meta-data in the form of attributes. Subscriber receives messages for service via pull and push mechanisms. Messages are sent and received asynchronously which guarantees that each message will be consumed and processed even in the event a subscriber is offline for some time.

According to Google, some of the common use cases for Pub/Sub service are:

  • Balancing workloads in network clusters.
  • Implementing asynchronous workflows
  • Distributing event notifications
  • Logging to multiple systems

Monitoring Google Cloud Pub/Sub with Instana

Instana has created a new sensor to monitor Google Cloud Pub/Sub. By installing a lightweight Instana agent on Google Compute Engine and providing credentials, the sensor is automatically activated and starts collecting metrics for Pub/Sub. Instana’s Pub/Sub sensor provides a comprehensive set of metrics, KPIs, distributed tracing, and built-in and custom events.

Pub/Sub services can contain labels and, although not required to monitor Pub/Sub, you can define which entities you want to include and which you want to exclude from your monitoring based on those labels (tags in Instana). To define what to monitor, simply fill in the include and exclude list in the configuration file. Additionally, you can set the level of granularity based on topic and subscription level for Pub/Sub down to 1 minute.

In terms of distributed tracing, Instana provides comprehensive support for the Pub/Sub SDKs that Google has published for a variety of languages, including but not limited to Java, .NET Core, Node.js, and Go. Instana traces every single request, message sent to Pub/Sub or message consumed from Pub/Sub, whether by polling Pub/Sub or by consuming HTTP-based push notifications. Instana also added out-of-the-box support for Spring Cloud Google Pub/Sub to provide maximum ease of setup.

Through its no-sampling, real-time distributed tracing, Instana provides the means to observe your application performance, enabling you to ensure that each message reaches its destination and that the message is delivered on time. If latency suddenly increases, Instana will send an alert indicating that a service is slowing down and/or some messages won’t be delivered on time. Additionally, Instana will raise an event or alert if the oldest message on the topic is older than the threshold value.

Word Image 346
^Google Cloud Pub/Sub main dashboard

The main Pub/sub dashboard contains important KPIs, metrics, and list of topics and subscriptions. By clicking on the topic or subscription in the list you can navigate to the dashboard for that particular entity. Giving you immediate access to the information needed to troubleshoot any issue.

Word Image 347
^Google Cloud Pub/Sub tracing

Instana traces every request flowing through GCE ensuring that you always have a complete distributed trace to get to the root cause of any issue. Instana tracks all dependencies, service, and infrastructure, by creating a logical model referred to as the Dynamic Graph. This model contains all of the contextual information needed to automatically determine the root cause of problems in applications that use Pub/Sub. An example of a distributed trace is shown in the screenshot above.

Instana’s Google Cloud Pub/Sub monitoring includes automatic discovery, metric monitoring, dependency mapping, anomaly detection, and analytics across the complete trace data set. Combined together, these features give you a holistic insight into the state of your Google services.

If you want to try Google Cloud Pub/Sub monitoring and experience its full power, you can sign up for a free trial of Instana today.

Play with Instana’s APM Observability Sandbox

Announcement, Developer, Product, Thought Leadership
At Instana, we recently improved the installation process for our self-hosted customers. Instana’s self-hosted platform now utilizes a fully Docker based installation process. In a previous blog post, Lessons Learned From Dockerizing...
|
Featured
An ever-increasing number of System architectures and deployment strategies depend on Kubernetes-based environments. Kubernetes (also known as k8s) is an orchestration platform and abstract layer for containerized applications and services. As such,...
|
Announcement, Developer, Product, Thought Leadership
To be successful in Observability, you must have the ability to observe the behavior of a system and derive its health state from it. But deriving the health state of any given...
|

Start your FREE TRIAL today!

As the leading provider of Automatic Application Performance Monitoring (APM) solutions for microservices, Instana has developed the automatic monitoring and AI-based analysis DevOps needs to manage the performance of modern applications. Instana is the only APM solution that automatically discovers, maps and visualizes microservice applications without continuous additional engineering. Customers using Instana achieve operational excellence and deliver better software faster. Visit https://www.instana.com to learn more.