Google Cloud Pub/Sub is a messaging middleware cloud service that allows collaboration of various loosely coupled applications and services by providing message exchange among them. According to the Google Cloud site “You can use Pub/Sub as messaging-oriented middleware or event ingestion and delivery for streaming analytics pipelines. Pub/Sub offers durable message storage and real-time message delivery with high availability and consistent performance at scale. Pub/Sub servers run in all Google Cloud regions around the world.”
^Pub/Sub use cases example
Using Google Cloud Pub/Sub
The Google Cloud Pub/Sub service is based on the publisher-subscriber paradigm and it is a central point for the data exchange between different applications and services. Publisher is an application responsible for creating a topic and sending messages to the topic while Subscriber applications consume these messages. In order to receive messages from a topic, Subscriber has to create a Subscription to the topic. The message is retained in the service until each subscriber subscribed to the topic receives the message and sends an acknowledgment. The message contains a payload and it can optionally contain additional meta-data in the form of attributes. Subscriber receives messages for service via pull and push mechanisms. Messages are sent and received asynchronously which guarantees that each message will be consumed and processed even in the event a subscriber is offline for some time.
According to Google, some of the common use cases for Pub/Sub service are:
- Balancing workloads in network clusters.
- Implementing asynchronous workflows
- Distributing event notifications
- Logging to multiple systems
Monitoring Google Cloud Pub/Sub with Instana
Instana has created a new sensor to monitor Google Cloud Pub/Sub. By installing a lightweight Instana agent on Google Compute Engine and providing credentials, the sensor is automatically activated and starts collecting metrics for Pub/Sub. Instana’s Pub/Sub sensor provides a comprehensive set of metrics, KPIs, distributed tracing, and built-in and custom events.
Pub/Sub services can contain labels and, although not required to monitor Pub/Sub, you can define which entities you want to include and which you want to exclude from your monitoring based on those labels (tags in Instana). To define what to monitor, simply fill in the include and exclude list in the configuration file. Additionally, you can set the level of granularity based on topic and subscription level for Pub/Sub down to 1 minute.
In terms of distributed tracing, Instana provides comprehensive support for the Pub/Sub SDKs that Google has published for a variety of languages, including but not limited to Java, .NET Core, Node.js, and Go. Instana traces every single request, message sent to Pub/Sub or message consumed from Pub/Sub, whether by polling Pub/Sub or by consuming HTTP-based push notifications. Instana also added out-of-the-box support for Spring Cloud Google Pub/Sub to provide maximum ease of setup.
Through its no-sampling, real-time distributed tracing, Instana provides the means to observe your application performance, enabling you to ensure that each message reaches its destination and that the message is delivered on time. If latency suddenly increases, Instana will send an alert indicating that a service is slowing down and/or some messages won’t be delivered on time. Additionally, Instana will raise an event or alert if the oldest message on the topic is older than the threshold value.
^Google Cloud Pub/Sub main dashboard
The main Pub/sub dashboard contains important KPIs, metrics, and list of topics and subscriptions. By clicking on the topic or subscription in the list you can navigate to the dashboard for that particular entity. Giving you immediate access to the information needed to troubleshoot any issue.
^Google Cloud Pub/Sub tracing
Instana traces every request flowing through GCE ensuring that you always have a complete distributed trace to get to the root cause of any issue. Instana tracks all dependencies, service, and infrastructure, by creating a logical model referred to as the Dynamic Graph. This model contains all of the contextual information needed to automatically determine the root cause of problems in applications that use Pub/Sub. An example of a distributed trace is shown in the screenshot above.
Instana’s Google Cloud Pub/Sub monitoring includes automatic discovery, metric monitoring, dependency mapping, anomaly detection, and analytics across the complete trace data set. Combined together, these features give you a holistic insight into the state of your Google services.
If you want to try Google Cloud Pub/Sub monitoring and experience its full power, you can sign up for a free trial of Instana today.
