Elasticsearch Monitoring

Elasticsearch Monitoring and Management

Elasticsearch monitoring with Instana’s Application Monitoring solutionis a key part of delivering high performance containerized microservice applications. Elasticsearch is a distributed and scalable search engine, document store, and analytics platform based on Apache Lucene and built to to integrate with a log parsing engine and analytics visualization platform. It provides a full-text search engine with an HTTP interface for schema free JSON documents. Because of these capabilities, it is a highly popular enterprise platform, implemented in many potential systems to which Instana APM is applicable.

If there are any Elasticsearch instances running in an environment where Instana’s Application Monitoring or Infrastructure monitoring solution is deployed, the Instana agent will automatically detect the technology and deploy the Elasticsearch monitoring sensor to monitor the technology instances, gather configuration data, and monitor search calls, ensuring that every Elsaticsearch request interacts properly within the monitored application.

Elasticsearch Performance and Configuration Metrics

Recognizing that ElasticSearch is a search engine focused on enterprise systems and applications, Instana’s Elasticsearch monitoirng automatically collects the right metrics to monitor the integration between Elasticsearch and other microservices, as well as Elasticsearch performance (from the applications’ perspective) on both a node and cluster level. It’s worth noting that the Elasticsearch Node level monitoring collects all the metrics as the cluster level Elasticsearch monitoring, plus additional data.

Typical tracked metrics at the node level include:

Lucene Segments
Rejected Threads
Queued Threads

Typical tracked metrics at the cluster level include:

Search performance, throughput
Indices
Shards

See the Instana Elasticsearch Sensor documentation for a complete list.

Managing Elasticsearch Health

In the course of monitoring applications utilizing Elasticsearch, Instana can infer the health of the application through capturing the metrics above, as well as certain key performance indicators. As a search engine, the search and write times are the primary KPIs that must be monitored. Tracking the number of threads queued up by either the cluster or the node that get stalled or rejected is a major indicator of poor application health.

Some more basic key performance indicators at the cluster level are:

Cluster Health
Split Brain
Response Time

More Elasticsearch KPIs collected at the node level are:

Rebalancing
High Heap Usage
Node Capacity Limit

As with the performance and configuration metrics, please see the Instana’s Elasticsearch Sensor documentation for a complete list.

Elasticsearch Configuration Changes

Instana’s Dynamic Graph is the mapping of all dependencies within the application and serves as the data store for all configuration changes detected by the Elasticsearch sensor. The Dynamic Graph will then provide context to the changes and route them to the timeline in the bottom of the Instana UI. From there, users can explore specific issues, events, and changes within the context of their whole application.