Home
Setup & Manage Instana
Supported Technologies
Using Instana
Core Capabilities
Integrations, SDKs & APIs
Web REST API
Agent REST API
DBmarlin
Grafana
Prometheus
Tracing
Logging
Alerting
Pipeline Feedback Integrations
Release Notes
Policies

Host Agent REST API

TABLE OF CONTENTS

Host Agent REST API

The Instana Agent provides a web service API which accepts custom event and trace data.

Event SDK web service

Endpoint: http://<agent_ip>:42699/com.instana.plugin.generic.event

Using the Event SDK REST Web Service, it is possible to integrate custom health checks and other event sources into Instana. Each one running the Instana Agent can be used to feed in manual events. The agent has an endpoint which listens on http://<agent_ip>:42699/com.instana.plugin.generic.event and accepts for example the following JSON via a POST request:

{
  "title": <string>,
  "text": <string>,
  "severity": <integer>,
  "timestamp": <integer>,
  "duration": <integer>,
}

The following event parameters are available:

  • title: The title shown in the Instana Events view.
  • text: The description shown in the event's details section.

  • severity: An optional integer with one of the following values depending on the intended event type:

    • Change event: -1 (default)
    • Warning event: 5
    • Critical event: 10

    Furthermore, a change event is considered as a presence event if the title field contains the keyword "online" or "offline".

  • timestamp: The timestamp of the event in milliseconds since the UNIX epoch. If not present, the current time is used instead.
  • duration: Optional duration of the event in milliseconds. If missing the event will be shown as an "instant" event with no duration. Furthermore, please note that change/presence events are always closed by default, even when the duration is non-zero.
  • incident: Optional boolean that turns it into an Incident triggering event if set to true. Please note that this requires severity to be positive.
  • path: Optional identifier that can be defined in case an event might need to be extended with a later request to this endpoint using the same path value.

The endpoint also accepts a batch of events when given an array:

[
  {
    "title": "First event",
    ...
  },
  {
    "title": "Second event",
    ...
  }
]

Ruby example

duration = (Time.now.to_f * 1000).floor - deploy_start_time_in_ms
payload = {}
payload[:title] = 'Deployed MyApp'
payload[:text] = 'pglombardo deployed [email protected]'
payload[:duration] = duration

uri = URI('http://localhost:42699/com.instana.plugin.generic.event')
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.body = payload.to_json
Net::HTTP.start(uri.hostname, uri.port) do |http|
  http.request(req)
end

Curl example

curl -XPOST http://localhost:42699/com.instana.plugin.generic.event -H "Content-Type: application/json" -d '{"title":"Custom API Events ", "text": "Failure Redeploying Service Duration", "duration": 60000, "severity": 5}'

PowerShell example

For PowerShell you can either use the standard Cmdlets Invoke-WebRequest or Invoke-RestMethod. The parameters to be provided are basically the same.

Invoke-RestMethod
	-Uri http://localhost:42699/com.instana.plugin.generic.event
	-Method POST
	-Body '{"title":"PowerShell Event ", "text": "You used PowerShell to create this event!", "severity": -1}'
Invoke-WebRequest
  -Uri http://localhost:42699/com.instana.plugin.generic.event
  -Method Post
  -Body '{"title":"PowerShell Event ", "text": "You used PowerShell to create this event!", "severity": -1}'

Mina integration

Learn more about our Mina event integration.

Trace SDK Web service

Endpoint: http://<agent_ip>:42699/com.instana.plugin.generic.trace

Using the Trace SDK REST Web Service, it is possible to integrate Instana into any application regardless of language. Each active Instana Agent can be used to feed manually captured traces into the Web Service, which can be joined with automatically captured traces or be completely separate. The Agent offers an endpoint which listens on the http://<agent_ip>:42699/com.instana.plugin.generic.trace URL and accepts the following JSON via a POST request:

{
  "spanId": <string>,
  "parentId": <string>,
  "traceId": <string>,
  "timestamp": <64 bit long>,
  "duration": <64 bit long>,
  "name": <string>,
  "type": <string>,
  "error": <boolean>,
  "data": {
    <string> : <string>
  }
}

spanId is the unique identifier for any particular span. The trace is defined by a root span, that is, a span that does not have a parentId. The traceId needs to be identical for all spans that belong to the same trace, and is allowed to be overlapping with a spanId. traceId, spanId and parentId are 64 bit unique values encoded as hex string like b0789916ff8f319f. Spans form a hierarchy by referencing the spanId of the parent as parentId. An example of a span hierarchy in a trace is shown below:

	root (spanId=1, traceId=1, type=Entry)
	  child A (spanId=2, traceId=1, parentId=1, type=Exit)
	    child A (spanId=3, traceId=1, parentId=2, type=Entry)
	      child B (spanId=4, traceId=1, parentId=3, type=Exit)
	child B (spanId=5, traceId=1, parentId=4, type=Entry)

The timestamp and duration fields are in milliseconds. The timestamp must be the epoch timestamp coordinated to UTC. 

The name field can be any string which is used to visualize and group traces, and can contain any text. However, simplicity is recommended.

The type field is optional, when absent is treated as ENTRY. Options are ENTRY, EXIT, INTERMEDIATE, or EUM. Setting the type is important for the UI. It is assumed that after an ENTRY, child spans are INTERMEDIATE or EXIT. After an EXIT an ENTRY should follow. This is visualized as a remote call. 

The data field is optional and can contain arbitrary key-value pairs. The behavior of supplying duplicate keys is unspecified. 

The error field is optional and can be set to true to indicate an erroneous span.

The endpoint also accepts a batch of spans, which then need to be given as an array:

[
  {
    // span as above
  },
  {
    // span as above
  }
]

For traces received via the Trace SDK Web Service the same rules regarding Conversion and Service/Endpoint Naming are applied as for the Java Trace SDK. In particular these key-value pairs in data are used for naming: service, endpoint and call.name.

Note: The optional Instana Agent Service provided on Kubernetes via the Instana Agent Helm Chart is very useful in combination with the Trace Web SDK support, as it ensures that the data is pushed to the Instana Agent running on the same Kubernetes node, ensuring the Instana Agent can correctly fill in the infrastructure correlation data.

Curl example

The following example shows how to send to the host agent data about a matching ENTRY and EXIT call, which simulates a process that receives an HTTP GET request targeted to the https://orders.happyshop.com/my/service/asdasd URL and routes it to an upstream service at the https://crm.internal/orders/asdasd URL.

#!/bin/bash

curl -0 -v -X POST 'http://localhost:42699/com.instana.plugin.generic.trace' -H 'Content-Type: application/json; charset=utf-8' -d @- <<EOF
[
  {
    "spanId": "8165b19a37094800",
    "traceId": "1368e0592a91fe00",
    "timestamp": 1591346182000,
    "duration": 134,
    "name": "GET /my/service/asdasd",
    "type": "ENTRY",
    "error": false,
    "data": {
      "http.url": "https://orders.happyshop.com/my/service/asdasd",
      "http.method": "GET",
      "http.status_code": 200,
      "http.path": "/my/service/asdasd",
      "http.host": "orders.happyshop.com"
    }
  },
  {
    "spanId": "7ddf6b31b320cc00",
    "parentId": "8165b19a37094800",
    "traceId": "1368e0592a91fe00",
    "timestamp": 1591346182010,
    "duration": 97,
    "name": "GET /orders/asdasd",
    "type": "EXIT",
    "error": false,
    "data": {
      "http.url": "https://crm.internal/orders/asdasd",
      "http.method": "GET",
      "http.status_code": 200,
      "http.path": "/orders/asdasd",
      "http.host": "crm.internal"
    }
  }
]
EOF

Infrastructure correlation

Starting with host agent version 1.1.582, Trace Web SDK data received by the host agent are correlated with the process sending them, provided that:

  1. the host agent is running on a Linux system with the following commands available: lsns, nsenter and ss;
  2. the process sending the tracing data is running on the same host as the host agent;
  3. the process sending the tracing data is not ignored using the Ignore Processes capability;
  4. report traces directly to the host agent, rather than running over a proxy; if the tracing data go over a proxy, the proxy will be associated with the traces instead.

The Application Perspective Service dashboard will correctly reference any process from which related traces were ingested, including for example infra changes that took place for this service.

If any of the preconditions above is not fulfilled, the tracing data will be corrwelated with the host on which the host agent is running.

Limitations

Adhere to the following rate limits for the trace web service:

  • Maximum API calls/sec: 20
  • Maximum payload per POST request: A span must not exceed 4 KiB. The request size must not exceed 4 MiB.
  • Maximum batch size (spans/array): 1000

FAQ

Is there a limit for the amount of calls from Agent to Backend? Data transmission between Instana agent and Instana backend depends on a lot of factors. It is done using a persistent HTTP2 connection and highly optimized.

What is the optimal package size (the split size) to send 50,000 spans with a size about 40 MiB over the Agent to Backend? Recommended strategy is to buffer spans for up to one second or until 500 spans were collected, then transmit the spans to the agent. For an implementation of this transmission strategy, see our repo.

Which requirements are recommended for best Agent performance (cpu, ram, etc.)? This depends on the host that should be monitored, e.g. the number of Docker containers per host etc.

How should the agent environment be configured? See our agent configuration docs.