Host Agent REST API
TABLE OF CONTENTS
Host Agent REST API
The Instana Agent provides a web service API which accepts custom event and trace data.
Event SDK web service
Endpoint: http://<agent_ip>:42699/com.instana.plugin.generic.event
Using the Event SDK REST Web Service, it is possible to integrate custom health checks and other event sources into Instana. Each one running the Instana Agent can be used to feed in manual events. The agent has an endpoint which listens on http://<agent_ip>:42699/com.instana.plugin.generic.event
and accepts for example the following JSON via a POST request:
{
"title": <string>,
"text": <string>,
"severity": <integer>,
"timestamp": <integer>,
"duration": <integer>,
}
The following event parameters are available:
- title: The title shown in the Instana Events view.
- text: The description shown in the event's details section.
-
severity: An optional integer with one of the following values depending on the intended event type:
- Change event:
-1
(default) - Warning event:
5
- Critical event:
10
Furthermore, a change event is considered as a presence event if the title field contains the keyword "online" or "offline".
- Change event:
- timestamp: The timestamp of the event in milliseconds since the UNIX epoch. If not present, the current time is used instead.
- duration: Optional duration of the event in milliseconds. If missing the event will be shown as an "instant" event with no duration. Furthermore, please note that change/presence events are always closed by default, even when the duration is non-zero.
- incident: Optional boolean that turns it into an Incident triggering event if set to
true
. Please note that this requiresseverity
to be positive. - path: Optional identifier that can be defined in case an event might need to be extended with a later request to this endpoint using the same path value.
The endpoint also accepts a batch of events when given an array:
[
{
"title": "First event",
...
},
{
"title": "Second event",
...
}
]
Ruby example
duration = (Time.now.to_f * 1000).floor - deploy_start_time_in_ms
payload = {}
payload[:title] = 'Deployed MyApp'
payload[:text] = 'pglombardo deployed [email protected]'
payload[:duration] = duration
uri = URI('http://localhost:42699/com.instana.plugin.generic.event')
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.body = payload.to_json
Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(req)
end
Curl example
curl -XPOST http://localhost:42699/com.instana.plugin.generic.event -H "Content-Type: application/json" -d '{"title":"Custom API Events ", "text": "Failure Redeploying Service Duration", "duration": 60000, "severity": 5}'
PowerShell example
For PowerShell you can either use the standard Cmdlets Invoke-WebRequest
or Invoke-RestMethod
. The parameters to be provided are basically the same.
Invoke-RestMethod
-Uri http://localhost:42699/com.instana.plugin.generic.event
-Method POST
-Body '{"title":"PowerShell Event ", "text": "You used PowerShell to create this event!", "severity": -1}'
Invoke-WebRequest
-Uri http://localhost:42699/com.instana.plugin.generic.event
-Method Post
-Body '{"title":"PowerShell Event ", "text": "You used PowerShell to create this event!", "severity": -1}'
Mina integration
Learn more about our Mina event integration.
Trace SDK Web service
Endpoint: http://<agent_ip>:42699/com.instana.plugin.generic.trace
Using the Trace SDK REST Web Service, it is possible to integrate Instana into any application regardless of language.
Each active Instana Agent can be used to feed manually captured traces into the Web Service, which can be joined with automatically captured traces or be completely separate.
The Agent offers an endpoint which listens on the http://<agent_ip>:42699/com.instana.plugin.generic.trace
URL and accepts the following JSON via a POST request:
{
"spanId": <string>,
"parentId": <string>,
"traceId": <string>,
"timestamp": <64 bit long>,
"duration": <64 bit long>,
"name": <string>,
"type": <string>,
"error": <boolean>,
"data": {
<string> : <string>
}
}
spanId
is the unique identifier for any particular span.
The trace is defined by a root span, that is, a span that does not have a parentId.
The traceId needs to be identical for all spans that belong to the same trace, and is allowed to be overlapping with a spanId. traceId
, spanId
and parentId
are 64 bit unique values encoded as hex string like b0789916ff8f319f
. Spans form a hierarchy by referencing the spanId
of the parent as parentId
. An example of a span hierarchy in a trace is shown below:
root (spanId=1, traceId=1, type=Entry)
child A (spanId=2, traceId=1, parentId=1, type=Exit)
child A (spanId=3, traceId=1, parentId=2, type=Entry)
child B (spanId=4, traceId=1, parentId=3, type=Exit)
child B (spanId=5, traceId=1, parentId=4, type=Entry)
The timestamp
and duration
fields are in milliseconds.
The timestamp must be the epoch timestamp coordinated to UTC.
The name
field can be any string which is used to visualize and group traces, and can contain any text. However, simplicity is recommended.
The type
field is optional, when absent is treated as ENTRY
. Options are ENTRY
, EXIT
, INTERMEDIATE
, or EUM
. Setting the type is important for the UI. It is assumed that after an ENTRY
, child spans are INTERMEDIATE
or EXIT
. After an EXIT
an ENTRY
should follow. This is visualized as a remote call.
The data
field is optional and can contain arbitrary key-value pairs. The behavior of supplying duplicate keys is unspecified.
The error
field is optional and can be set to true
to indicate an erroneous span.
The endpoint also accepts a batch of spans, which then need to be given as an array:
[
{
// span as above
},
{
// span as above
}
]
For traces received via the Trace SDK Web Service the same rules regarding Conversion and Service/Endpoint Naming are applied as for the Java Trace SDK. In particular these key-value pairs in data
are used for naming: service
, endpoint
and call.name
.
Note: The optional Instana Agent Service provided on Kubernetes via the Instana Agent Helm Chart is very useful in combination with the Trace Web SDK support, as it ensures that the data is pushed to the Instana Agent running on the same Kubernetes node, ensuring the Instana Agent can correctly fill in the infrastructure correlation data.
Curl example
The following example shows how to send to the host agent data about a matching ENTRY
and EXIT
call, which simulates a process that receives an HTTP GET request targeted to the https://orders.happyshop.com/my/service/asdasd
URL and routes it to an upstream service at the https://crm.internal/orders/asdasd
URL.
#!/bin/bash
curl -0 -v -X POST 'http://localhost:42699/com.instana.plugin.generic.trace' -H 'Content-Type: application/json; charset=utf-8' -d @- <<EOF
[
{
"spanId": "8165b19a37094800",
"traceId": "1368e0592a91fe00",
"timestamp": 1591346182000,
"duration": 134,
"name": "GET /my/service/asdasd",
"type": "ENTRY",
"error": false,
"data": {
"http.url": "https://orders.happyshop.com/my/service/asdasd",
"http.method": "GET",
"http.status_code": 200,
"http.path": "/my/service/asdasd",
"http.host": "orders.happyshop.com"
}
},
{
"spanId": "7ddf6b31b320cc00",
"parentId": "8165b19a37094800",
"traceId": "1368e0592a91fe00",
"timestamp": 1591346182010,
"duration": 97,
"name": "GET /orders/asdasd",
"type": "EXIT",
"error": false,
"data": {
"http.url": "https://crm.internal/orders/asdasd",
"http.method": "GET",
"http.status_code": 200,
"http.path": "/orders/asdasd",
"http.host": "crm.internal"
}
}
]
EOF
Infrastructure correlation
Starting with host agent version 1.1.582
, Trace Web SDK data received by the host agent are correlated with the process sending them, provided that:
- the host agent is running on a Linux system with the following commands available:
lsns
,nsenter
andss
; - the process sending the tracing data is running on the same host as the host agent;
- the process sending the tracing data is not ignored using the Ignore Processes capability;
- report traces directly to the host agent, rather than running over a proxy; if the tracing data go over a proxy, the proxy will be associated with the traces instead.
The Application Perspective Service dashboard will correctly reference any process from which related traces were ingested, including for example infra changes that took place for this service.
If any of the preconditions above is not fulfilled, the tracing data will be corrwelated with the host on which the host agent is running.
Limitations
Adhere to the following rate limits for the trace web service:
- Maximum API calls/sec: 20
- Maximum payload per POST request: A span must not exceed 4 KiB. The request size must not exceed 4 MiB.
- Maximum batch size (spans/array): 1000
FAQ
Is there a limit for the amount of calls from Agent to Backend? Data transmission between Instana agent and Instana backend depends on a lot of factors. It is done using a persistent HTTP2 connection and highly optimized.
What is the optimal package size (the split size) to send 50,000 spans with a size about 40 MiB over the Agent to Backend? Recommended strategy is to buffer spans for up to one second or until 500 spans were collected, then transmit the spans to the agent. For an implementation of this transmission strategy, see our repo.
Which requirements are recommended for best Agent performance (cpu, ram, etc.)? This depends on the host that should be monitored, e.g. the number of Docker containers per host etc.
How should the agent environment be configured? See our agent configuration docs.