Amazon Web Services (AWS)

The AWS Agent collects data from AWS APIs, such as CloudWatch, S3 and X-Ray, for which there is not possible to install a host agent or any other type of agent.

Platforms

Monitored Services

Note: Due to the nature of the AWS CloudWatch API, a delay in metrics retrieval is possible. The displayed values for the metrics are also delayed to ensure consistency. This delay depends on the actual AWS service and availability of data at that time, but usually is around 10 minutes.

Installation

Note: This documentation covers installing the AWS Agent, which allows you to collect data from CloudWatch and other datasources to monitor services managed by Amazon. If you want to monitor an Amazon Elastic Computing (EC2) Virtual Machine, a Kubernetes cluster running on AWS, either installed and managed by you or using Amazon Elastic Kubernetes Service, or an Amazon Elastic Container cluster, refer to the Install the Instana Host Agent documentation.

Note: Install only one AWS Agent per combination of AWS account and AWS region. Installing multiple AWS Agents for the same combination of AWS account and AWS region can incur additional costs from AWS, without added benefit in terms of quality of monitoring via Instana.

Installation on EC2

We recommend that you run the Instana AWS sensor on an Current Generation General Purpose machine running Linux. For example, m5.large instances are ideal.

  1. Sign in to Instana and click More -> Agents -> Instana agent installation -> AWS.
  2. From the Technology list, select Instana AWS Sensor.
  3. From the Run your AWS Agent on list, select Elastic Cloud Compute (EC2) (it's the default)

    Your agent key and location are pre-populated in the script to be used as User Data on EC2. The script looks like the following, except that all the necessary information is provided:

    #!/bin/bash
    
    curl -o setup_agent.sh https://setup.instana.io/agent
    chmod 700 ./setup_agent.sh
    sudo ./setup_agent.sh -y -a <your-agent-key> -m aws -t dynamic -e <location> -s
  4. Spin up a dedicated EC2 virtual machine and use the script as User Data; refer to the Running Commands on Your Linux Instance at Launch documentation for more information on how to user User Data on EC2.
  5. Copy the configuration.json file for the IAM role. This is the required IAM role for the EC2 virtual machine running the Instana AWS sensor, and enables the Instana agent to discover and monitor your AWS resources.
  6. For the IAM role to perform the AssumedRole action, edit the Trust Relationship. For example:

    {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "Service": "ec2.amazonaws.com"
         },
         "Action": "sts:AssumeRole"
       }
     ]
    }

Installation on Fargate on ECS

  1. Sign in to Instana and click More -> Agents -> Instana agent installation -> AWS.
  2. From the Technology list, select Instana AWS Sensor.
  3. From the Run your AWS Agent on list, select Elastic Container Service (ECS)

    The JSON of a Task Definition is pre-generated for you in the Task Definition JSON template, and you can use it with the Configure via JSON functionality of the Task Definition user interface.

  4. Assign to the new Task Definition an IAM role that contains at least the permissions listed under the IAM permissions JSON document.
  5. Create an ECS Service with one instance of the newly created Task Definition.

Configuration

Installation outside your AWS infrastructure

You can also dedicate any agent running outside of your AWS infrastructure. For this you will need to specify the following environment variables within the /opt/instana/agent/bin/setenv file:

  • The region you want to monitor:

    INSTANA_AWS_REGION_CONFIG=
  • The credentials to access AWS resources. Please note that these credentials should belong to a user which is allowed to access the resources already described in the Amazon Web Services IAM Configuration section.

    AWS_ACCESS_KEY_ID=
    AWS_SECRET_ACCESS_KEY=

Enable Monitoring for Specific Services

For each service, you need to add the required permissions noted below on the individual service pages, linked to from the Monitored Services section. To exclude a service from monitoring, leave out the respective permission.

Alternatively Another way to exclude a service from monitoring is to add the appropriate enabled flag in <agent_install_dir>/etc/instana/configuration.yml as described on the individual service pages below.

Monitoring multiple AWS accounts

The AWS agent supports monitoring services from multiple AWS accounts in the same region.

In order to monitor multiple accounts, you need to define named profiles in the same way you would with the AWS CLI, or creating the file manually. The .aws/credentials file must be created within the home directory of the user which is running Instana agent, which is usually root. The AWS CLI uses credentials files ~/.aws/credentials like the following:

[default]
aws_access_key_id = accessKey1
aws_secret_access_key = secretAccessKey1

[profile2]
aws_access_key_id = accessKey2
aws_secret_access_key = secretAccessKey2

[profile3]
aws_access_key_id = accessKey3
aws_secret_access_key = secretAccessKey3

Every profile represents the mapping between AWS named profile and AWS access credentials. Upon installation of an AWS agent, a default AWS profile will be created; there is no need to add the default profile to ~/.aws/credentials file.

The additional profiles to be used by the AWS agent must be listed as follows in the agent configuration file /opt/instana/agent/etc/instana/configuration.yaml:

com.instana.plugin.aws:
  profile_names:
    - 'profile2'
    - 'profile3'

The monitoring of multiple AWS accounts can also be specified for particular AWS services within their specific configuration. The list of profiles specified for a specific service will override the general configuration shown above.

Note: Be sure to follow recommendations when it comes to creation of credentials file. To avoid any security threats, you should either set more strict rules for EC2 instance or create a separate Instana AWS user and that way avoid adding root user security credentials to the ~/.aws/credentials file.

Filtering and Tagging

Note: To enable filtering, the IAM permission tag:GetResources should be included in the IAM permissions granted to the AWS Agent.

After enabling the monitoring of a service, it is possible to filter which of its instances Instana will monitor based on AWS tags by modifying the agent configuration file /opt/instana/agent/etc/instana/configuration.yaml of the AWS Agent.

Available options in the configuration file are:

com.instana.plugin.aws:
  # Comma-separated list of tags in key:value format.
  # Any AWS resource containing at least one of the specified tags is discovered.
  include_tags: ...
  # Comma-separated list of tags in key:value format.
  # Any AWS resource containing at least one of the specified tags is omitted from discovery.
  exclude_tags: ...

Multiple tags can be defined, separated by a comma. Tags should be a key-value pair separated by :. When defining tags in both lists (include and exclude), the exclude list has higher priority. If there is no need for services filtering, the configuration should not be defined.

Note: The Instana agent automatically retrieves data from CloudWatch for all AWS resources that have no tags assigned.

Filtering can also be configured on the service level. In this case, the default configuration is overridden for the particular service. For more details about discovery filtering by specific service, see the specific service documentation.

Poll Rate

To specify how often sensors poll the AWS tagged resource, use the tagged-services-poll-rate configuration property. The default is 300 seconds.

The poll-rate configuration for the tagged resources is:

com.instana.plugin.aws:
  tagged-services-poll-rate: 60 #default 300