Amazon CloudWatch ( YAML )

Apica Ascent connects to Amazon CloudWatch using the boto3 client with the help of the AWS CloudWatch data source making it easy for you to query CloudWatch metrics using its natural syntax, analyze, monitor, and create Visualization of data.

Before you query your CloudWatch data, you should set up authentication credentials. Credentials for your AWS account can be found in the IAM Console. You can create or use an existing user. Go to manage access keys and generate a new set of keys.

Adding Amazon CloudWatch ( YAML ) data source

The first step is to create an Amazon CloudWatch data source and provide all details such as the Name, AWS Region, AWS Access Key, AWS Secret Key

  • Name: Name of the Data Source

  • AWS Region: Region of your AWS account

  • AWS Access Key: access_key_id of your IAM Role

  • AWS Secret Key: secret_access_key of your IAM Role

Querying CloudWatch

These instructions assume you are familiar with the CloudWatch ad-hoc query language. To make exploring your data easier the schema browser will show which Namespaces and Metrics ( optionally dimensions ) you can query.

CloudWatch query designer wizard

Apica Ascent includes a simple point-and-click wizard for creating CloudWatch queries. You can launch the query wizard by selecting the CloudWatch YAML data source and selecting the "Construct CloudWatch query" icon.

In the query designer, you can select the Namespace, Metric, and Dimensions along with the Stat. You can add one or more Namespaces, Metric using a simple point-and-click interface.

You are now ready to run and plot the metric. Running Execute will automatically create the built-in line graph for your metric. You can further create additional visualizations using "New Visualization".

Deep-dive into query language for CloudWatch queries

For the curious, here is a breakdown of the YAML syntax and what the various attributes mean. NOTE: You don't need to write or type these to query data. The No-code built-in WYSIWYG editor makes it easy to query CloudWatch without writing any code. Let us look at the YAML syntax now. It should be an array of MetricDataQuery objects under a key called MetricsDataQueries.

Here's an example that sends MetricDataQuery

MetricDataQueries: 
  - Id: q1
    MetricStat:
      Metric:
        Namespace: AWS/Logs
        MetricName: IncomingLogEvents
        Dimensions:
          - Name: LogGroupName
            Value: flowlogs
      Period: 300
      Stat: Sum
StartTime: "2022-07-04 00:00:00"

Your query can include the following keys:

Key

LogGroupName

string

LogGroupNames

array of strings

StartTime

integer or timestring

EndTime

integer or timestring

QueryString

string

Limit

integer

Querying AWS Lambda Metrics

Let's look at a slightly more complex example and query AWS Lambda metrics for AWS Lambda Errors. In this example, we are using the MetricName: "Errors" for the "AWS/Lambda" Namespace.

When selecting the AWS/Lambda Namespace, you can see the available MetricNames

  • AWS/Lambda

    • Errors

    • ConcurrentExecutions

    • Invocations

    • Throttles

    • Duration

    • IteratorAge

    • UnreservedConcurrentExecutions

Below is an example query that tracks AWS Lambda errors as an aggregate metric. The StartTime is templatized and allows dynamic selection.

MetricDataQueries: 
  - Id: q1
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Errors
      Period: 300
      Stat: Sum
StartTime: "{{StartTime}}"

You can further click on the Errors MetricName and it will expand to show you Dimensions available for further querying. For AWS/Lambda, the Dimension FunctionName provides further drill down to show Cloudwatch metrics by Lambda Function Name.

MetricDataQueries: 
  - Id: q1
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Errors
        Dimensions:
          - Name: FunctionName
            Value: <My lambda function name>
      Period: 300
      Stat: Sum
StartTime: "{{StartTime}}"

The query can be further enhanced by making the lambda function name, a templatized parameter. This allows you to pull metrics using a dropdown selection e.g. a list of lambda functions. The FunctionName template below can also be retrieved from another database as a separate query.

MetricDataQueries: 
  - Id: q1
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Errors
        Dimensions:
          - Name: FunctionName
            Value: {{FunctionName}}
      Period: 300
      Stat: Sum
StartTime: "{{StartTime}}"

Examples Queries

Query using a single Expression

StartTime: 1518867432,
EndTime: 1518868432,
MetricDataQueries :
    -Id: errorRate,
    Label: Error Rate,
    Expression: errors/requests

An expression can be a mathematical expression of metrics or an sql query.

Query using a list of expressions

  StartTime: 1518867432
  EndTime: 1518868432
  MetricDataQuerie:
      - Id: errorRate
        Label: Error Rate
        Expression: errors/requests
      - Id: errorRatePercent
        Label: %Error Rate
        Expression: errorRate*100

Query using metric-stat or a list of metric-stats:

StartTime: 1518867432
EndTime: 1518868432
MetricDataQueries:
  - Id: invocations
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Invocations
      Period: 600
      Stat: Sum
  - Id: errors
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Errors
      Period: 600
      Stat: Sum

Each list item in the MetricDataQueries list in the above mentioned examples can contain either an Expression or a MetricStat Query item. we can provide a combination of both also.

Query using a combination of MetricStat and Expression:

  StartTime: 1518867432
  EndTime: 1518868432
  MetricDataQuerie:
      - Id: errorRate
        Label: Error Rate
        Expression: errors*500
      - Id: errors
        MetricStat:
          Metric:
            Namespace: AWS/Lambda
            MetricName: Errors
          Period: 600
          Stat: Sum

In the above example the second item uses MetricStat syntax to fetch data and the first item uses expression syntax to fetch the data. here, first item is used to perform a math expression on the data fetched by second item.

Query Example to Perform math Expression on fetched data

StartTime: 1518867432
EndTime: 1518868432
MetricDataQueries:
  - Id: invocations
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Invocations
      Period: 600
      Stat: Sum
  - Id: errors
    MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Errors
      Period: 600
      Stat: Sum
  - Id: errorRatio
    Expression: errors/invocations*100

In the above example first and second items are used to fetch metric data. the third item is used to perform a mathematical expression on the data fetched using the first and second items.

Query to use Period and Stat in MetricDataQueries items

The period indicates granularity and stat indicates the group by operation to be performed on the fetched data.

Id: errors
MetricStat:
      Metric:
        Namespace: AWS/Lambda
        MetricName: Invocations
      Period: 600
      Stat: Sum

or

Id: errors
Expression: 'some SQL query or a math expression'
period: 600
Stat: Avg

for some detailed information on querying cloud-watch metrics, follow the below links https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html https://docs.aws.amazon.com/cli/latest/reference/cloudwatch/get-metric-data.html

PreviousAmazon AthenaNextAmazon Elasticsearch Service

Last updated