AWS IAM Resources

NOTE: LOGIQ can use Athena and Glue on AWS optionally to power the SQL queries. Instructions below for Athena/Glue are only needed if you choose to use those services.
LOGIQ User Role
You need to provide AWS access key and secret in LOGIQ server configuration with the following permissions. 
1
{
2
    "Version": "2012-10-17",
3
    "Statement": [
4
        {
5
            "Sid": "VisualEditor0",
6
            "Effect": "Allow",
7
            "Action": [
8
                "s3:PutObject",
9
                "s3:GetObject",
10
                "s3:ListBucket"
11
            ],
12
            "Resource": [
13
                "arn:aws:s3:::<YOUR_BUCKET_NAME>",
14
                "arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
15
            ]
16
        },
17
        {
18
            "Sid": "VisualEditor1",
19
            "Effect": "Allow",
20
            "Action": [
21
                "iam:PassRole",
22
                "s3:ListAllMyBuckets",
23
                "glue:CreateCrawler"
24
            ],
25
            "Resource": "*"
26
        }
27
    ]
28
}
Copied!
​
You also need to provide AWS access key and secret in LOGIQ server configuration with AmazonAthenaFullAccess (arn:aws:iam::aws:policy/AmazonAthenaFullAccess) policy.
IAM Service Role For Glue
You need to grant your IAM role permissions that AWS Glue can assume when calling other services on your behalf. This includes access to Amazon S3 for any sources, targets, scripts, and temporary directories that you use with AWS Glue. Permission is needed by crawlers, jobs, and development endpoints.
Please refer to the following guide for creating a service role for glue
Create an IAM Role for AWS Glue
Please refer to the following guide for creating a service role for glue
Use the below Inline policy for Glue Service Role. You need to update the policy with your actual S3 Bucket name. Alternately you can use the Cloud formation template below to create the Glue Service Role
1
{
2
    "Version": "2012-10-17",
3
    "Statement": [
4
        {
5
            "Effect": "Allow",
6
            "Action": [
7
                "glue:*",
8
                "s3:GetBucketLocation",
9
                "s3:ListBucket",
10
                "s3:ListAllMyBuckets",
11
                "s3:GetBucketAcl",
12
                "iam:ListRolePolicies",
13
                "iam:GetRole",
14
                "iam:GetRolePolicy",
15
                "cloudwatch:PutMetricData"
16
            ],
17
            "Resource": [
18
                "*"
19
            ]
20
        },
21
        {
22
            "Effect": "Allow",
23
            "Action": [
24
                "s3:GetObject",
25
                "s3:PutObject",
26
                "s3:DeleteObject"
27
            ],
28
            "Resource": [
29
                "arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
30
            ]
31
        },
32
        {
33
            "Effect": "Allow",
34
            "Action": [
35
                "logs:CreateLogGroup",
36
                "logs:CreateLogStream",
37
                "logs:PutLogEvents"
38
            ],
39
            "Resource": [
40
                "arn:aws:logs:*:*:/aws-glue/*"
41
            ]
42
        }
43
    ]
44
}
Copied!
AWS Cloud formation template for Glue Service Role
Glue-Service-Role-Cloudformation.yml
1
AWSTemplateFormatVersion: '2010-09-09'
2
Description: >-
3
  This template will build out the IAM Roles for Logiq Glue Crawler
4
​
5
Parameters:
6
  S3BucketName:
7
    Description: Name of the Logiq S3 bucket 
8
    Type: String
9
​
10
Resources:
11
  LogiqGlueCrawlerPolicy:
12
    Type: 'AWS::IAM::ManagedPolicy'
13
    Properties:
14
      ManagedPolicyName: Logiq-Glue-Crawler-Policy
15
      Description: Policy for Logiq Glue Crawler
16
      Roles: 
17
        - !Ref LogiqGlueCrawlerRole
18
      PolicyDocument:
19
        Version: 2012-10-17
20
        Statement:
21
          - Effect: Allow
22
            Action:
23
              - 'glue:*'
24
              - 's3:GetBucketLocation'
25
              - 's3:ListBucket'
26
              - 's3:ListAllMyBuckets'
27
              - 's3:GetBucketAcl'
28
              - 'iam:ListRolePolicies'
29
              - 'iam:GetRole'
30
              - 'iam:GetRolePolicy'
31
              - 'cloudwatch:PutMetricData'
32
            Resource:
33
              - '*'   
34
          - Effect: Allow
35
            Action:
36
              - 's3:DeleteObject'
37
              - 's3:PutObject'
38
              - 's3:GetObject'
39
            Resource:
40
              - !Sub 'arn:aws:s3:::${S3BucketName}/*'   
41
          - Effect: Allow
42
            Action:
43
              - 'logs:PutLogEvents'
44
              - 'logs:CreateLogStream'
45
              - 'logs:CreateLogGroup'
46
            Resource:
47
              - 'arn:aws:logs:*:*:/aws-glue/*'  
48
​
49
  LogiqGlueCrawlerRole:
50
    Type: 'AWS::IAM::Role'
51
    Properties: 
52
      AssumeRolePolicyDocument: 
53
        Version: '2012-10-17'
54
        Statement: 
55
          - 
56
            Effect: 'Allow'
57
            Principal: 
58
              Service: 
59
                - 'glue.amazonaws.com'
60
            Action: 
61
              - 'sts:AssumeRole'
62
      RoleName: Logiq-Glue-Service-Role
Copied!

RUNNING ON AWS - Previous

Getting started

Next - RUNNING ON AWS

1-Click deployment using CloudFormation

Last modified 1yr ago

Export as PDF

Copy link

Contents

LOGIQ User Role

IAM Service Role For Glue

Create an IAM Role for AWS Glue

AWS Cloud formation template for Glue Service Role