Skip to content

Using AWS API Gateway

DRAFT

AWS API Gateway unable to import valid OAS 3.0 spec

After some experiments trying to import our complete OAS 3.0 spec, I've found that there are a number of limitations to AWS API Gateway:

Can't handle parameters with square brackets:

1
Unable to put method 'GET' on resource at path '/course_terms': Invalid mapping expression specified: Validation Result: warnings : [], errors : [Parameter name should match the following regular expression: ^[a-zA-Z0-9._$-]+$]

Changing page[size] to page_size works -- but is incorrect. Totally breaks all the JSONAPI stuff like filter[], etc.

Can't handle http basicAuth type

1
Unsupported security definition type 'http' for 'basicAuth'. Ignoring.

See https://forums.aws.amazon.com/thread.jspa?threadID=305421

Can't handle oauth2

1
Unsupported security definition type 'oauth2' for 'oauth'. Ignoring.

So, instead, let's just create a simple proxy for now to at least test out some of the other AWS API GW features:

Create a proxy+ gateway

In lieu of importing an OAS 3.0 spec, one can create a "wildcard" proxy+ API gateway which just passes everything through to the backend, but can still add valuable gateway functionality such as rate-limiting, OAuth 2.0 access token introspection, etc.

A basic OAuth 2.0 API Authorization Lambda Function

See aws/lambda_function.py which is based on this example lambda function.

I've added an introspect() function which simply checks for a valid active Bearer Token and allows the API through if it is present. If the function is configured with the scopeAlternatives environment variable, it also performs required scope checking. If not, the token is only checked to make sure it is active.

First let's create some assumeable roles:

  • lambda_basic_execution allows executing the lambda function and letting it append to the logs???
  • lambda_invoke_function_assume_apigw_role allows the API GW authorizer to run our lambda function.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ aws iam create-role --role-name lambda_basic_execution --assume-role-policy-document file://lambda_basic_assume_role_policy.json --profile alan:CTO
{
    "Role": {
        "Path": "/",
        "RoleName": "lambda_basic_execution",
        "RoleId": "AROAZDZCSVJODEMVKCAES",
        "Arn": "arn:aws:iam::123456789012:role/lambda_basic_execution",
        "CreateDate": "2019-07-24T16:30:01Z",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "lambda.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}
$ aws iam attach-role-policy --role-name lambda_basic_execution --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole --profile alan:CTO
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
$ aws iam create-policy --policy-name lambda_execute --policy-document file://lambda_execute_policy.json --profile alan:CTO

    "Policy": {
        "PolicyName": "lambda_execute",
        "PolicyId": "ANPAZDZCSVJOI4UPMG6TQ",
        "Arn": "arn:aws:iam::123456789012:policy/lambda_execute",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2019-07-24T17:42:14Z",
        "UpdateDate": "2019-07-24T17:42:14Z"
    }
}
$ aws iam create-role --role-name lambda_invoke_function_assume_apigw_role --assume-role-policy-document file://apigateway_assume_role_lambda.json --profile alan:CTO
{
    "Role": {
        "Path": "/",
        "RoleName": "lambda_invoke_function_assume_apigw_role",
        "RoleId": "AROAZDZCSVJOAQRAVLCWQ",
        "Arn": "arn:aws:iam::123456789012:role/lambda_invoke_function_assume_apigw_role",
        "CreateDate": "2019-07-24T17:30:52Z",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "",
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "lambda.amazonaws.com",
                            "apigateway.amazonaws.com"
                        ]
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}
$ aws iam attach-role-policy --role-name lambda_invoke_function_assume_apigw_role --policy-arn arn:aws:iam::123456789012:policy/lambda_execute --profile alan:CTO

The lambda function uses environment variables to configure it so let's install both a test and prod flavor:

1
2
3
4
5
6
$ rm -f lambda.zip
$ # replace dummy account number
$ sed -i.bak -e s/999999999999/123456789012/g lambda_function.py
$ zip lambda.zip lambda_function.py
$ aws lambda create-function --function-name introspect_test --runtime python3.7 --role arn:aws:iam::123456789012:role/lambda_basic_execution  --handler lambda_function.lambda_handler --zip-file fileb://lambda.zip --environment "Variables={clientId=demo_resource_server,clientSecret=wL0pgS5RcNOgdOSSmejzZNA605d3MtkoXMVSDaJxmaTU70XnYQPOabBAYtfkWXay,introspectionUrl=https://oauth-test.cc.columbia.edu/as/introspect.oauth2}" --profile alan:CTO
$ aws lambda create-function --function-name introspect_prod --runtime python3.7 --role arn:aws:iam::123456789012:role/lambda_basic_execution  --handler lambda_function.lambda_handler --zip-file fileb://lambda.zip --environment "Variables={clientId=demo_resource_server,clientSecret=wL0pgS5RcNOgdOSSmejzZNA605d3MtkoXMVSDaJxmaTU70XnYQPOabBAYtfkWXay,introspectionUrl=https://oauth.cc.columbia.edu/as/introspect.oauth2}" --profile alan:CTO

TODO: Document adding scopeAlternatives. For now, use the AWS Console and cut-n-paste the contents of scopeAlternatives.json.

Testing the lambda function

You can test the lambda function in the AWS console or via the CLI as follows:

1
2
3
4
5
6
7
$ aws lambda invoke --function-name introspect_test --payload '{  "methodArn": "arn:aws:execute-api:us-east-1:123456789012:bc28rnvr33/test/GET/v1/courses",  "authorizationToken": "Bearer y18albT1cyVRPbrt2UkSzfyM8nij"}' t.json --profile alan:CTO
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
$ cat t.json 
{"principalId": "ac45@columbia.edu", "policyDocument": {"Version": "2012-10-17", "Statement": [{"Action": "execute-api:Invoke", "Effect": "Allow", "Resource": ["arn:aws:execute-api:us-east-1:123456789012:bc28rnvr33/test/GET/v1/courses"]}]}}

In the above, you'll need to replace the Bearer token with an active one and make sure the required scopes as defined in scopeAlternatives are set to get a succesful response.

Import an OAS 3.0 document to speed up creating the API

There are a lot of steps in setting up an API manually. Let's use the result of a bunch of manual setup via the AWS Console which was then exported for later re-import.

Here's the export command we used:

1
$ aws apigateway get-export --rest-api-id bc28rnvr33 --stage-name test --export-type oas30 --parameters {"extensions":"integrations,authorizers,apigateway"} --profile alan:CTO foo.json

And here's the OAS 3.0 spec, with a bunch of AWS-specific extension fields:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
{
  "openapi" : "3.0.1",
  "info" : {
    "title" : "jsonapi proxy+",
    "version" : "2019-07-24T18:09:15Z"
  },
  "servers" : [ {
    "url" : "https://bc28rnvr33.execute-api.us-east-1.amazonaws.com/{basePath}",
    "variables" : {
      "basePath" : {
        "default" : "/test"
      }
    }
  } ],
  "paths" : {
    "/{proxy+}" : {
      "options" : {
        "responses" : {
          "200" : {
            "description" : "200 response",
            "headers" : {
              "Access-Control-Allow-Origin" : {
                "schema" : {
                  "type" : "string"
                }
              },
              "Access-Control-Allow-Methods" : {
                "schema" : {
                  "type" : "string"
                }
              },
              "Access-Control-Allow-Headers" : {
                "schema" : {
                  "type" : "string"
                }
              }
            },
            "content" : { }
          }
        },
        "x-amazon-apigateway-integration" : {
          "responses" : {
            "default" : {
              "statusCode" : "200",
              "responseParameters" : {
                "method.response.header.Access-Control-Allow-Methods" : "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'",
                "method.response.header.Access-Control-Allow-Headers" : "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'",
                "method.response.header.Access-Control-Allow-Origin" : "'*'"
              }
            }
          },
          "requestTemplates" : {
            "application/json" : "{\"statusCode\": 200}"
          },
          "passthroughBehavior" : "when_no_match",
          "type" : "mock"
        }
      },
      "x-amazon-apigateway-any-method" : {
        "parameters" : [ {
          "name" : "proxy",
          "in" : "path",
          "required" : true,
          "schema" : {
            "type" : "string"
          }
        } ],
        "responses" : {
          "200" : {
            "description" : "200 response",
            "content" : { }
          }
        },
        "security" : [ {
          "introspection" : [ ]
        }, {
          "api_key" : [ ]
        } ],
        "x-amazon-apigateway-integration" : {
          "uri" : "http://ac45devapp01.cc.columbia.edu:9123/{proxy}/",
          "responses" : {
            "default" : {
              "statusCode" : "200"
            }
          },
          "requestParameters" : {
            "integration.request.path.proxy" : "method.request.path.proxy"
          },
          "passthroughBehavior" : "when_no_match",
          "httpMethod" : "ANY",
          "cacheNamespace" : "psuvk6",
          "cacheKeyParameters" : [ "method.request.path.proxy" ],
          "type" : "http_proxy"
        }
      }
    }
  },
  "components" : {
    "securitySchemes" : {
      "introspection" : {
        "type" : "apiKey",
        "name" : "Authorization",
        "in" : "header",
        "x-amazon-apigateway-authtype" : "custom",
        "x-amazon-apigateway-authorizer" : {
          "authorizerUri" : "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:123456789012:function:introspect_test/invocations",
          "authorizerCredentials" : "arn:aws:iam::123456789012:role/lambda_invoke_function_assume_apigw_role",
          "authorizerResultTtlInSeconds" : 300,
          "identityValidationExpression" : "Bearer .*$",
          "type" : "token"
        }
      },
      "api_key" : {
        "type" : "apiKey",
        "name" : "x-api-key",
        "in" : "header"
      }
    }
  }
}

Let's import it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
$ # replace dummy AWS account number
$ sed -i.bak -e s/999999999999/123456789012/g proxy+oas3.json
$ aws apigateway import-rest-api --body file://proxy+oas3.json --parameters endpointConfigurationTypes=REGIONAL --profile alan:CTO
{
    "id": "p32r23u5jb",
    "name": "jsonapi proxy+",
    "createdDate": 1563992583,
    "version": "2019-07-24T18:09:15Z",
    "apiKeySource": "HEADER",
    "endpointConfiguration": {
        "types": [
            "REGIONAL"
        ]
    }
}

Deploy the API

1
2
3
4
5
$ aws apigateway create-deployment --rest-api-id bc28rnvr33 --stage-name test --profile alan:CTO
{
    "id": "zrkle6",
    "createdDate": 1563997322
}

The base URL for the deployment follows the pattern https://<restApiId>.execute-api.<awsRegion>.amazonaws.com/<stageName> so our's is https://bc28rnvr33.execute-api.us-east-1.amazonaws.com/test.

Next Steps

  • learn more about available features such as load-balancing, rate-limiting, caching, etc.

AWS API Developer Portal

AWS has published a serverless developer portal which is easily launched via a CloudFormation template.

The portal sucks

It's not a very good developer portal and has a number of gotchas:

In short, it's nowhere near as nice as products like Gravitee APIM -- which doesn't support configuring AWS API GW.