AWS/CloudFormation

From r00tedvw.com wiki
Jump to: navigation, search

Quick Reference | AWS CLI | CloudFormation

Contents

AWS Cloudformation CLI

Testing template

https://docs.aws.amazon.com/cli/latest/reference/cloudformation/validate-template.html

~$ aws cloudformation validate-template --template-body file://$HOME/s3_ec2.yaml
{
    "CapabilitiesReason": "The following resource(s) require capabilities: [AWS::IAM::Policy]",
    "Capabilities": [
        "CAPABILITY_IAM"
    ],
    "Parameters": [
        {
            "DefaultValue": "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2",
            "NoEcho": false,
            "ParameterKey": "LatestAmiId"
        },
        {
            "NoEcho": false,
            "Description": "Random String",
            "ParameterKey": "Entropy"
        }
    ]
}

Creating stack

https://docs.aws.amazon.com/cli/latest/reference/cloudformation/create-stack.html

~$ aws cloudformation create-stack --stack-name s3ec2stack01 --template-body file://$HOME/s3_ec2.yaml --parameters ParameterKey=Entropy,ParameterValue=testentropy ParameterKey=BranchName,ParameterValue=testbranch --capabilities CAPABILITY_IAM
{
    "StackId": "arn:aws:cloudformation:us-east-1:548975612458:stack/s3ec2stack01/177a94e0-9923-23et-b86a-0a5462e42a87"
}

Template Examples

S3 Bucket

Here is a CloudFormation Template (in yaml) that I used to create a S3 bucket with the following requirements:

  • Public Read
  • 7 day retention policy
  • Bucket Policy allowing access to all resources.
  • Export the S3 bucket name, Secure URL, and Website URL for usage in other templates.

AWS Template:

AWSTemplateFormatVersion: 2010-09-09

Parameters:
  BranchName:
    Description: Branch Name
    Type: String

Resources:

  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      LifecycleConfiguration:
        Rules:
          - Status: Enabled
            ExpirationInDays: 7
      WebsiteConfiguration:
        IndexDocument: index.html
  
  S3BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: S3Bucket
      PolicyDocument:
        Statement:
        - Sid: AddPerm
          Effect: 'Allow'
          Principal: '*'
          Action:
          - 's3:GetObject'
          Resource:
            Fn::Join:
            - ''
            - - 'arn:aws:s3:::'
              - Ref: 'S3Bucket'
              - '/**'

Outputs:
  S3BucketName:
    Value:
      Ref: S3Bucket
    Description: The S3 bucket name
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-S3BucketName

  S3BucketSecureURL:
    Value:
      Fn::Join: ['', ['http://', !GetAtt [S3Bucket, DomainName]]]
    Description: Name of S3 bucket to hold Tenant Management website content

  S3WebsiteURL:
    Description: Website URL of the S3 Bucket
    Value:  !Select [1, !Split ["//", !GetAtt S3Bucket.WebsiteURL]]
    Export: 
      Name: !Sub "${AWS::StackName}-S3WebsiteURL"

Lambda & CloudFront

Here is a CloudFormation Template (in yaml) that I used to create a 2 Lambda functions and a cloudfront distribution with the following requirements:

  • use outputs from S3 bucket stack
  • create lambda functions that will work with lambda edge.
  • setup the needed IAM roles for lambda functions to execute on lambda edge.
  • create cloudfront distribution that uses the lambda edge functions and configures the needed behaviors to use them.

AWS Template:

AWSTemplateFormatVersion: 2010-09-09

Parameters:
  BranchName:
    Description: Branch Name
    Type: String
  S3BucketStack:
    Description: S3 Bucket Stack Name
    Type: String
    Default: SYNC-s3-dev-ui-stack
  Entropy:
    Description: Random String
    Type: String

Resources:

  LambdaEdgeRoleRequest:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
            - edgelambda.amazonaws.com
          Action: sts:AssumeRole
      Path: "/service-role/"
      RoleName:
        Fn::Sub: 'LambaEdgeRoleRequest-${BranchName}-${Entropy}'
      Policies:
        - PolicyName: 
            Fn::Sub: 'AWSLambdaBasicExecutionRole-OriginRequest-${Entropy}'
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: "Allow"
                Action:
                  - "logs:CreateLogGroup"
                Resource: 
                  Fn::Sub: "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*"
              - Effect: "Allow"
                Action:
                  - "logs:CreateLogStream"
                  - "logs:PutLogEvents"
                Resource: 
                  Fn::Sub: "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/SyncUI-${BranchName}-OriginRequest-${Entropy}:*"

  LambdaFunctionRequest:
    Type: AWS::Lambda::Function
    DependsOn: LambdaEdgeRoleRequest
    DeletionPolicy: Retain
    Properties:
      Handler: "index.handler"
      Role:
        Fn::GetAtt: 
        - "LambdaEdgeRoleRequest"
        - "Arn"
      Runtime: "nodejs10.x"
      Code:
        S3Bucket: 
          Fn::ImportValue:
            !Sub ${S3BucketStack}-S3BucketName
        S3Key: !Sub "${BranchName}/OriginRequest.zip"
      FunctionName: 
        Fn::Sub: 'SyncUI-${BranchName}-OriginRequest-${Entropy}'
  
  LambdaFunctionRequestVersion:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: 
        Ref: "LambdaFunctionRequest"

  LambdaEdgeRoleResponse:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
            - edgelambda.amazonaws.com
          Action: sts:AssumeRole
      Path: "/service-role/"
      RoleName:
        Fn::Sub: 'LambaEdgeRoleResponse-${BranchName}-${Entropy}'
      Policies:
        - PolicyName: 
            Fn::Sub: 'AWSLambdaBasicExecutionRole-OriginResponse-${Entropy}'
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: "Allow"
                Action:
                  - "logs:CreateLogGroup"
                Resource: 
                  Fn::Sub: "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*"
              - Effect: "Allow"
                Action:
                  - "logs:CreateLogStream"
                  - "logs:PutLogEvents"
                Resource: 
                  Fn::Sub: "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/SyncUI-${BranchName}-OriginResponse-${Entropy}:*"

  LambdaFunctionResponse:
    Type: AWS::Lambda::Function
    DependsOn: LambdaEdgeRoleResponse
    DeletionPolicy: Retain
    Properties:
      Handler: "index.handler"
      Role:
        Fn::GetAtt: 
          - "LambdaEdgeRoleResponse"
          - "Arn"
      Runtime: "nodejs10.x"
      Code:
        S3Bucket: 
          Fn::ImportValue:
            !Sub ${S3BucketStack}-S3BucketName
        S3Key: !Sub "${BranchName}/OriginResponse.zip"
      FunctionName: 
        Fn::Sub: 'SyncUI-${BranchName}-OriginResponse-${Entropy}'  

  LambdaFunctionResponseVersion:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: 
        Ref: "LambdaFunctionResponse"

  CloudFrontDistribution:
    Type: AWS::CloudFront::Distribution
    DependsOn: 
    - LambdaFunctionRequest 
    - LambdaFunctionResponse
    Properties:
      DistributionConfig:
        Enabled: 'true'
        Origins:
        - DomainName:
            Fn::ImportValue:
              !Sub "${S3BucketStack}-S3WebsiteURL"
          OriginPath:
            Fn::Sub: '/${BranchName}'
          CustomOriginConfig:
            OriginProtocolPolicy: match-viewer
          Id: !Sub "CFID-${BranchName}-${Entropy}"
        DefaultCacheBehavior:
          TargetOriginId: !Sub "CFID-${BranchName}-${Entropy}"
          ForwardedValues:
            QueryString: 'false'
          ViewerProtocolPolicy: allow-all
          LambdaFunctionAssociations:
          - EventType: origin-request
            LambdaFunctionARN: !Ref LambdaFunctionRequestVersion
          - EventType: origin-response
            LambdaFunctionARN: !Ref LambdaFunctionResponseVersion
              
        CacheBehaviors:
        - PathPattern:
            Fn::Sub: '/${BranchName}'
          ForwardedValues:
            QueryString: false
          TargetOriginId: !Sub "CFID-${BranchName}-${Entropy}"
          ViewerProtocolPolicy: allow-all
          LambdaFunctionAssociations:
          - EventType: origin-request
            LambdaFunctionARN: !Ref LambdaFunctionRequestVersion
          - EventType: origin-response
            LambdaFunctionARN: !Ref LambdaFunctionResponseVersion
        - PathPattern:
            Fn::Sub: '/${BranchName}/*'
          ForwardedValues:
            QueryString: false
          TargetOriginId: !Sub "CFID-${BranchName}-${Entropy}"
          ViewerProtocolPolicy: allow-all
          LambdaFunctionAssociations:
          - EventType: origin-request
            LambdaFunctionARN: !Ref LambdaFunctionRequestVersion
          - EventType: origin-response
            LambdaFunctionARN: !Ref LambdaFunctionResponseVersion

OriginRequest.js

'use strict';
 
const pointsToFile = uri => /\/[^/]+\.[^/]+$/.test(uri);
const appName = '{branchname}';
 
exports.handler = (event, context, callback) => {
 
  console.log(JSON.stringify(event, null, 2));
 
  // Extract the request from the CloudFront event that is sent to [email protected]
  const request = event.Records[0].cf.request;
 
  // Extract the URI from the request
  const oldUri = request.uri;
  let newUri;
 
  if (!pointsToFile(oldUri) && !oldUri.endsWith('/')) {
       
    newUri = request.querystring ? `${oldUri}/?${request.querystring}` : `${oldUri}/`;
     
    console.log('new uri for path: ' + newUri);
    return callback(null, {
      body: '',
      status: '301',
      statusDescription: 'Moved Permanently',
      headers: {
        location: [{
          key: 'Location',
          value: newUri
        }],
      }
    });
  } else {
    newUri = oldUri;
  }
 
  // Match any '/' that occurs at the end of a URI. Replace it with a default index
  newUri = newUri.replace(`/${appName}`, '');
 
  // Log the URI as received by CloudFront and the new URI to be used to fetch from origin
  console.log("Original URI: " + oldUri);
  console.log("Updated URI: " + newUri);
 
  // Replace the received URI with the URI that includes the index page
  request.uri = newUri;
 
  // Return to CloudFront
  return callback(null, request);
};

OriginResponse.js

'use strict';
 
const http = require('https');
 
const indexPage = 'index.html';
const appName = '{branchname}'; // TODO: Get from ENV VAR
 
exports.handler = async (event, context, callback) => {
     
    console.log(JSON.stringify(event, null, 2));
     
    const cf = event.Records[0].cf;
    const request = cf.request;
    const response = cf.response;
    const statusCode = response.status;
 
    // Only replace 403 and 404 requests typically received
    // when loading a page for a SPA that uses client-side routing
    const doReplace = request.method === 'GET'
                    && (statusCode == '403' || statusCode == '404');
 
    const result = doReplace
        ? await generateResponseAndLog(cf, request, indexPage)
        : response;
 
    callback(null, result);
};
 
async function generateResponseAndLog(cf, request, indexPage){
 
    const domain = cf.config.distributionDomainName;
    //const appPath = getAppPath(request.uri);
    //const indexPath = `/${appPath}/${indexPage}`;
    const indexPath = `/${appName}/${indexPage}`;
    console.log(`generateResponseAndLog - domain: ${domain}`);
    console.log(`generateResponseAndLog - indexPath: ${indexPath}`);
     
    const response = await generateResponse(domain, indexPath);
 
    console.log('response: ' + JSON.stringify(response));
 
    return response;
}
 
async function generateResponse(domain, path){
    try {
         
        // Load HTML index from the CloudFront cache
        const s3Response = await httpGet({ hostname: domain, path: path });
 
        const headers = s3Response.headers ||
            {
                'content-type': [{ value: 'text/html;charset=UTF-8' }]
            };
 
        return {
            status: '200',
            headers: wrapAndFilterHeaders(headers),
            body: s3Response.body
        };
    } catch (error) {
        return {
            status: '500',
            headers:{
                'content-type': [{ value: 'text/plain' }]
            },
            body: 'An error occurred loading the page'
        };
    }
}
 
function httpGet(params) {
    return new Promise((resolve, reject) => {
        http.get(params, (resp) => {
            console.log(`httpGet - Fetching ${params.hostname}${params.path}, status code : ${resp.statusCode}`);
            let result = {
                headers: resp.headers,
                body: ''
            };
            resp.on('data', (chunk) => { result.body += chunk; });
            resp.on('end', () => { resolve(result); });
        }).on('error', (err) => {
            console.log(`httpGet - Couldn't fetch ${params.hostname}${params.path} : ${err.message}`);
            reject(err, null);
        });
    });
}
 
//
 
// Cloudfront requires header values to be wrapped in an array
function wrapAndFilterHeaders(headers){
    const allowedHeaders = [
        'content-type',
        'content-length',
        'last-modified',
        'date',
        'etag',
        'transfer-encoding'
    ];
 
    const responseHeaders = {};
 
    if(!headers){
        return responseHeaders;
    }
 
    for(var propName in headers) {
        // only include allowed headers
        if(allowedHeaders.includes(propName.toLowerCase())){
            var header = headers[propName];
 
            if (Array.isArray(header)){
                // assume already 'wrapped' format
                responseHeaders[propName] = header;
            } else {
                // fix to required format
                responseHeaders[propName] = [{ value: header }];
            }   
        }
 
    }
 
    console.log(`wrapAndFilterHeaders - headers: ${JSON.stringify(responseHeaders)}`);

    return responseHeaders;
}
Personal tools
Namespaces

Variants
Actions
Navigation
Mediawiki
Confluence
DevOps Tools
Ubuntu
Oracle Linux
AWS
Windows
OpenVPN
Grafana
Pivotal
osTicket
OTRS
phpBB
WordPress
VmWare ESXI 5.1
Crypto currencies
HTML
CSS
Python
Java Script
PHP
Raspberry Pi
Canvas LMS
Kaltura Media Server
Plex Media Server
MetaSploit
Zoneminder
Photoshop CS2
Fortinet
Uploaded
Certifications
General Info
Games
Meal Plans
NC Statutes
Toolbox