Two tips for writing CloudFormation templates

Posted on by milancermak

Here are two tips for writing more readable CloudFormation templates.

1) Use dot notation to access attributes

The Fn::GetAtt intrinsic function supports the use of dot notation to refer to a resource’s attribute. It works both the long Fn::GetAtt and short !GetAtt forms of the syntax. Instead of the long, enumerated, YAML array syntax:

Value:
  Fn::GetAtt:
    - MyResource
    - Arn

Use the one-liner version:

Value: !GetAtt MyResource.Arn

2) Use !Sub instead of !Join

When creating a string, using !Sub is often a much better option than using !Join. Here’s an example of building an ECR repository URI:

Value:
  Fn::Join:
    - '.'
    - - !Ref 'AWS::AccountID'
      - dkr
      - ecr
      - !Ref 'AWS::Region'
      - amazonaws.com
      - '/'
      - !Ref 'ServiceName'

With this approach, it’s hard to grasp what the string looks like, it’s hard to write correctly and it’s hard to debug. Now compare it to using !Sub:

Value: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ServiceName}'

A cool thing to notice is the explicit !Ref on pseudo-parameters and template parameters. This makes the whole construction so much nicer. If you need to access an attribute, you can use the dot notation as mentioned above.

I hope you find these tips useful and apply them in your practice.

DynamoDB, NodeJS vs. Python and persistent connections

Posted on by milancermak

Recently, Yan Cui wrote an enlightening blogpost about using keep-alive HTTP connections to significantly speed up DynamoDB operations. He gave an example of how to do it in NodeJS. I was curious how to do it in Python.

To my surprise, I found out I did not have to do anything at all. DynamoDB keeps the connection open. See for yourself – using the CLI, run aws dynamodb list-tables --debug. Notice the response headers section, which looks something like this:

 Response headers:
 {'Server': 'Server', 
  'Date': 'Thu, 07 Mar 2019 19:42:55 GMT', 
  'Content-Type': 'application/x-amz-json-1.0', 
  'Content-Length': '328', 
  'Connection': 'keep-alive', 
  'x-amzn-RequestId': '38N9IJV176MACH027DNIRT5C53VV4KQNSO5AEMVJF66Q9ASUAAJG', 
  'x-amz-crc32': '2150813651'}

The Connection: keep-alive header is set by DynamoDB. Unless it’s explicitly set to close, the connection will stay open. Yet this is exactly what NodeJS does. Thank you to Stefano Buliani for providing additional visibility into this. This behaviour is inherited by the aws-js-sdk. I think that’s a mistake so I’ve opened a bug in the GitHub repo. Until then, if you’re writing code in JS, be sure to follow Yan’s recommendation.

Connection: keep-alive vs. close in Python

I was still curious if  I could replicate Yan’s findings in Python. Here’s a log of running a single putItem operation using vanilla boto3 DynamoDB client:

boto3-dynamodb-default

Except for the first one, most of them are sub 10 ms, since the connection is kept open.

However, when I explicitly did add the Connection: close header, things looked a lot different:

boto3-dynamodb-connection-close

Operations took at least 50 ms, often longer. This is in line with Yan’s findings.

Granted, my approach was not very rigorous. For the sake of replicability, here’s the code I used. Feel free to run your own experiments and let me know what you found.