This add-on is operated by Mixable, Inc.

Use AWS DynamoDB from your Heroku Application

Last updated December 22, 2018

Autodyne is an add-on for using AWS DynamoDB from your Heroku app.

Adding DynamoDB to an application lets you store and retrieve documents with higher performance, throughput, and availability than traditional SQL databases.

DynamoDB is a “serverless” database, offering multi-master and multi-region data access with no servers to provision, update, or operate.

It offers performance at scale, providing consistent single-digit millisecond response times at any scale.

It offers on-demand capacity that instantly ramps up or down to handle dynamic application workloads.

It offers transactions, giving you atomicity, consistency, isolation, and durability (ACID) guarantees across one or more tables.

Autodyne is accessible via an API and has supported client libraries for applications written in any Heroku-supported language.

Provisioning the add-on

Autodyne can be attached to a Heroku application via the CLI:

$ heroku addons:create autodyne
Creating autodyne:test on ⬢ myapp... free
Successfully created autodyne-sinuous-68626 table
Created autodyne-sinuous-68626 as AUTODYNE_AWS_ACCESS_KEY_ID, AUTODYNE_AWS_DEFAULT_REGION, AUTODYNE_AWS_SECRET_ACCESS_KEY, AUTODYNE_TABLE_NAME
Use heroku addons:docs autodyne to view documentation

After you provision Autodyne, the AUTODYNE_AWS_ACCESS_KEY_ID, AUTODYNE_AWS_SECRET_ACCESS_KEY, and AUTODYNE_TABLE_NAME config vars are available in your app’s configuration. They contain access tokens and the table name to use for the AWS DynamoDB PutItem, GetItem, etc. APIs. You can confirm this via the heroku config command:

$ heroku config
=== damp-waters-72648 Config Vars
AUTODYNE_AWS_ACCESS_KEY_ID:     AKIAJNN2M647OGACKWGA
AUTODYNE_AWS_DEFAULT_REGION:    us-east-1
AUTODYNE_AWS_SECRET_ACCESS_KEY: PuWYvm9DeMCGwho7mT0AkieYUkUziI/lC07W1+L5
AUTODYNE_TABLE_NAME:            autodyne-sinuous-68626

After you install Autodyne, your application should be configured to fully integrate with the add-on.

Local setup

Environment setup

After you provision the add-on, it’s necessary to locally replicate its config vars so your development environment can operate against the service.

Use the Heroku Local command-line tool to configure, run, and manage process types specified in your app’s Procfile. Heroku Local reads configuration variables from a .env file. To view all of your app’s config vars, type heroku config. Use the following command for each value that you want to add to your .env file:

$ heroku config:get AUTODYNE_AWS_ACCESS_KEY_ID     -s  >> .env
$ heroku config:get AUTODYNE_AWS_DEFAULT_REGION    -s  >> .env
$ heroku config:get AUTODYNE_AWS_SECRET_ACCESS_KEY -s  >> .env
$ heroku config:get AUTODYNE_TABLE_NAME            -s  >> .env

For more information, see the Heroku Local article.

Using with the AWS CLI

Autodyne can be used from your local development machine via the aws CLI. Typically this entails installing the aws CLI, and configuring it with AUTODYNE_AWS_ACCESS_KEY_ID, etc.

To manage indexes with the aws dynamodb update-table command, install the execute-api AWS CLI plugin too.

# install with awscli bundled pip on OS X / Homebrew
$ /usr/local/opt/awscli/libexec/bin/pip install awscli-plugin-execute-api

# install with system pip on Windows / Linux
$ pip install awscli-plugin-execute-api
Successfully installed awscli-plugin-execute-api-0.2.0

Then register the plugin:

$ aws configure set plugins.execute-api awscli_plugin_execute_api

Next, configure a new profile for the addon credentials:

$ aws configure --profile autodyne
AWS Access Key ID [None]: AKIAJNN2M647OGACKWGA
AWS Secret Access Key [None]: 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
Default region name [None]: us-east-1
Default output format [None]: json

Activate the profile:

$ export AWS_PROFILE=autodyne

And configure the aws dynamodb.update-table command to use the plugin:

$ aws configure set dynamodb.update-table https://api.autodyne.mixable.net/dynamodb

Now you can describe table properties:

$ aws dynamodb describe-table --table-name $TABLE_NAME
{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "id",
                "AttributeType": "S"
            }
        ],
        "TableName": "autodyne-sinuous-68626",
        "KeySchema": [
            {
                "AttributeName": "id",
                "KeyType": "HASH"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": 1544399550.762,
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0,
            "ReadCapacityUnits": 0,
            "WriteCapacityUnits": 0
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:us-east-1:505718836660:table/autodyne-sinuous-68626",
        "TableId": "947270a7-b6e3-4a60-80bc-f3b315ade050"
    }
}

Put and get items:

$ cat <<EOF >item.json
{
    "id": {"S": "424b0514-be1d-4ee3-9d91-7e122718d7eb"},
    "Artist": {"S": "No One You Know"},
    "SongTitle": {"S": "Call Me Today"},
    "AlbumTitle": {"S": "Somewhat Famous"}
}
EOF

$ aws dynamodb put-item --table-name $TABLE_NAME --item file://item.json --return-consumed-capacity TOTAL
{
    "ConsumedCapacity": {
        "TableName": "autodyne-sinuous-68626",
        "CapacityUnits": 1.0
    }
}

$ cat <<EOF >key.json
{
    "id": {"S": "424b0514-be1d-4ee3-9d91-7e122718d7eb"}
}
EOF

$ aws dynamodb get-item --table-name $TABLE_NAME --key file://key.json
{
    "Item": {
        "id": {
            "S": "424b0514-be1d-4ee3-9d91-7e122718d7eb"
        },
        "AlbumTitle": {
            "S": "Somewhat Famous"
        },
        "Artist": {
            "S": "No One You Know"
        },
        "SongTitle": {
            "S": "Call Me Today"
        }
    }
}

And manage secondary indexes:

$ cat <<EOF >update.json
{
  "AttributeDefinitions": [
    {
      "AttributeName": "uuid",
      "AttributeType": "S"
    }
  ],
  "GlobalSecondaryIndexUpdates": [
    {
      "Create": {
        "IndexName": "uuid",
        "KeySchema": [
          {
            "AttributeName": "uuid",
            "KeyType": "HASH"
          }
        ],
        "Projection": {
          "ProjectionType": "ALL"
        }
      }
    }
  ]
}
EOF

$ aws dynamodb update-table --table-name $TABLE_NAME --cli-input-json file://update.json

Using with Rails

Ruby on Rails applications need to add the following entry to their Gemfile to include the Autodyne AWS client library:

gem 'aws-sdk-dynamodb'

Update application dependencies with bundler:

$ bundle install

Use the dynamodb client:

require 'aws-sdk-dynamodb'

client = Aws::DynamoDB::Client.new(
  credentials: Aws::Credentials.new(
    ENV['AUTODYNE_AWS_ACCESS_KEY_ID'],
    ENV['AUTODYNE_AWS_SECRET_ACCESS_KEY'],
  ),
  region: ENV['AUTODYNE_AWS_DEFAULT_REGION'],
)

begin
  client.put_item({
    table_name: ENV['AUTODYNE_TABLE_NAME'],
    item: {
      id: "424b0514-be1d-4ee3-9d91-7e122718d7eb",
      year: 2015,
      title: 'The Big New Movie',
      info: {
          plot: 'Nothing happens at all.',
          rating: 0
      }
    }
  })

  result = client.get_item({
    table_name: ENV['AUTODYNE_TABLE_NAME'],
    key: {
      id: "424b0514-be1d-4ee3-9d91-7e122718d7eb",
    }
  })

  puts result.item
  # {"year"=>0.2015e4, "info"=>{"rating"=>0.0, "plot"=>"Nothing happens at all."}, "id"=>"424b0514-be1d-4ee3-9d91-7e122718d7eb", "title"=>"The Big New Movie"}
rescue  Aws::DynamoDB::Errors::ServiceError => error
  puts 'Unable to put/get item:'
  puts error.message
end

Dashboard

The Autodyne dashboard allows you to see on-demand capacity usage.

You can access the dashboard via the CLI:

$ heroku addons:open autodyne
Opening autodyne for damp-waters-72648

or by visiting the Heroku Dashboard and selecting the application in question. Select Autodyne from the Add-ons menu.

On-demand capacity and hard limits

Autodyne uses DynamoDB On-Demand Capacity.

In this mode, DynamoDB instantly accommodates your workloads as they ramp up or down. This mode offers pay-per-request pricing for read and write requests so that you pay only for what you use.

Write operations are 5x more expensive than read operations.

If your application reads 10x more than writes:

Migrating between plans

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade autodyne:small
-----> Upgrading aytodyne:small to damp-waters-72648... done, v18 ($50/mo)
       Your plan has been updated to: aytodyne:small

Removing the add-on

You can remove Autodyne via the CLI:

$ heroku addons:destroy autodyne
-----> Removing autodyne from damp-waters-72648... done, v20 (free)

Before removing Autodyne, you can export your data by using the dynamodump utility.

Support

All Autodyne support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at support@mixable.net.

Troubleshooting

AccessDeniedException

To prevent sensitive operations, your addon IAM user does not have dynamodb:UpdateTable permissions. You will get an AccessDeniedException error updating a table if you use the AWS DynamoDB API directly:

$ aws dynamodb update-table --table-name $TABLE_NAME --cli-input-json file://update.json

An error occurred (AccessDeniedException) when calling the UpdateTable operation: User: arn:aws:iam::505718836660:user/57688518-f6b4-4942-a0d9-b6ecae8a9eeb is not authorized to perform: dynamodb:UpdateTable on resource: arn:aws:dynamodb:us-east-1:505718836660:table/autodyne-infinite-12346

A restricted update table API is available at https://api.autodyne.mixable.net/dynamodb. See the Autodyne Postman collection for documentation.

For convenience, the execute-api AWS CLI plugin configures the CLI to always use this custom API endpoint. When configured correctly, your ~/.aws/config file has the plugin and dynamodb.update-table endpoint:

[plugins]
execute-api = awscli_plugin_execute_api

[profile autodyne]
region = us-east-1
output = json
dynamodb =
    update-table = https://api.autodyne.mixable.net/dynamodb

See the execute-api AWS CLI plugin for more instructions.