Ship it! with ecs-ship

Written by Oscar Arbeláez, February 23, 2021

Having a fast and reliable way to update your deployment environments is a game changer in terms of team velocity and confidence. ecs-ship is a simple yet powerful tool that can help your team ship faster.

10 minute read

Introduction

Most of our teams at NextRoll leverage the power of AWS ECS for container orchestration. Each team manages their AWS resources, providing them with ultimate freedom when choosing technologies or what or when to deploy. In this article, we want to share ecs-ship, a small tool the likes of ecs-deploy that we wrote to solve some of our common workflows and will soon be open source (Update: it’s now opensource yay!!)

Some typical workflows

Let’s see by example which workflows you might want to use ecs-ship for by looking at two typical tasks we can accomplish with it:

Deploying a new version of an application.
Changing the resource allocation of an application (aka right-sizing).

Deploying with `ecs-ship`

When deploying a new version of a containerized application to AWS ECS, there are three resources you need to account for: container images, ECS task definitions, and ECS services. Container images are a pretty common artifact these days. They represent a “unit” containing your application and all the dependencies it might need to run correctly. ECS Task definitions represent a containerized workload; essentially, they let ECS know what’s required to run your tasks. For example, I need three containers running my application image and 1 running a database image. You express these requirements via container definitions inside the task definition. Finally, ECS services reference a version of a task definition and are responsible for letting AWS know “we do need these containers running now”.

In this framework, to deploy a new version of your application, you need to follow these steps:

Create and push a new version of your application’s image to a registry.
Create a new version of your task definition that references the freshly uploaded image.
Finally, update your service to the newly created task definition.

Let’s go through them step by step.

Building and pushing your images to a registry would usually look like this:

export VERSION=`git rev-parse HEAD`
docker build -t your.registry.io/yourusername/yourapp:$VERSION .
docker push your.registry.io/yourusername/yourapp:$VERSION

Once you have your new image (or images) up on the registry and ready to use, ecs-ship comes in to help you with the rest:

cat << EOF | ecs-ship your-cluster yourapp-service
containerDefinitions:
  yourapp-definition:
    image: your.registry.io/yourusername/yourapp:$VERSION
    environment:
      VERSION: $VERSION
      SHIPPED_BY: ecs-ship
EOF

As you can see, ecs-ship takes the cluster you want to update and the service you want to update in that cluster. Furthermore, it takes a configuration file to non-destructively modify some aspects of the chosen service, including the container definitions. Here inside the container definition, we have decided to update the image to refer to the new VERSION and upserted a couple of environment variables to later check on the task definition. You can check the README on ecs-ship’s repo to get familiar with which aspects of ecs services you can change.

Running this command will perform these steps for you:

Pull the task definition of the specified cluster and service.
Create a copy of this task definition performing the updates you requested on the input file (only if anything is updated).
Update the specified service to point to the newly created version of the task definition.
Wait for your service to reflect the changes you requested correctly.

If for some reason, your service doesn’t reach stability with your new configuration, ecs-ship will automatically:

Revert to the last task definition (if it was stable).
De-register the new bugged task definition.

Right-sizing your services

We just saw how ecs-ship would help you in deploying new versions of ECS services. Now let’s explore another simple yet powerful ecs-ship that can help you change the number of resources a particular service requests from your ECS cluster.

One essential part of your ECS service configuration is the amount of resources a given container or a given service requires. These resources are controlled by the cpu, memory, and memoryReservation settings. cpu represents the amount of “shares” of a virtual CPU your service needs. Simultaneously, memory and memoryReservation are hard and soft caps to the amount of memory your service can use, respectively. These variables cost you money in the following sense: if the sum of either the memory or the cpu required by the services running in your cluster surpasses the number of actual resources in the cluster, you need to spin up a new instance incurring in some new costs.

However, not always do your services use as many resources as you reserve for them, and since you’re charged for the amount of resources you do request, it is a good idea to keep those as close as possible to the actual usage values. To work out the amount of resources your ECS services need, you can enable and monitor CloudWatch metrics, and once you have worked out the number of resources you need, you can update the configuration using ecs-ship:

cat << EOF | ecs-ship your-cluster yourapp-service
containerDefinitions:
  yourapp-definition:
    cpu: 123
    memoryReservation: 123
EOF

As a side note, if you specify memory, you need to make sure the value is comfortably higher than memoryReservation.

How `ecs-ship` came to be

Those familiar with fabfuel’s ecs-deploy will notice that there’s no much difference between it and ecs-ship. ecs-ship can be seen as a new take on the already popular project, with just a few different opinions. ecs-ship focuses only on updating a given service with a new version of its task definition, leaving out other add-on features like slack reporting or newrelic metrics for deployments.

In terms of new features, ecs-ship can change resource allocations and could be extended to change other aspects of a task definition. Furthermore, and we will see an application of this on a further post, ecs-ship’s input file-based API makes it easier to interoperate with other third-party or homebrewed tools.

Besides these minor differences, one of the primary motivations for building ecs-ship was our bi-annual HackWeek events. These events give us some time to explore technologies, work on a little side-project or solve issues in our workflow that are not directly related to our backlog. In our case, with ecs-ship, we experimented with golang (our team working mostly with Python and JavaScript) and solved the need to update resource allocations for our services.

Conclusion

ecs-ship is a little tool you can use either as a static linked binary or as a containerized tool to manage your ECS deployments from your local machine or your continuous delivery pipelines.

Update (2021-04-08)

As of yesterday ecs-ship is now open source! You can grab the source code, binaries and docker images following the instructions at https://github.com/AdRoll/ecs-ship.