Announcing Hologram, taking EC2 Instance Roles everywhere
Written by , December 22, 2014 We love Amazon Web Services here at AdRoll. All of our production, test, and staging infrastructure runs on AWS. Really, the only systems we operate outside of EC2 are developers’ laptops. Most of the time this isn’t an issue, as we use platform-agnostic tools like Python, Java and Erlang. When it came to interacting with AWS APIs, though, we had a challenge: how do we manage sensitive AWS Keys across dozens of developer’s laptops and many teams?
Key management in EC2 is best handled using “IAM Roles” where a special endpoint in the Instance Metadata service (http://169.254.169.254/…) exposes temporary AWS API access credentials that have permissions defined by the instance’s Role, configured at launch time. This way applications can be designed that do not require checking secret keys into their repositories, reducing the chance of malicious key usage. This service only exists in EC2, though, so many developers who use AWS keep long-lived, highly privileged credentials in a file on their local development machine. This isn’t particularly secure, and it creates a difference between development and production that can lead to broken deployments.
Several months ago, a couple of us embarked on a project to bridge this gap. Today, we’re proud to announce the release of Hologram, a system for bringing EC2 Role-like key management to non-EC2 hosts. Hologram exposes an imitation of the EC2 Instance Metadata service on developer workstations that exposes temporary credentials to your software the same way that EC2 does. It behaves just like EC2, so your code can use the same process in both development and production. The keys that Hologram provisions are temporary. EC2 access is then centrally controlled without direct administrative access to developer workstations.
In the past, I’ve worked at companies that give up on local development and require developers to run all of their code, even when developing, on an EC2 instance. We could have gone down that path here at AdRoll, but then we would have lost the richness of local development: powerful IDE tooling, instant access to files, interactive graphical debugging, and no waiting for instances to come up. Instead, we’ve decided to make local development the best experience we can, and Hologram enables us to do that.
Hologram is available on GitHub under the Apache License, Version 2.0. The current implementation depends on having an LDAP server in your organization and isn’t super easy to set up, but we plan to add support for simpler authentication backends and streamline installation over the coming months. Pull Requests are, of course, welcome!