Interested in working with us? We are hiring!

See open positions

How to Run a Front-End Infrastructure Team

Jyri Tuulos Written by Jyri Tuulos, August 29, 2017

Over the past years, AdRoll has grown from a humble startup built around a single feature to a global marketing platform with a diverse suite of products. Along with the growth of the company, we have put a lot of work into building a solid infrastructure for user interface development. In this post, we talk about the human aspects of front-end projects that are shared between multiple engineering teams.

A screenshot of an AdRoll dashboard with analytics, i18n, CSS and UI components visualized as layers
Every web application at AdRoll relies on multiple layers of front-end infrastructure

Our current front-end infrastructure consists of UI components, a UX pattern library and various JavaScript packages for internationalization (i18n), analytics and A/B testing. We consider all these projects “internally open sourced”. This means we encourage our engineers to use contributions to these projects as a way of sharing their knowledge across team boundaries. Ultimately, we believe leveraging a wider range of ideas makes software more robust.

If one of our vertical product teams wants to propose a change to the infrastructure, they are expected to gather feedback from other teams that might be affected by it. In the simplest case, this could mean making a style in a UI component customizable through a property. In more advanced cases, this could mean building an entirely new shared library (such as an API client) to abstract out complexity from other engineering teams.

While this model allows each team to build the exact solution they need, it’s also important to have people whose sole purpose is to think about how everything fits together. Without proper ownership and oversight, shared infrastructure tends to become a complex mess of patches that feels more like a bottleneck than a helpful tool.

The Frontend Core Team

Since 2015 each new product and internal tool we have built has been implemented as a microservice. This loosely coupled architecture has let us iterate on new functionality faster but it has also fragmented our approach to UI development. Initially, a group of senior engineers was able to build a foundation for our current front-end infrastructure, but it was still unclear who should act as the first point of contact for prioritizing tasks and triaging issues.

In 2016 we formalized our approach to maintaining front-end infrastructure by founding a team we call Frontend Core. Today, this team works closely with our front-end engineers, UI designers and product leaders to make sure our web applications are built consistently across all teams. Most importantly, the Frontend Core team oversees and maintains all of our collaborative front-end projects.

The rest of this post covers the UI development guidelines set forth by our Frontend Core team in five sections. Each section explains our general approach to the topic and provides a list of practical tips for adopting similar practices in other organizations:

Remove Bottlenecks for Iteration

As infrastructure projects become more complex and more teams depend on them, it becomes harder to make changes without breaking existing functionality. With this in mind, we are constantly streamlining the development workflow around our front-end infrastructure. Our ultimate goal is to make our shared projects a joy to work with so that more people will adopt them and keep making improvements.

Since the beginning of 2016, we have had 47 internal contributors make changes to our front-end infrastructure, resulting in over 500 pull requests. Each change has gone through code review and a suite of automated tests to prevent unintended side-effects. Our Frontend Core team keeps a close eye on open pull requests and makes sure they get reviewed as quickly as possible.

To measure improvements in the process, we also gather statistics on pull request lifetime over time. The chart below shows a typical bimodal distribution for code reviews: Most changes take about a week to review but there’s a subset of quick changes (such as production patches) that are deployed within an hour of opening the pull request.

A bar chart of pull request lifetime
To get charts like this for your GitHub project, we recommend checking out this open source tool

Here’s how you can remove common bottlenecks in your UI development process:

Communicate Across Team Boundaries

Internal open source projects rely heavily on communication for announcing new features and breaking changes. One of the most important roles for an infrastructure team is to share context and act as a relay between contributors.

At AdRoll, the Frontend Core team holds a meeting every two weeks with engineers and designers from all of our product teams. In the meeting, teams share updates on any major front-end changes they are working on, even if they don’t directly affect our infrastructure. A recurring meeting like this has proven to be very useful for identifying surprising connections between teams.

When a team mentions they are working on a new feature, another team might mention that they already have a solution for it. In other cases, we have identified pain points in the infrastructure that everyone feels but nobody has tackled yet. These discussions often result in new tickets in our Frontend Core backlog or updates to the development process.

Conversations around front-end infrastructure can be challenging because UI development involves so many different specialties. Designers, engineers and product managers often use different words for the same things depending on their background (e.g. “dropdown” vs. “pull-down menu” vs. “select input”). It can also be difficult for a single developer to get a high level overview of all the resources available to them (e.g. global color variables and CSS classes).

To establish a common conceptual language and help with discoverability, we built a UX pattern library and released it as a resource both inside and outside the company. To complement the pattern library, we also have an internal site that lists all of our UI components and lets developers compare their functionality over multiple versions (see video below). These sites allow everyone in the company to get a complete picture of our front-end infrastructure and use explicit URLs when referring to specific UI elements.

Our internal site for demoing shared UI components, built using the AWS JavaScript S3 Explorer

Here are some ways you can improve communication around infrastructure:

Look for Common Problems and Solutions

Even with open and active channels of communication, pain points aren’t always surfaced in an explicit way. When one of your coworkers faces an issue, they might not be aware that it’s a common problem for others. A good maintainer of front-end infrastructure is able to distill the chatter around UI development into clear problem statements and work on solutions for the most common ones.

Here are some typical messages and their implications from an infrastructure point of view:

  • I wish we had…
  • Since we don’t have X, we can’t…
  • I’ve been meaning to build…
  • Do we have support for X yet?

Messages like these can be a sign that the infrastructure doesn’t support some use cases. Reach out to the person or team to understand the context and see if they could become a contributor.

  • I’m seeing this error when…
  • How do you run…
  • I couldn’t figure out X
  • Do you know what I’m doing wrong?

When people mention errors or ask for help, it can either be a real bug or a sign that the project isn’t documented well enough. Help the person resolve the issue and see how you can update the instructions in your project afterwards.

  • It was taking forever so I…
  • X is too simple so I had to…
  • I couldn’t use that option so…
  • This is hacky but…

Software engineers take pride in working around issues. Take note when people mention hacky solutions and improve the infrastructure accordingly. Official solutions are easier to maintain over time than application specific hacks.

  • Oh, I forgot to run the tests again
  • Joe, could you deploy this?
  • I’m waiting for Ted to…
  • Do you think that’s ready to merge?

Procedural discussion around internal open source projects can be a sign that the development process isn’t as streamlined as it could be. Consider automating tests and train more people to act as maintainers.

Over time, maintainers should automate the day-to-day tasks of their projects as much as they can so they can focus on long term improvements. The tasks that can’t be automated should be well documented so they are easily repeatable. For example, when reviewing changes to our shared UI components, we always look out for changes in global namespaces – CSS classes, global SASS variables and Redux actions.

Documentation and guidance can be automated as well: Instead of referring people to static documents like wiki pages, design a development workflow that gives users enough instructions so they can solve problems on their own. React’s error code system is a great example of self-documenting software: The library itself warns developers if they are doing something wrong and points them to a helpful documentation page.

One of the most effective ways of keeping your infrastructure stable is abstracting out parts of the codebase that most contributors don’t need to modify. This means thinking of your development workflow a series of black boxes. Just like you don’t typically modify the source code of your code editor or web browser, the contributors in infrastructure projects shouldn’t need to modify the build process or the test harness.

For us, one such improvement was automating the publish process for our shared UI components. Previously publishing a new version of a component required multiple manual steps and only a few senior engineers had the credentials required to run them. The results also varied slightly depending on each publisher’s local environment. Once we moved the entire process to a Jenkins job, we could publish a component just by picking its name from a list and entering a version number. This has allowed more contributors to publish changes on their own and reduced the workload on our Frontend Core team.

A screenshot from Slack showing notifications from Jenkins and GitHub
Slack notifications from the Jenkins job that publishes our shared UI components

Here are some of our tips for building self-documenting projects:

Lower the Barrier to Success

Success in UI development doesn’t just mean giving end users access to new functionality, it means that new features have to seamlessly blend in with everything else in the product. Bad practices in UI development directly result in a bad user experience. This puts a lot of pressure on front-end engineers and forces them to think of shared infrastructure earlier in a company’s lifetime compared to their back-end oriented colleagues.

When an engineer builds something on top of existing infrastructure, they should feel like they’re standing on the shoulders of giants – like they can achieve something they didn’t know they could. Front-end infrastructure should enable every engineer to build beautiful, user-friendly features that can be tested, monitored and deployed easily.

In order to remain useful, infrastructure has to grow with the company. In our experience, the best way to guarantee that a project survives over time is to focus on developer engagement. This is why we believe the open source process is such a good match with software infrastructure: When people know they had a part in building a project, they are more likely to care about it (also known as the IKEA effect).

An excerpt of IKEA's assembly instructions
IKEA helps people succeed with their famous assembly instructions

Here’s how you can make developers engaged in collaborative projects:

Keep on Experimenting

Almost everything we have learned about front-end infrastructure has been a result of controlled experimentation. We’re constantly trying out new promising technologies and improvements in our development process. Whenever we identify different approaches to the same problem, we try out the most promising ones and review the results afterwards.

It is good to remember that even the best infrastructure comes with some overhead. Every piece of shared code will eventually break or become outdated. An important part of the experimentation mindset is questioning the choices you have made and knowing when standardization is not the right approach.

XKCD comic #1319 with 'automation' changed to 'infrastructure'
Adapted from XKCD #1319 "Automation"

Some of our ongoing experiments in front-end infrastructure are:

Thanks for reading! We hope this post is helpful for people who are looking to improve their UI development processes. If you would like to hear more about a specific topic or share your experiences on maintaining front-end infrastructure, don’t hesitate to post a comment below.

PS: We are hiring both remote and local engineers!