How to Run a Front-End Infrastructure Team
Over the past years, AdRoll has grown from a humble startup built around a single feature to a global marketing platform with a diverse suite of products. Along with the growth of the company, we have put a lot of work into building a solid infrastructure for user interface development. In this post, we talk about the human aspects of front-end projects that are shared between multiple engineering teams.
Our current front-end infrastructure consists of UI components, a UX pattern library and various JavaScript packages for internationalization (i18n), analytics and A/B testing. We consider all these projects “internally open sourced”. This means we encourage our engineers to use contributions to these projects as a way of sharing their knowledge across team boundaries. Ultimately, we believe leveraging a wider range of ideas makes software more robust.
If one of our vertical product teams wants to propose a change to the infrastructure, they are expected to gather feedback from other teams that might be affected by it. In the simplest case, this could mean making a style in a UI component customizable through a property. In more advanced cases, this could mean building an entirely new shared library (such as an API client) to abstract out complexity from other engineering teams.
While this model allows each team to build the exact solution they need, it’s also important to have people whose sole purpose is to think about how everything fits together. Without proper ownership and oversight, shared infrastructure tends to become a complex mess of patches that feels more like a bottleneck than a helpful tool.
The Frontend Core Team
Since 2015 each new product and internal tool we have built has been implemented as a microservice. This loosely coupled architecture has let us iterate on new functionality faster but it has also fragmented our approach to UI development. Initially, a group of senior engineers was able to build a foundation for our current front-end infrastructure, but it was still unclear who should act as the first point of contact for prioritizing tasks and triaging issues.
In 2016 we formalized our approach to maintaining front-end infrastructure by founding a team we call Frontend Core. Today, this team works closely with our front-end engineers, UI designers and product leaders to make sure our web applications are built consistently across all teams. Most importantly, the Frontend Core team oversees and maintains all of our collaborative front-end projects.
The rest of this post covers the UI development guidelines set forth by our Frontend Core team in five sections. Each section explains our general approach to the topic and provides a list of practical tips for adopting similar practices in other organizations:
- Remove Bottlenecks for Iteration
- Communicate Across Team Boundaries
- Look for Common Problems and Solutions
- Lower the Barrier to Success
- Keep on Experimenting
Remove Bottlenecks for Iteration
As infrastructure projects become more complex and more teams depend on them, it becomes harder to make changes without breaking existing functionality. With this in mind, we are constantly streamlining the development workflow around our front-end infrastructure. Our ultimate goal is to make our shared projects a joy to work with so that more people will adopt them and keep making improvements.
Since the beginning of 2016, we have had 47 internal contributors make changes to our front-end infrastructure, resulting in over 500 pull requests. Each change has gone through code review and a suite of automated tests to prevent unintended side-effects. Our Frontend Core team keeps a close eye on open pull requests and makes sure they get reviewed as quickly as possible.
To measure improvements in the process, we also gather statistics on pull request lifetime over time. The chart below shows a typical bimodal distribution for code reviews: Most changes take about a week to review but there’s a subset of quick changes (such as production patches) that are deployed within an hour of opening the pull request.
Here’s how you can remove common bottlenecks in your UI development process:
- Make simple changes fast and complex ones possible. Production issues can often be mitigated with UI changes so make sure patches can be reviewed and released as quickly as possible. Encourage talking through large changes before the actual programming work.
- Consider bundling similar JavaScript packages in a monorepo. A single code repository allows packages to share build tools and test scripts with less overhead compared to completely separate projects. See Babel for a popular example and Lerna for a pre-existing toolkit.
- Use pull request templates and CODEOWNERS files for GitHub projects. PR templates should include a checklist for commonly overlooked aspects of UI development such as reusability, translatability and style consistency. Use code reviews as a way of transferring knowledge between UI teams.
- Set up staging deployments for feature branches. Having a live demo means the reviewer doesn’t always have to run the code locally. Storybook is an excellent tool for building demos for React components.
- Don’t over-optimize. Wondering how much time you should spend improving common tasks? XKCD has an answer.
Communicate Across Team Boundaries
Internal open source projects rely heavily on communication for announcing new features and breaking changes. One of the most important roles for an infrastructure team is to share context and act as a relay between contributors.
At AdRoll, the Frontend Core team holds a meeting every two weeks with engineers and designers from all of our product teams. In the meeting, teams share updates on any major front-end changes they are working on, even if they don’t directly affect our infrastructure. A recurring meeting like this has proven to be very useful for identifying surprising connections between teams.
When a team mentions they are working on a new feature, another team might mention that they already have a solution for it. In other cases, we have identified pain points in the infrastructure that everyone feels but nobody has tackled yet. These discussions often result in new tickets in our Frontend Core backlog or updates to the development process.
Conversations around front-end infrastructure can be challenging because UI development involves so many different specialties. Designers, engineers and product managers often use different words for the same things depending on their background (e.g. “dropdown” vs. “pull-down menu” vs. “select input”). It can also be difficult for a single developer to get a high level overview of all the resources available to them (e.g. global color variables and CSS classes).
To establish a common conceptual language and help with discoverability, we built a UX pattern library and released it as a resource both inside and outside the company. To complement the pattern library, we also have an internal site that lists all of our UI components and lets developers compare their functionality over multiple versions (see video below). These sites allow everyone in the company to get a complete picture of our front-end infrastructure and use explicit URLs when referring to specific UI elements.
Here are some ways you can improve communication around infrastructure:
- Set up a dedicated discussion channel for front-end infrastructure. Public Slack channels or email lists work well for this purpose. If necessary, set up a separate support channel for urgent issues.
- Communicate visually whenever possible. Screenshots, screen captures and live demos are worth a thousand chat messages.
- Use Semantic Versioning when publishing shared packages. Proper versioning helps build trust between engineering teams. Thinking in terms of breaking changes, new features and bug fixes is a helpful framework for updates that affect user-facing functionality.
- Help teams through breaking changes. In an internal open source project, you have a direct line of communication to every user. Do your best to make breaking changes less painful by supporting the teams working on upgrading components.
- Be open to feedback and suggestions. The infrastructure is your product and engineers are your users. Encourage new engineers to question technical choices to surface new ideas.
Look for Common Problems and Solutions
Even with open and active channels of communication, pain points aren’t always surfaced in an explicit way. When one of your coworkers faces an issue, they might not be aware that it’s a common problem for others. A good maintainer of front-end infrastructure is able to distill the chatter around UI development into clear problem statements and work on solutions for the most common ones.
Here are some typical messages and their implications from an infrastructure point of view:
- I wish we had…
- Since we don’t have X, we can’t…
- I’ve been meaning to build…
- Do we have support for X yet?
Messages like these can be a sign that the infrastructure doesn’t support some use cases. Reach out to the person or team to understand the context and see if they could become a contributor.
- I’m seeing this error when…
- How do you run…
- I couldn’t figure out X
- Do you know what I’m doing wrong?
When people mention errors or ask for help, it can either be a real bug or a sign that the project isn’t documented well enough. Help the person resolve the issue and see how you can update the instructions in your project afterwards.
- It was taking forever so I…
- X is too simple so I had to…
- I couldn’t use that option so…
- This is hacky but…
Software engineers take pride in working around issues. Take note when people mention hacky solutions and improve the infrastructure accordingly. Official solutions are easier to maintain over time than application specific hacks.
- Oh, I forgot to run the tests again
- Joe, could you deploy this?
- I’m waiting for Ted to…
- Do you think that’s ready to merge?
Procedural discussion around internal open source projects can be a sign that the development process isn’t as streamlined as it could be. Consider automating tests and train more people to act as maintainers.
Over time, maintainers should automate the day-to-day tasks of their projects as much as they can so they can focus on long term improvements. The tasks that can’t be automated should be well documented so they are easily repeatable. For example, when reviewing changes to our shared UI components, we always look out for changes in global namespaces – CSS classes, global SASS variables and Redux actions.
Documentation and guidance can be automated as well: Instead of referring people to static documents like wiki pages, design a development workflow that gives users enough instructions so they can solve problems on their own. React’s error code system is a great example of self-documenting software: The library itself warns developers if they are doing something wrong and points them to a helpful documentation page.
One of the most effective ways of keeping your infrastructure stable is abstracting out parts of the codebase that most contributors don’t need to modify. This means thinking of your development workflow a series of black boxes. Just like you don’t typically modify the source code of your code editor or web browser, the contributors in infrastructure projects shouldn’t need to modify the build process or the test harness.
For us, one such improvement was automating the publish process for our shared UI components. Previously publishing a new version of a component required multiple manual steps and only a few senior engineers had the credentials required to run them. The results also varied slightly depending on each publisher’s local environment. Once we moved the entire process to a Jenkins job, we could publish a component just by picking its name from a list and entering a version number. This has allowed more contributors to publish changes on their own and reduced the workload on our Frontend Core team.
Here are some of our tips for building self-documenting projects:
- Use type checking to enforce assumptions in your code. Our shared UI components make heavy use of React PropTypes. We’re also experimenting with TypeScript.
- Test and lint your code extensively. Make sure tests and linters are run automatically against every proposed change. Linters like ESLint and scss-lint can even help improve the runtime performance of your applications.
- Log instructions and next steps in command-line tools. For example, when starting a development server, make sure the local URL is shown in the console output.
- Send notifications from build systems. Contributors should be notified if they break a build or a deployment. Maintainers should be cc’d so they can spot recurring issues.
- Write actionable error messages. Don’t expect everyone to be familiar with the latest front-end tools. Error pages in internal services should give users ways to debug the issue.
Lower the Barrier to Success
Success in UI development doesn’t just mean giving end users access to new functionality, it means that new features have to seamlessly blend in with everything else in the product. Bad practices in UI development directly result in a bad user experience. This puts a lot of pressure on front-end engineers and forces them to think of shared infrastructure earlier in a company’s lifetime compared to their back-end oriented colleagues.
When an engineer builds something on top of existing infrastructure, they should feel like they’re standing on the shoulders of giants – like they can achieve something they didn’t know they could. Front-end infrastructure should enable every engineer to build beautiful, user-friendly features that can be tested, monitored and deployed easily.
In order to remain useful, infrastructure has to grow with the company. In our experience, the best way to guarantee that a project survives over time is to focus on developer engagement. This is why we believe the open source process is such a good match with software infrastructure: When people know they had a part in building a project, they are more likely to care about it (also known as the IKEA effect).
Here’s how you can make developers engaged in collaborative projects:
- Be transparent about decision making. Your fellow engineers should know why something is built the way it is and why you chose a certain technology over another.
- Help people learn more about the infrastructure and how it is used. Encourage people to read design documents and source code. Open up access to usage metrics and site analytics.
- Make sure maintainers follow the same guidelines as everyone else. Nobody should be above code reviews or testing guidelines. When exceptions have to be made, be open about the reasons behind them.
- Highlight achievements on a personal level. Celebrate wins from engineers who make their first front-end contributions. Let people show their own work in meetings through presentations and screencasts.
- Let others lead and take responsibility. Having contributors you can trust is necessary for scaling the development process. Delegate the ownership of specialized UI components to the teams that built them.
Keep on Experimenting
Almost everything we have learned about front-end infrastructure has been a result of controlled experimentation. We’re constantly trying out new promising technologies and improvements in our development process. Whenever we identify different approaches to the same problem, we try out the most promising ones and review the results afterwards.
It is good to remember that even the best infrastructure comes with some overhead. Every piece of shared code will eventually break or become outdated. An important part of the experimentation mindset is questioning the choices you have made and knowing when standardization is not the right approach.
Some of our ongoing experiments in front-end infrastructure are:
- Replacing Browserify and Gulp with Webpack as a standard front-end build process
- Integrating Redux actions into our shared React components
- Extending our public style guide
- Unifying usage analytics, exception tracking and A/B testing using a custom JavaScript library
- Standardizing integration testing with Enzyme (as presented at the ReactJS SF Meetup on July 13th)
- Building a Yeoman generator for bootstrapping web applications
- Using Storybook for the live demos in our shared UI component library
Thanks for reading! We hope this post is helpful for people who are looking to improve their UI development processes. If you would like to hear more about a specific topic or share your experiences on maintaining front-end infrastructure, don’t hesitate to post a comment below.
PS: We are hiring both remote and local engineers!