Interested in working with us? We are hiring!

See open positions

Removing Erlang dead code with Xref

Brujo Benavides Written by Brujo Benavides, October 09, 2018

Dead code (as in functions that are not used anywhere) tends to pile up in big projects if you leave them unattended. Using one of Xref’s most underrated features, you will be able to detect and remove what you do not need anymore.

10-15 minute read

We have already written several articles in this blog explaining how we use Erlang/OTP extensively to build our real-time bidding platform servers, among other things.

These systems are big and they have been around for a long time by now. Just like any big old system, they contain some pieces of code that are not used anymore. To be clear: they are not broken, they are even properly covered by tests and all, but they’re not in use in production.

In Erlang these pieces of dead code manifest themselves as unused functions. To be precise: they are unused *exports* since unused but not-exported functions are detected at compile time.

Finding unused exports in a big system can be tough. Luckily, Erlang/OTP already gives us a tool to do just that: Xref.

Xref is a cross reference tool that can be used for finding dependencies between functions, modules, applications and releases.

If you manage your projects with rebar3, you can use Xref by simply running the following command:

$ rebar3 xref

If you had not specified anything about Xref in your rebar.config, that will check your entire project and perform all possible checks. Since, for big projects, the list of warnings that generates tends to be long, people usually have something like this in their configuration:

{xref_checks, [
    undefined_function_calls,
    locals_not_used,
    deprecated_function_calls
]}.

In other words, with that list the report will only include:

You can find the full list of available checks in rebar3 docs, but I want you to notice that since the last 2 are also detected by the compiler (if you have the proper warnings enabled) the only effective check that’s being performed is undefined_function_calls. It’s a fine check to run, but it won’t help us with our original dead code issue.

Let’s take a look at the checks we’re not performing then. In general, undefined_functions will report the same results as undefined_function_calls but without the reference to the actual function call (Not very useful). Same goes for deprecated_functions and deprecated_function_calls. But then, we have exports_not_used which is exactly the check we’re looking for.

Adding exports_not_used to our list of checks will emit a warning for each function that we have exported but not used anywhere. It’s amazing.

But then why is nobody using it?🤔

There are a few caveats when using exports_not_used. I’ll list them now and I’ll tell you how to solve or at least work around them.

Dynamically Evaluated Functions

Xref will report a warning for each function that is exported for which it could not find where it’s used in the code. But, the fact that Xref couldn’t find it, doesn’t mean that a place where the function is actually used doesn’t actually exist. For instance, Xref can’t deal with dynamic function calls but they’re perfectly valid. So, let’s say you have a module that looks like the following one (don’t ask me why):

-module(sample).
-exports([some_function/1, some_other_function/1]).

some_function(M) ->
    M:some_other_function(an_argument).

some_other_function(X) -> {called, X}.

And some other module where you call sample:some_function(sample). Xref will not be smart enough to detect that sample:some_other_function/1 is actually used, because it’s only used through dynamic evaluation. And the one above is just one way of performing dynamic evaluation. You can see some others here:

% Classic dynamic evaluation
Module:Function(Argument, Argument2),

% Using erlang:apply/3
erlang:apply(Module, Function, Arguments),

% Using spawn[_link]/3
erlang:spawn(Module, Function, Arguments),

% Using timer:tc/3
timer:tc(Module, Function, Arguments),

% In a supervisor spec
{ChildName, {Module, Function, Arguments}, permanent, 5000, worker, dynamic},

Side note: if you add {xref_warnings, true}. to your rebar.config file, Xref will at least print out warnings for these dynamic calls that it could not parse. Like this one:

sample: 1 unresolved call

In any case, as soon as your system becomes slightly bigger than a prototype, you’ll start having this unused exports everywhere. But don’t panic, there is actually a way to avoid those warnings, and it comes with some extra benefits. You can use ignore_xref.

-ignore_xref is an attribute you can add to your modules to prevent Xref from emitting warnings about certain functions. It looks like this:

-module(sample).
-exports([some_function/1, some_other_function/1]).

%% This function should be dynamically invoked through sample:some_function/1
-ignore_xref([{?MODULE, some_other_function, 1}]).

...

Now, if you go check the Xref docs from OTP, you won’t find any mention to it. That’s because it’s not an official attribute. ignore_xref is an undocumented feature of rebar3 xref (and also xref_runner). It’s an attribute you can add to your modules where you can list all the functions you don’t want Xref to complain about. The syntax is as follows:

-ignore_xref([{module(), function(), arity()} | {module(), function()}]).

Using this, you can effectively remove all warnings related to functions that are exported so they can be dynamically evaluated. As a bonus, as you can see in our example above, you can also use this place in the code to add some documentation stating where these functions are expected to be used.

Dynamically Generated Code

You might not be a fan of dynamically generated code, but sometimes there is no way around it.

For instance, we use protocol buffers in several places and that lead us to use gpb and its rebar3 plugin. When gpb is writing a module, it doesn’t know how it will be used so it has no way to tell if certain functions should not be exported/are not needed. That means when we run Xref we get warnings about all the unused exports generated by it.

How to avoid those warnings? We can’t use -ignore_xref since we’re not writing those modules. Turns out, there is another way. We can use the dyslexically named xref_ignores attribute in rebar.config. It basically allows you to have a global list of functions to ignore everywhere. It looks like this:

{xref_ignores, [
    {my_gpb_generated_module, some_function, 1},
    {my_gpb_generated_module, some_other_function, 0},
    {my_gpb_generated_module, a_function_with_various_arities},
    ...
]}.

There is no way yet to ignore all the functions in a module, but I already wrote a ticket requesting it. Maybe you can tackle that as a #hacktoberfest project?

Functions Exported to be used Externally

What if you have some functions that are only exported because you use them in the shell when you log in remotely to your servers in production? What if they’re used by external scripts that execute rpc calls into your nodes or stuff like that?

Well, in that case, I encourage you to use -ignore_xref and add a proper comment there stating how/when/where those functions are expected to be used. It will pay off in the future, I promise.

Functions Exported just for Tests

A different situation I’ve seen sometimes (particularly when people work with Legacy Code) are functions that are exported so they can be used in tests. The idea is to either mock them or to have access to some internal logic that should otherwise be hidden to the system in production.

First of all, if you’re using eunit you don’t need to export them. You can use not-exported functions in your tests.

Now, if you use common test or other frameworks that require tests to be written outside the module under test, that’s a different story. I think it’s important to consider that exporting functions just to use them in tests is something that’s undesirable in general since…

  • If your functions are not exported and unused they are detected by the compiler, as we stated before, allowing you to find bugs much earlier.
  • If you are adding functions that should not be available in production and/or doing stuff that should not be done in production, then your test is not simulating the real scenario accurately which may lead to test passing with code that still doesn’t work as expected.

Still, sometimes there is just no way around it: You need to mock some stuff that is otherwise invisible to the external world, you need to verify some data that’s only exposed in very complex formats or detect side effects that are really hard to capture. In those scenarios, once again, ignore_xref and a neat comment are wonderful tools to avoid surprises and frustration for future developers that, finding an unused function, decide to remove it.

Library Facades

Finally, there is one other scenario where you do need to export functions that you don’t actually use within your application: When your application is a library (i.e. when you’re building an app to be used as a dependency in other systems). In that case, some functions constitute the facade of your app and they’re not to be used by it. They’re exposed so your users can invoke them in their apps.

Those functions will all be reported as unused exports and it’s not nice to have to write ignore_xref / xref_ignores for all of them. But what is really nice is to cover them in tests. And if you do that, you have a way to avoid the warnings and actually only generate warnings for functions that are exported, unused and not tested. You can run xref as…

$ rebar3 as test xref

Using the test profile, rebar3 will include all your test modules into the analysis and, since your facade functions will be used there, it will not warn you about them.

Conclusion

While Xref is a powerful tool, it needs some tweaks to extract its full potential.

First of all, you have to use the right checks. Our recommended list is…

{xref_checks, [
    undefined_function_calls,
    exports_not_used
]}.

Then you have to use -ignore_xref attributes and xref_ignores configuration param appropriately to identify all the functions that are intentionally exported and unused. If you’re writing a library, you should also consider your tests in the analysis using rebar3 as test xref.

With all that in place, you should expect to have 0 warnings reported and therefore you can be sure that you don’t have any dead code in your project.

Well… actually… You don’t have any dead functions (unused exports). Which is a lot, but you can still have dead code in the form of unused function clauses, unused case clauses, etc… Xref will not detect those problems.

For that, you need a much more powerful tool: dialyzer. We won’t cover it in this article, but stay tuned…

Do you enjoy building high-quality large-scale systems? Roll with Us!