OTP releases by hand

Although rebar3, reltool, & relx are here to save us pain when it comes to building OTP releases, after reading up on the task, it became crystal clear that it’s important to know the lay of the land when it comes to shipping our code. After all, OTP itself is a manifestation of the release ecosystem; it’s probably best that Erlang hackers have a decent grasp of the layout & working parts of this corner of OTP. And what a better way to understand this than by creating some releases by hand for a change?

Our application

We will begin this journey by creating an application called ‘cache.’ Instead of using a storage technique like ‘ets’ or ‘mnesia’ to hold key/values, we don’t care if our data is lost so we will hold the values in the loop-data of dynamically generated ‘gen_server’ processes. A mapping of each process to a key will be stored using the global registry, and the value will be stored in each server’s loop state.

Let’s begin by creating the basic OTP application foundations. First, let’s manually create the files/directories we need:

mkdir -p cache/{src,ebin}
cd cache/src
touch cache.app.src cache_app.erl cache_sup.erl cache_srv.erl
cd ..

Respective to our new files, we edit them so they have some boilerplate OTP-compliant Erlang code:

cache.app.src :

{application,cache,[
  {description,"Dummy cache app."},
  {vsn,"1986.a"},
  {registered,[cache_sup]},
  {applications,[kernel,stdlib]},
  {mod,{cache_app,[]}},
  {env,[]},
  {modules,[cache_app,cache_sup,cache_srv]}]}.

cache_app.erl :

-module(cache_app).
-behaviour(application).

%% api
-export([start/2, stop/1]).

%%
%% behavior callbacks
%%

start(_StartType, _StartArgs) -> 
    cache_sup:start_link().

stop(_State) -> ok.

cache_sup.erl :

-module(cache_sup).
-behaviour(supervisor).

%% api
-export([start_link/0]).

%% behavior callbacks
-export([init/1]).

-define(SERVER, ?MODULE).
-define(ON_DEMAND, {undefined, {
  cache_srv, start_link, []}, 
  transient, 5000, worker, [cache_srv]}).

%%
%% api routines
%%

start_link() ->
    supervisor:start_link({local, ?SERVER}, ?MODULE, []).

%%
%% behavior callbacks
%%

init([]) ->
    Specs = [?ON_DEMAND],
    {ok, {{simple_one_for_one, 5, 10}, Specs}}.

cache_srv.erl :

-module(cache_srv).
-behaviour(gen_server).

%% api
-export([start_link/1]).

%% behavior callbacks
-export([
  init/1,
  handle_call/3,
  handle_cast/2,
  handle_info/2,
  terminate/2,
  code_change/3]).

-record(loop_data, {value :: term()}).

%%
%% api routines
%%

start_link({K, V}) ->
    gen_server:start_link({global, K}, ?MODULE, [V], []).

%%
%% callback routines
%%

init([V]) -> 
    LoopData = #loop_data{value=V},
    {ok, LoopData}.

handle_call(_Request, _From, LoopData) ->
    Reply = ok,
    {reply, Reply, LoopData}.

handle_cast(_Msg, LoopData) ->
    {noreply, LoopData}.

handle_info(_Info, LoopData) ->
    {noreply, LoopData}.

terminate(_Reason, _L) -> ok.

code_change(_OldVsn, LoopData, _Extra) ->
    {ok, LoopData}.

With our application stubbed out, let’s test to make sure it starts before adding any functionality. We can do that as follows:

erlc -o ebin src/*.erl
cat src/cache.app.src > ebin/cache.app

The first line compiles our Erlang and places it in the standard ‘ebin’ directory. The second line creates our application resource file and also places it in ‘ebin.’

Let’s fire up an Erlang shell and see if the cache application loads — this will indicate that our Erlang code adheres to the OTP application rules. We can test it out as follows:

erl -pa ../cache/ebin 

1> ok = application:start(cache).
2> application:which_applications().

If all went well, our application should show up in the list of applications from what’s returned in line two.

The cache server

The purpose of our cache is to hold values in a manner that isn’t mission critical, but just so we can get and put values into something for the sake of this tutorial. Again, we decided to use the state of a ‘gen_server’ for value storage, and use the global name registry to map our dynamic ‘gen_server’ ‘pids’ to the many keys we create.

To start, we want to have our ‘gen_server’ spit out its state (the cached value for that particular key) when asked for it. We change ‘cache_srv.erl’ in the following way:

cache_srv.erl :

...

handle_call(get, _From, #loop_data{value=V} = LoopData) ->
    Reply = {result, V},
    {reply, Reply, LoopData};
handle_call(_Request, _From, LoopData) ->
    Reply = ok,
    {reply, Reply, LoopData}.

...

In addition, we want our cache to expire. After twenty seconds, the ‘gen_server’ should timeout, releasing the key from the global name registry and reducing the cache by one storage value.

All said and done, our final ‘gen_server’ looks like this:

cache_srv.erl :

-module(cache_srv).
-behaviour(gen_server).

%% api
-export([start_link/1]).

%% behavior callbacks
-export([
  init/1,
  handle_call/3,
  handle_cast/2,
  handle_info/2,
  terminate/2,
  code_change/3]).

-define(EXPIRE, (1000 * 20)). % twenty seconds

-record(loop_data, {value :: term()}).

%%
%% api routines
%%

start_link({K, V}) ->
    gen_server:start_link({global, K}, ?MODULE, [V], []).

%%
%% callback routines
%%

init([V]) -> 
    LoopData = #loop_data{value=V},
    new(cache_expire, ?EXPIRE),
    {ok, LoopData}.

handle_call(get, _From, #loop_data{value=V} = LoopData) ->
    Reply = {result, V},
    {reply, Reply, LoopData};
handle_call(_Request, _From, LoopData) ->
    Reply = ok,
    {reply, Reply, LoopData}.

handle_cast(_Msg, LoopData) ->
    {noreply, LoopData}.

handle_info(cache_expire, LoopData) ->
    {stop, normal, LoopData};
handle_info(_Info, LoopData) ->
    {noreply, LoopData}.

terminate(_Reason, _L) -> ok.

code_change(_OldVsn, LoopData, _Extra) ->
    {ok, LoopData}.

%% 
%% business routines
%%

new(cache_expire, When) ->
    S = self(),
    erlang:send_after(When, S, cache_expire).

A helpful API

Next, we want to build out an API for our application. Let’s offer a ‘put’ & ’get’ routine in a new module called ‘cache_lib.erl.’ It will also live in the ‘src’ directory and it looks as follows:

cache_lib.erl :

-module(cache_lib).

%% api
-export([get/1, put/2]).

%%
%% api routines
%%

get(Key) -> 
    X = (catch gen_server:call({global, Key}, get)),
    handle_get(X).

put(Key, Val) when is_binary(Key) ->
    X = supervisor:start_child(cache_sup, [{Key, Val}]),
    handle_put(X).

%%
%% business routines
%%

handle_get({'EXIT', {noproc, {gen_server, call, [_Key, get]}}}) ->
    {error, undefined_key};
handle_get(X) -> X.

handle_put({error, {already_started, _Pid}}) ->
    {error, duplicate_key};
handle_put({ok, _Pid}) -> ok.

Since we added a new module, it would be forgetful of us to exclude it from our application resource file. Let’s fix that. That file now looks as follows:

cache.app.src :

{application,cache,[
  {description,"Dummy cache app."},
  {vsn,"1986.a"},
  {registered,[cache_sup]},
  {applications,[kernel,stdlib]},
  {mod,{cache_app,[]}},
  {env,[]},
  {modules,[cache_app,cache_sup,cache_srv,cache_lib]}]}.

And of course, we need to freshen the actual application resource file in ‘ebin’ too:

cat src/cache.app.src > ebin/cache.app

If we open a new Unix shell/tab and head to our cache project root, we don’t have to stop the Erlang application. We can load our new code and test it. Running the following will compile the new code:

erlc -o ebin src/*.erl

And in our running Erlang shell, we can load what’s new without bringing our service down by performing the following:

3> l(cache_srv), l(cache_lib).

Testing our complete OTP application via the Erlang shell will suffice for now (making sure it behaves the way we want). Using our new API, we are able to get and put key/value pairs with ease. For example:

4> cache_lib:put(<<"foo">>, bar).
5> cache_lib:get(<<"foo">>).
6> cache_lib:put(<<"baz">>, "luhrmann").
7> cache_lib:get(<<"3">>).
8> cache_lib:get(<<"baz">>).
9> cache_lib:get(<<"foo">>).

All went well with our back-of-the-envelope tests. If you hadn’t noticed, in the ‘cache_lib.erl’ source, we designed it so keys must be bitstrings. If you were to attempt to have your key as anything else, you’d be met with an exception.

Building a release from hand

Creating a release is all about minding your ‘Ps’ & ‘Qs.’ It doesn’t take too long to do it all manually, and there’s oodles to be gained by doing so.

So, dismissing all the automation tools out there for the time being, let’s turn our cache application into a release!

Step one

First off, we need to create a release resource file.

Creating a release resource file is not unlike how we create an application resource file (cache.app). Its contents are just an Erlang term that provides information to the build system. However, we must gather a few details in order to know what to place in our new ‘.rel’ file.

Do you still have your Erlang shell still going? Let’s poke and prod in there & take notes all the while. First, run the following in the Erlang shell:

10> application:which_applications().

From that, note the version numbers of ‘kernel,’ ‘stdlib,’ and our new ‘cache’ application (the version is the last element in each tuple).

Also, we need to know the version of the OTP runtime system we want to bundle with our release. The Erlang runtime system version can be discovered in the following way:

11> erlang:system_info(version).

Finally, we want to include the ‘sasl’ application with our release (it includes some useful tools). You can discover the version to include in many ways, but here’s one method:

12> application:start(sasl).
13> application:which_applications().

And as before, note the ‘sasl’ version from what’s yielded.

(Another method is via the Erlang shell, call m(sasl). & notice the version there.)

We should have everything we need to create our release resource file. Just to be clear, we will name our release ‘cacheapp’ so not to confuse the ‘cache’ application for the sake of this tutorial. It will need to sit in our project root; this is what the file looks like for me (yours will certainly vary slightly) :

cacheapp.rel :

{release,
  {"cacheapp","1986.a"},
  {erts,"6.4"},[
    {kernel,"3.2"},
    {stdlib,"2.4"},
    {sasl,"2.4.1"},
    {cache,"1986.a"}]}.

Step two

From there, we can create our binary boot file. The boot file requires our release resource specification, after which, it’s a one-liner to create it.

In the Erlang shell, perform the following:

14> systools:make_script("cacheapp", [{path, ["ebin"]}]).

If all went well, the above should have returned ‘ok.’ But to further probe, let’s stop our Erlang shell and start it up utilizing our new boot file.

15> init:stop().

erl -pa ../cache/ebin -boot cacheapp -sasl sasl_error_logger false 

1> cache_lib:get(foo).
2> init:stop().

It looks like our application works with our generated boot file. It’s time to move on!

Step three

Believe it or not, our release is almost complete, and the experience hasn’t been that grueling thus far. Of course, along the way, we have picked the most vanilla of approaches. Almost every step can be embellished with further options & configurations. All one has to do is read the apt manuals.

That said, it’s time to create a release package. In this step, we mostly wear our ‘sys-admin’ hat and move things around via the Unix shell. Follow along with some care:

mkdir -p _rel/cacheapp
echo '[].' > sys.config
erl

1> R = code:root_dir().
2> systools:make_tar("cacheapp", [{erts, R}, {path, ["ebin"]}, {outdir, "_rel/cacheapp"}]).
3> init:stop().

cd _rel/cacheapp
tar xvf cacheapp.tar.gz

For those who have built releases with, say, rebar3, or relx, when we ‘guntar’ in the last line above, we should be in familiar waters. At the minimum, acknowledge all your hard work for a moment by perusing what files were just unpackaged. Do this as follows:

find . | less

Behold: our complete embedded OTP system. But wait! Because we did this all by hand, sadly, we are missing a few helpful things build tools usually supply. So, let’s take care of that in a final step.

Step four

We want to be able to start our system in some canonical way. Follow along with some care & we will achieve just that in the last stretch of things.

Working all the time in our ‘_rel/cacheapp’ directory, we do the following:

mkdir bin log
cp erts-6.4/bin/to_erl bin
cp erts-6.4/bin/run_erl bin
cp erts-6.4/bin/start.src bin/start
cp erts-6.4/bin/start_erl.src bin/start_erl

N.B. The above ‘erts’ directory naming will vary for you.

Then, grab the result of running pwd in that same Unix shell, as we need the full path for setting our root directory in that new ‘bin/start’ script. With that in hand, we change it as follows:

bin/start :

...

#ROOTDIR=%FINAL_ROOTDIR%
ROOTDIR=/change/me/to/your/pwd/cache/_rel/cacheapp

...

Lastly, we need to specify both the ‘erts’ version and the ‘cacheapp’ version in this one place. Perform the following, being sure to use your version of ‘erts,’ not mine:

echo '6.4 1986.a' > releases/start_erl.data

Done. We have made it though the weeds in four somewhat brief steps. Congratulations! We have an embedded target system ready to be used & abused.

Trying things out

In our targeted directory — ‘_rel/cacheapp’ — let’s go ahead and test our cache. We can start the system & attach to it as follows:

./bin/start
./bin/to_erl /tmp/

N.B. The argument in the second line above requires a trailing slash.

You should be attached to the system now & can detach with CTRL-D. Go ahead and try a few caching ‘gets’ & ‘puts’ before stopping the system with init:stop().

Wrapping up

So, there you have it. With an OTP application in hand, in four strides, we can make an embedded target system with not too much pain.

It’s never regretful to go deep once in a while & learn things from first principles. Moreover, I hope this tutorial gave some perspective on how the building blocks of OTP permeates into our own systems.

Update Nov. 2016

Even a toy system will probably want some emulator flags specified. Perhaps not obvious, but ‘bin/start_erl’ is the place we can sneak in whatever we want. For example, the following change now accommodates a short-name for the node:

...

exec $BINDIR/erlexec -sname cacheapp -boot $RELDIR/$VSN/start -config $RELDIR/$VSN/sys ${1+"$@"}

Update Jun. 2023

A post that has a some parts scripted with ‘escript’ has been written, in case there is a need for more reference in these matters: here.