377 lines
14 KiB
XML
377 lines
14 KiB
XML
|
<chapter xmlns="http://docbook.org/ns/docbook"
|
|||
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|||
|
|
xml:id="users-guide-to-the-erlang-infrastructure">
|
|||
|
|
|
|||
|
|
<title>User's Guide to the Beam Infrastructure</title>
|
|||
|
|
<section xml:id="beam-introduction">
|
|||
|
|
<title>Beam Languages (Erlang & Elixir) on Nix</title>
|
|||
|
|
<para>
|
|||
|
|
In this document and related Nix expressions we use the term
|
|||
|
|
<emphasis>Beam</emphasis> to describe the environment. Beam is
|
|||
|
|
the name of the Erlang Virtial Machine and, as far as we know,
|
|||
|
|
from a packaging perspective all languages that run on Beam are
|
|||
|
|
interchangable. The things that do change, like the build
|
|||
|
|
system, are transperant to the users of the package. So we make
|
|||
|
|
no distinction.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="build-tools">
|
|||
|
|
<title>Build Tools</title>
|
|||
|
|
<section xml:id="build-tools-rebar3">
|
|||
|
|
<title>Rebar3</title>
|
|||
|
|
<para>
|
|||
|
|
By default Rebar3 wants to manage it's own dependencies. In the
|
|||
|
|
normal non-Nix, this is perfectly acceptable. In the Nix world it
|
|||
|
|
is not. To support this we have created two versions of rebar3,
|
|||
|
|
<literal>rebar3</literal> and <literal>rebar3-open</literal>. The
|
|||
|
|
<literal>rebar3</literal> version has been patched to remove the
|
|||
|
|
ability to download anything from it. If you are not running it a
|
|||
|
|
nix-shell or a nix-build then its probably not going to work for
|
|||
|
|
you. <literal>rebar3-open</literal> is the normal, un-modified
|
|||
|
|
rebar3. It should work exactly as would any other version of
|
|||
|
|
rebar3. Any Erlang package should rely on
|
|||
|
|
<literal>rebar3</literal> and thats really what you should be
|
|||
|
|
using too.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="build-tools-other">
|
|||
|
|
<title>Mix & Erlang.mk</title>
|
|||
|
|
<para>
|
|||
|
|
Both Mix and Erlang.mk work exactly as you would expect. There
|
|||
|
|
is a bootstrap process that needs to be run for both of
|
|||
|
|
them. However, that is supported by the
|
|||
|
|
<literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
|
|||
|
|
</section>
|
|||
|
|
|
|||
|
|
<section xml:id="how-to-install-beam-packages">
|
|||
|
|
<title>How to install Beam packages</title>
|
|||
|
|
<para>
|
|||
|
|
Beam packages are not registered in the top level simply because
|
|||
|
|
they are not relevant to the vast majority of Nix users. They are
|
|||
|
|
installable using the <literal>beamPackages</literal> attribute
|
|||
|
|
set.
|
|||
|
|
|
|||
|
|
You can list the avialable packages in the
|
|||
|
|
<literal>beamPackages</literal> with the following command:
|
|||
|
|
</para>
|
|||
|
|
|
|||
|
|
<programlisting>
|
|||
|
|
$ nix-env -f "<nixpkgs>" -qaP -A beamPackages
|
|||
|
|
beamPackages.esqlite esqlite-0.2.1
|
|||
|
|
beamPackages.goldrush goldrush-0.1.7
|
|||
|
|
beamPackages.ibrowse ibrowse-4.2.2
|
|||
|
|
beamPackages.jiffy jiffy-0.14.5
|
|||
|
|
beamPackages.lager lager-3.0.2
|
|||
|
|
beamPackages.meck meck-0.8.3
|
|||
|
|
beamPackages.rebar3-pc pc-1.1.0
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
To install any of those packages into your profile, refer to them by
|
|||
|
|
their attribute path (first column):
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
$ nix-env -f "<nixpkgs>" -iA beamPackages.ibrowse
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
The attribute path of any Beam packages corresponds to the name
|
|||
|
|
of that particular package in Hex or its OTP Application/Release name.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="packaging-beam-applications">
|
|||
|
|
<title>Packaging Beam Applications</title>
|
|||
|
|
<section xml:id="packaging-erlang-applications">
|
|||
|
|
<title>Erlang Applications</title>
|
|||
|
|
<section xml:id="rebar3-packages">
|
|||
|
|
<title>Rebar3 Packages</title>
|
|||
|
|
<para>
|
|||
|
|
There is a Nix functional called
|
|||
|
|
<literal>buildRebar3</literal>. We use this function to make a
|
|||
|
|
derivation that understands how to build the rebar3 project. For
|
|||
|
|
example, the epression we use to build the <link
|
|||
|
|
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
|
|||
|
|
project follows.
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
|
|||
|
|
|
|||
|
|
buildRebar3 rec {
|
|||
|
|
name = "hex2nix";
|
|||
|
|
version = "0.0.1";
|
|||
|
|
|
|||
|
|
src = fetchFromGitHub {
|
|||
|
|
owner = "ericbmerritt";
|
|||
|
|
repo = "hex2nix";
|
|||
|
|
rev = "${version}";
|
|||
|
|
sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
|
|||
|
|
};
|
|||
|
|
|
|||
|
|
beamDeps = [ ibrowse jsx erlware_commons ];
|
|||
|
|
}
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
The only visible difference between this derivation and
|
|||
|
|
something like <literal>stdenv.mkDerivation</literal> is that we
|
|||
|
|
have added <literal>erlangDeps</literal> to the derivation. If
|
|||
|
|
you add your Beam dependencies here they will be correctly
|
|||
|
|
handled by the system.
|
|||
|
|
</para>
|
|||
|
|
<para>
|
|||
|
|
If your package needs to compile native code via Rebar's port
|
|||
|
|
compilation mechenism. You should add <literal>compilePort =
|
|||
|
|
true;</literal> to the derivation.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="erlang-mk-packages">
|
|||
|
|
<title>Erlang.mk Packages</title>
|
|||
|
|
<para>
|
|||
|
|
Erlang.mk functions almost identically to Rebar. The only real
|
|||
|
|
difference is that <literal>buildErlangMk</literal> is called
|
|||
|
|
instead of <literal>buildRebar3</literal>
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
{ buildErlangMk, fetchHex, cowlib, ranch }:
|
|||
|
|
buildErlangMk {
|
|||
|
|
name = "cowboy";
|
|||
|
|
version = "1.0.4";
|
|||
|
|
src = fetchHex {
|
|||
|
|
pkg = "cowboy";
|
|||
|
|
version = "1.0.4";
|
|||
|
|
sha256 =
|
|||
|
|
"6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
|
|||
|
|
};
|
|||
|
|
beamDeps = [ cowlib ranch ];
|
|||
|
|
|
|||
|
|
meta = {
|
|||
|
|
description = ''Small, fast, modular HTTP server written in
|
|||
|
|
Erlang.'';
|
|||
|
|
license = stdenv.lib.licenses.isc;
|
|||
|
|
homepage = "https://github.com/ninenines/cowboy";
|
|||
|
|
};
|
|||
|
|
}
|
|||
|
|
</programlisting>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="mix-packages">
|
|||
|
|
<title>Mix Packages</title>
|
|||
|
|
<para>
|
|||
|
|
Mix functions almost identically to Rebar. The only real
|
|||
|
|
difference is that <literal>buildMix</literal> is called
|
|||
|
|
instead of <literal>buildRebar3</literal>
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
{ buildMix, fetchHex, plug, absinthe }:
|
|||
|
|
buildMix {
|
|||
|
|
name = "absinthe_plug";
|
|||
|
|
version = "1.0.0";
|
|||
|
|
src = fetchHex {
|
|||
|
|
pkg = "absinthe_plug";
|
|||
|
|
version = "1.0.0";
|
|||
|
|
sha256 =
|
|||
|
|
"08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
|
|||
|
|
};
|
|||
|
|
beamDeps = [ plug absinthe];
|
|||
|
|
|
|||
|
|
meta = {
|
|||
|
|
description = ''A plug for Absinthe, an experimental GraphQL
|
|||
|
|
toolkit'';
|
|||
|
|
license = stdenv.lib.licenses.bsd3;
|
|||
|
|
homepage = "https://github.com/CargoSense/absinthe_plug";
|
|||
|
|
};
|
|||
|
|
}
|
|||
|
|
</programlisting>
|
|||
|
|
</section>
|
|||
|
|
</section>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="how-to-develop">
|
|||
|
|
<title>How to develop</title>
|
|||
|
|
<section xml:id="accessing-an-environment">
|
|||
|
|
<title>Accessing an Environment</title>
|
|||
|
|
<para>
|
|||
|
|
Often, all you want to do is be able to access a valid
|
|||
|
|
environment that contains a specific package and its
|
|||
|
|
dependencies. we can do that with the <literal>env</literal>
|
|||
|
|
part of a derivation. For example, lets say we want to access an
|
|||
|
|
erlang repl with ibrowse loaded up. We could do the following.
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
~/w/nixpkgs ❯❯❯ nix-shell -A beamPackages.ibrowse.env --run "erl"
|
|||
|
|
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
|
|||
|
|
|
|||
|
|
Eshell V7.0 (abort with ^G)
|
|||
|
|
1> m(ibrowse).
|
|||
|
|
Module: ibrowse
|
|||
|
|
MD5: 3b3e0137d0cbb28070146978a3392945
|
|||
|
|
Compiled: January 10 2016, 23:34
|
|||
|
|
Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
|
|||
|
|
Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
|
|||
|
|
debug_info,debug_info,nowarn_shadow_vars,
|
|||
|
|
warn_unused_import,warn_unused_vars,warnings_as_errors,
|
|||
|
|
{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
|
|||
|
|
Exports:
|
|||
|
|
add_config/1 send_req_direct/7
|
|||
|
|
all_trace_off/0 set_dest/3
|
|||
|
|
code_change/3 set_max_attempts/3
|
|||
|
|
get_config_value/1 set_max_pipeline_size/3
|
|||
|
|
get_config_value/2 set_max_sessions/3
|
|||
|
|
get_metrics/0 show_dest_status/0
|
|||
|
|
get_metrics/2 show_dest_status/1
|
|||
|
|
handle_call/3 show_dest_status/2
|
|||
|
|
handle_cast/2 spawn_link_worker_process/1
|
|||
|
|
handle_info/2 spawn_link_worker_process/2
|
|||
|
|
init/1 spawn_worker_process/1
|
|||
|
|
module_info/0 spawn_worker_process/2
|
|||
|
|
module_info/1 start/0
|
|||
|
|
rescan_config/0 start_link/0
|
|||
|
|
rescan_config/1 stop/0
|
|||
|
|
send_req/3 stop_worker_process/1
|
|||
|
|
send_req/4 stream_close/1
|
|||
|
|
send_req/5 stream_next/1
|
|||
|
|
send_req/6 terminate/2
|
|||
|
|
send_req_direct/4 trace_off/0
|
|||
|
|
send_req_direct/5 trace_off/2
|
|||
|
|
send_req_direct/6 trace_on/0
|
|||
|
|
trace_on/2
|
|||
|
|
ok
|
|||
|
|
2>
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
Notice the <literal>-A beamPackages.ibrowse.env</literal>.That
|
|||
|
|
is the key to this functionality.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="creating-a-shell">
|
|||
|
|
<title>Creating a Shell</title>
|
|||
|
|
<para>
|
|||
|
|
Getting access to an environment often isn't enough to do real
|
|||
|
|
development. Many times we need to create a
|
|||
|
|
<literal>shell.nix</literal> file and do our development inside
|
|||
|
|
of the environment specified by that file. This file looks a lot
|
|||
|
|
like the packageing described above. The main difference is that
|
|||
|
|
<literal>src</literal> points to project root and we call the
|
|||
|
|
package directly.
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
{ pkgs ? import "<nixpkgs"> {} }:
|
|||
|
|
|
|||
|
|
with pkgs;
|
|||
|
|
|
|||
|
|
let
|
|||
|
|
|
|||
|
|
f = { buildRebar3, ibrowse, jsx, erlware_commons }:
|
|||
|
|
buildRebar3 {
|
|||
|
|
name = "hex2nix";
|
|||
|
|
version = "0.1.0";
|
|||
|
|
src = ./.;
|
|||
|
|
erlangDeps = [ ibrowse jsx erlware_commons ];
|
|||
|
|
};
|
|||
|
|
drv = beamPackages.callPackage f {};
|
|||
|
|
|
|||
|
|
in
|
|||
|
|
drv
|
|||
|
|
</programlisting>
|
|||
|
|
<section xml:id="building-in-a-shell">
|
|||
|
|
<title>Building in a shell</title>
|
|||
|
|
<para>
|
|||
|
|
We can leveral the support of the Derivation, regardless of
|
|||
|
|
which build Derivation is called by calling the commands themselv.s
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
# =============================================================================
|
|||
|
|
# Variables
|
|||
|
|
# =============================================================================
|
|||
|
|
|
|||
|
|
NIX_TEMPLATES := "$(CURDIR)/nix-templates"
|
|||
|
|
|
|||
|
|
TARGET := "$(PREFIX)"
|
|||
|
|
|
|||
|
|
PROJECT_NAME := thorndyke
|
|||
|
|
|
|||
|
|
NIXPKGS=../nixpkgs
|
|||
|
|
NIX_PATH=nixpkgs=$(NIXPKGS)
|
|||
|
|
NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
|
|||
|
|
# =============================================================================
|
|||
|
|
# Rules
|
|||
|
|
# =============================================================================
|
|||
|
|
.PHONY= all test clean repl shell build test analyze configure install \
|
|||
|
|
test-nix-install publish plt analyze
|
|||
|
|
|
|||
|
|
all: build
|
|||
|
|
|
|||
|
|
guard-%:
|
|||
|
|
@ if [ "${${*}}" == "" ]; then \
|
|||
|
|
echo "Environment variable $* not set"; \
|
|||
|
|
exit 1; \
|
|||
|
|
fi
|
|||
|
|
|
|||
|
|
clean:
|
|||
|
|
rm -rf _build
|
|||
|
|
rm -rf .cache
|
|||
|
|
|
|||
|
|
repl:
|
|||
|
|
$(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
|
|||
|
|
|
|||
|
|
shell:
|
|||
|
|
$(NIX_SHELL)
|
|||
|
|
|
|||
|
|
configure:
|
|||
|
|
$(NIX_SHELL) --command 'eval "$$configurePhase"'
|
|||
|
|
|
|||
|
|
build: configure
|
|||
|
|
$(NIX_SHELL) --command 'eval "$$buildPhase"'
|
|||
|
|
|
|||
|
|
install:
|
|||
|
|
$(NIX_SHELL) --command 'eval "$$installPhase"'
|
|||
|
|
|
|||
|
|
test:
|
|||
|
|
$(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
|
|||
|
|
|
|||
|
|
plt:
|
|||
|
|
$(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
|
|||
|
|
|
|||
|
|
analyze: build plt
|
|||
|
|
$(NIX_SHELL) --run "mix dialyzer --no-compile"
|
|||
|
|
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
If you add the <literal>shell.nix</literal> as described and
|
|||
|
|
user rebar as follows things should simply work. Aside from the
|
|||
|
|
<literal>test</literal>, <literal>plt</literal>, and
|
|||
|
|
<literal>analyze</literal> the talks work just fine for all of
|
|||
|
|
the build Derivations.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
</section>
|
|||
|
|
</section>
|
|||
|
|
<section xml:id="generating-packages-from-hex-with-hex2nix">
|
|||
|
|
<title>Generating Packages from Hex with Hex2Nix</title>
|
|||
|
|
<para>
|
|||
|
|
Updating the Hex packages requires the use of the
|
|||
|
|
<literal>hex2nix</literal> tool. Given the path to the Erlang
|
|||
|
|
modules (usually
|
|||
|
|
<literal>pkgs/development/erlang-modules</literal>). It will
|
|||
|
|
happily dump a file called
|
|||
|
|
<literal>hex-packages.nix</literal>. That file will contain all
|
|||
|
|
the packages that use a recognized build system in Hex. However,
|
|||
|
|
it can't know whether or not all those packages are buildable.
|
|||
|
|
</para>
|
|||
|
|
<para>
|
|||
|
|
To make life easier for our users, it makes good sense to go
|
|||
|
|
ahead and attempt to build all those packages and remove the
|
|||
|
|
ones that don't build. To do that, simply run the command (in
|
|||
|
|
the root of your <literal>nixpkgs</literal> repository). that follows.
|
|||
|
|
</para>
|
|||
|
|
<programlisting>
|
|||
|
|
$ nix-build -A beamPackages
|
|||
|
|
</programlisting>
|
|||
|
|
<para>
|
|||
|
|
That will build every package in
|
|||
|
|
<literal>beamPackages</literal>. Then you can go through and
|
|||
|
|
manually remove the ones that fail. Hopefully, someone will
|
|||
|
|
improve <literal>hex2nix</literal> in the future to automate
|
|||
|
|
that.
|
|||
|
|
</para>
|
|||
|
|
</section>
|
|||
|
|
</chapter>
|