I was Wrong about Nix

Published on , 1877 words, 7 minutes to read

From time to time, I am outright wrong on my blog. This is one of those times. In my last post about Nix, I didn't see the light yet. I think I do now, and I'm going to attempt to clarify below.

Let's talk about a more simple scenario: writing a service in Go. This service will depend on at least the following:

A popular way to model this is with a Dockerfile. Here's the Dockerfile I use for my website (the one you are reading right now):

FROM xena/go:1.13.6 AS build
ENV GOPROXY https://cache.greedo.xeserv.us
COPY . /site
WORKDIR /site
RUN CGO_ENABLED=0 go test -v ./...
RUN CGO_ENABLED=0 GOBIN=/root go install -v ./cmd/site

FROM xena/alpine
EXPOSE 5000
WORKDIR /site
COPY --from=build /root/site .
COPY ./static /site/static
COPY ./templates /site/templates
COPY ./blog /site/blog
COPY ./talks /site/talks
COPY ./gallery /site/gallery
COPY ./css /site/css
HEALTHCHECK CMD wget --spider http://127.0.0.1:5000/.within/health || exit 1
CMD ./site

This fetches the Go compiler from an image I made, copies the source code to the image, builds it (in a way that makes the resulting binary a static executable), and creates the runtime environment for it.

Let's let it build and see how big the result is:

$ docker build -t xena/christinewebsite:example1 .
<output omitted>
$ docker images | grep xena
xena/christinewebsite  example1  4b8ee64969e8  24 seconds ago  111MB

Investigating this image with dive, we see the following:

Granted, Alpine Linux does a good job at keeping this chaff to a minimum, but it is still there, still needs to be updated (causing all of my docker images to be rebuilt and applications to be redeployed) and still takes up space in transfer quotas and on the disk.

Let's compare this to the same build process but done with Nix. My Nix setup is done in a few phases. First I use niv to manage some dependencies a-la git submodules that don't hate you:

$ nix-shell -p niv
[nix-shel]$ niv init
<writes nix/*>

Now I add the tool vgo2nix in niv:

[nix-shell]$ niv add adisbladis/vgo2nix

And I can use it in my shell.nix:

let
  pkgs = import <nixpkgs> { };
  sources = import ./nix/sources.nix;
  vgo2nix = (import sources.vgo2nix { });
in pkgs.mkShell { buildInputs = [ pkgs.go pkgs.niv vgo2nix ]; }

And then relaunch nix-shell with vgo2nix installed and convert my go modules dependencies to a Nix expression:

$ nix-shell
<some work is done to compile things, etc>
[nix-shell]$ vgo2nix
<writes deps.nix>

Now that I have this, I can follow the buildGoPackage instructions from the upstream nixpkgs documentation and create site.nix:

{ pkgs ? import <nixpkgs> {} }:
with pkgs;

assert lib.versionAtLeast go.version "1.13";

buildGoPackage rec {
  name = "christinewebsite-HEAD";
  version = "latest";
  goPackagePath = "xeiaso.net";
  src = ./.;

  goDeps = ./deps.nix;
  allowGoReference = false;
  preBuild = ''
    export CGO_ENABLED=0
    buildFlagsArray+=(-pkgdir "$TMPDIR")
  '';

  postInstall = ''
    cp -rf $src/blog $bin/blog
    cp -rf $src/css $bin/css
    cp -rf $src/gallery $bin/gallery
    cp -rf $src/static $bin/static
    cp -rf $src/talks $bin/talks
    cp -rf $src/templates $bin/templates
  '';
}

And this will do the following:

This Nix build manifest doesn't just work on Linux. It works on my mac too. The dockerfile approach works great for Linux boxes, but (unlike what the me of a decade ago would have hoped) the whole world just doesn't run Linux on their desktops. The real world has multiple OSes and Nix allows me to compensate.

So, now that we have a working cross-platform build, let's see how big it comes out as:

$ readlink ./result-bin
/nix/store/ayvafpvn763wwdzwjzvix3mizayyblx5-christinewebsite-HEAD-bin
$ du -hs result-bin/
89M     ./result-bin/
$ du -hs result-bin/
11M     ./result-bin/bin
888K    ./result-bin/blog
40K     ./result-bin/css
44K     ./result-bin/gallery
77M     ./result-bin/static
28K     ./result-bin/talks
64K     ./result-bin/templates

As expected, most of the build results are static assets. I have a lot of larger static assets including an entire copy of TempleOS, so this isn't too surprising. Let's compare this to on the mac:

$ du -hs result-bin/
 91M	result-bin/
$ du -hs result-bin/*
 14M	result-bin/bin
872K	result-bin/blog
 36K	result-bin/css
 40K	result-bin/gallery
 77M	result-bin/static
 24K	result-bin/talks
 60K	result-bin/templates

Which is damn-near identical save some macOS specific crud that Go has to deal with.

I mentioned this is used for Docker builds, so let's make docker.nix:

{ system ? builtins.currentSystem }:

let
  pkgs = import <nixpkgs> { inherit system; };

  callPackage = pkgs.lib.callPackageWith pkgs;

  site = callPackage ./site.nix { };

  dockerImage = pkg:
    pkgs.dockerTools.buildImage {
      name = "xena/christinewebsite";
      tag = pkg.version;

      contents = [ pkg ];

      config = {
        Cmd = [ "/bin/site" ];
        WorkingDir = "/";
      };
    };

in dockerImage site

And then build it:

$ nix-build docker.nix
<output omitted>
$ docker load -i result
c6b1d6ce7549: Loading layer [==================================================>]  95.81MB/95.81MB
$ docker images | grep xena
xena/christinewebsite  latest  0d1ccd676af8  50 years ago  94.6MB

And the output is 16 megabytes smaller.

The image age might look weird at first, but it's part of the reproducibility Nix offers. The date an image was built is something that can change with time and is actually a part of the resulting file. This means that an image built one second after another has a different cryptographic hash. It helpfully pins all images to Unix timestamp 0, which just happens to be about 50 years ago.

Looking into the image with dive, the only packages installed into this image are:

And that's it. This is fantastic. Nearly all of the disk usage has been eliminated. If someone manages to trick my website into executing code, that attacker cannot do anything but run more copies of my website (that will immediately fail and die because the port is already allocated).

This strategy pans out to more complicated projects too. Consider a case where a frontend and backend need to be built and deployed as a unit. Let's create a new setup using niv:

$ niv init

Since we are using Elm for this complicated project, let's add the elm2nix tool so that our Elm dependencies have repeatable builds, and gruvbox-css for some nice simple CSS:

$ niv add cachix/elm2nix
$ niv add Xe/gruvbox-css

And then add it to our shell.nix:

let
  pkgs = import <nixpkgs> {};
  sources = import ./nix/sources.nix;
  elm2nix = (import sources.elm2nix { });
in
pkgs.mkShell {
  buildInputs = [
    pkgs.elmPackages.elm
    pkgs.elmPackages.elm-format
    elm2nix
  ];
}

And then enter nix-shell to create the Elm boilerplate:

$ nix-shell
[nix-shell]$ cd frontend
[nix-shell:frontend]$ elm2nix init > default.nix
[nix-shell:frontend]$ elm2nix convert > elm-srcs.nix
[nix-shell:frontend]$ elm2nix snapshot

And then we can edit the generated Nix expression:

let
  sources = import ./nix/sources.nix;
  gcss = (import sources.gruvbox-css { });
# ...
      buildInputs = [ elmPackages.elm gcss ]
        ++ lib.optional outputJavaScript nodePackages_10_x.uglify-js;
# ...
        cp -rf ${gcss}/gruvbox.css $out/public
        cp -rf $src/public/* $out/public/
# ...
  outputJavaScript = true;

And then test it with nix-build:

$ nix-build
<output omitted>

And now create a name.nix for your Go service like I did above. The real magic comes from the docker.nix file:

{ system ? builtins.currentSystem }:

let
  pkgs = import <nixpkgs> { inherit system; };
  sources = import ./nix/sources.nix;
  backend = import ./backend.nix { };
  frontend = import ./frontend/default.nix { };
in

pkgs.dockerTools.buildImage {
  name = "xena/complicatedservice";
  tag = "latest";

  contents = [ backend frontend ];

  config = {
    Cmd = [ "/bin/backend" ];
    WorkingDir = "/public";
  };
};

Now both your backend and frontend services are built with the dependencies in the Nix store and shipped as a repeatable Docker image.

Sometimes it might be useful to ship the dependencies to a service like Cachix to help speed up builds.

You can install the cachix tool like this:

$ nix-env -iA cachix -f https://cachix.org/api/v1/install

And then follow the steps at cachix.org to create a new binary cache. Let's assume you made a cache named teddybear. When you've created a new cache, logged in with an API token and created a signing key, you can pipe nix-build to the Cachix client like so:

$ nix-build | cachix push teddybear

And other people using that cache will benefit from your premade dependency and binary downloads.

To use the cache somewhere, install the Cachix client and then run the following:

$ cachix use teddybear

I've been able to use my Go, Elm, Rust and Haskell dependencies on other machines using this. It's saved so much extra download time.

tl;dr

I was wrong about Nix. It's actually quite good once you get past the documentation being baroque and hard to read as a beginner. I'm going to try and do what I can to get the documentation improved.

As far as getting started with Nix, I suggest following these posts:

Also, I really suggest trying stuff as a vehicle to understand how things work. I got really far by experimenting with getting this Discord bot I am writing in Rust working in Nix and have been very pleased with how it's turned out. I don't need to use rustup anymore to manage my Rust compiler or the language server. With a combination of direnv and lorri, I can avoid needing to set up language servers or the like at all. I can define them as part of the project environment and then trust the tools I build on top of to take care of that for me.

Give Nix a try. It's worth at least that much in my opinion.

Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.

Tags: nix, witchcraft