As software engineers we use package managers on a daily basis. We use them to install dependencies we need to run and build software we write. Probably every software engineer can relate to the frustration that will eventually arise from using these package managers. Sometimes packages that seem to work on your colleagues machine just fine, are broken on yours. Even though package managers have improved substantially over time, issues like these still arise. Maybe there is some fundamental design flaw in the way we approach package management. There is a package manager that tries to do things different and it is called Nix. Let’s take a look at what Nix is and how you can use it on your machine today.
Enter the Nix
Nix is many things, a language, an operating system (nixos), a deployment system, and a continuous integration system (Hydra). But at its core it is a package manager, specifically a package manager that is designed from the ground up to be pure and immutable. Pure in the sense that building a package can not perform any uncontrolled side-effects, and immutable in the sense that no global package state is mutated in place. This is a fundamental shift from the current approach to installing packages where we install all binaries to a global and mutable directory e.g.
/usr/local/bin.This shift allows us to support different versions of the same executable or library on our operating system.
Nix is pure, which means it can distinguish the same package build with different input parameters. It captures all input parameters, which could be anything from the version of GCC, to the library dependencies (e.g. glibc). So in Nix a bundled package is not just referenced by it’s name and version, but is prefixed with the aggregation of all parameters used to build that package. This aggregation is represented as a hash, so when any of the input parameters changes this hash will also change.
Referencing binaries by these hashes would be really unproductive, we want to be able to type
git in our terminal and not
86b4f4cd27e63c8e05c745807dd281ec04c38521-git. Nix solves this problem by creating a symlink under the name git inside a directory exposed on your
PATH to the actual location of the git binary in the nix store. A collection of such symlinks is called a profile.
Symlinks enable immutability. Because we no longer have to mutate the binaries directly inside directories exposed on our
/usr/local/bin), but instead rely on the
/nix/store directory. And atomic operations, because the creation of symlinks happen as a final step in our installation procedure. So an installation that fails half-way through will not have any noticeable side-effects.
In this tutorial I will explain how you can use nix to install binaries and create isolated and reproducible development environments. It is, by no stretch of the imagination, a complete guide on how to use Nix. But hopefully it will capture your imagination of Nix’s possibilities.
Install the nix package manager:
$ curl https://nixos.org/nix/install | sh
After installation we have some new commands available:
nix-envused to install dependencies and alter your current profile
nix-shellused to start shells based on nix expressions
nix-storeused to query or manipulate the nix store
And a directory has been created for the nix store at
/nix, this is directory containing all installed packages, profiles and configuration. Your global environment will only be mutated by symlinking into the nix-store.
We can now execute several actions to search, install, and uninstall packages:
You can inspect the
/nix/store directory to see all the dependencies that are currently installed on your machine. You will see that all packages are prefixed by a hash, which as mentioned in the first section represent all the input parameters used to build this package.
When we install a second version of Node it would override the symlink in our nix-profile.
$ node --version
Since the binary of node is a symlink to the actual binary stored within the nix store (at /nix), we can also load an isolated shell which is points node to a specific version of node in the nix store.
With the nix language it is also possible to write a configuration file that we can reuse. These configuration files are called derivations and are the most important building block of nix. They describe how package are to be build and with what parameters. They are written in the nix language. Take for example this derivation which will load nodejs v12 in the environment:
If we store this file as default.nix we can load it inside a nix shell with the following command:
$ nix-shell default.nix
We can now create reproducible development environments by writing a nix derivation, to enter the environment we still require an invocation of nix-shell, we could also leverage a tool called direnv to automatically load the nix derivation while entering the directory containing our default.nix derivation.
Bonus: in tandem with direnv
direnv is an environment management tool developed by zimbatm. It essentially sets and removes environment variables in the shell depending on the current directory and the presence of a .envrc file.
Before each prompt, direnv checks for the existence of a .envrc file in the current and parent directories. If the file exists (and is authorized), it is loaded into a bash sub-shell and all exported variables are then captured by direnv and then made available to the current shell.
Since direnv supports nix out of the box we only need to create a .envrc and enable the nix integration, to do this run the following in the directory containing your default.nix file:
$ direnv edit .
Add the following line to the file to enable nix integration, and save:
Once the file is saved direnv will automatically reload the environment we should see something along these lines:
direnv: loading .envrc direnv: using nix direnv: export +AR +AS +CC +CMAKE_OSX_ARCHITECTURES +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_x86_64_apple_darwin_TARGET_HOST +NIX_BUILD_CORES +NIX_BUILD_DONT_SET_RPATH +NIX_BUILD_TOP +NIX_CC +NIX_CC_WRAPPER_x86_64_apple_darwin_TARGET_HOST +NIX_CFLAGS_COMPILE +NIX_COREFOUNDATION_RPATH +NIX_CXXSTDLIB_COMPILE +NIX_CXXSTDLIB_LINK +NIX_DONT_SET_RPATH +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_INDENT_MAKE +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +NODE_PATH +PATH_LOCALE +RANLIB +SDKROOT +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +TEMP +TEMPDIR +TMP +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +buildInputs +builder +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +gl_cv_func_getcwd_abort_bug +name +nativeBuildInputs +out +outputs +patches +propagatedBuildInputs +propagatedNativeBuildInputs +shell +stdenv +strictDeps +system ~PATH
If we run which node we should see that node is now loaded from the nix store:
Managing multiple binary versions or reproducible build environments is just the start of nix’s capabilities. It can be used to manage entirely reproducible operating systems or to create reproducible container images. Nix is in potential a great tool for managing reproducible development environments. While the tooling is still a bit rough from a UX perspective and the package registry support could be better on OS X. The philosophy of nix shows great potential for the future iterations of it’s operating system and package manager.
- Nix is only supported on Linux and Mac OS, and currently the derivation support for Linux is superior to that of Mac OS.
- Since the Nix store is immutable it is not possible to install global dependencies of your programming language package manager inside the nix store. As this is exactly what npm will try to do when you run
npm install -gthis operation will fail. The following blog post describe a way in which you can circumvent this issue: http://nicknovitski.com/nix-npm-install