Ooooookay. This is a really, really long section. This is like… a reference. Dang. Okay.

15.5. Built-in Functions

Some built-ins, such as derivation, are always in scope of every Nix expression; you can just access them right away. But to prevent polluting the namespace too much, most built-ins are not in scope. Instead, you can access them through the builtins built-in value, which is a set that contains all built-in functions and values. For instance, derivation is also available as builtins.derivation.

Okay. It’d be nice to start with the ones that are in scope by default, as they are likely to be the most common or useful functions. But the manual just lists them alphabetically. I skim the list, to try to find those…

abort
baseNameOf
dirOf
fetchTarball
import
isNull
map
removeAttrs
throw
toString

Okay, not too many. Even without type signatures, the names are pretty self-explanatory.

I’m curious how abort and throw work…

throw says:

Throw an error message s. This usually aborts Nix expression evaluation, but in nix-env -qa and other commands that try to evaluate a set of derivations to get information about those derivations, a derivation that throws an error is silently skipped (which is not the case for abort).

Ooookay. Don’t totally get that – what is “a derivation that throws an error?” What does that mean? Why would I ever use throw or abort? Just for debugging? I don’t know.

toString works on sets if they have an attribute called __toString. toString for booleans has the same behavior as the stringification of derivation attributes into environment variables – the "1"/"" thing. I wonder if they are identical, but don’t bother to find out. There is nothing in the description of toString about derivations, for example.

Math stuff. List functions you’d expect. String functions you’d expect. Nothing weird here. Functions on “sets.” Bitwise operations, but I still have no idea what size numbers are in Nix.

Ah, builtins.currentSystem. So I don’t need to rebuild Nix after all…

I learn that fetchTarball caches downloads in ~/.cache/nix/tarballs/. Okay.

There’s fetchGit, with all the arguments you would expect. But this doesn’t take a hash? Even in the cases where you omit ref. That seems weird.

Okay, filterSource is the first function that I don’t intuitively understand.

This function allows you to copy sources into the Nix store while filtering certain files.

Okaaay. So I can refer to, like, the path to a git repo, and it gives me a path back, but with stuff removed? I can use this to exclude the .git directory. Neat. How does that… work? Not explained.

Nix has a foldl' but no foldl. Ha. Oh Haskell, you rascal.

Another weird one: builtins.functionArgs:

Return a set containing the names of the formal arguments expected by the function f. The value of each attribute is a Boolean denoting whether the corresponding argument has a default value. For instance, functionArgs ({ x, y ? 123}: ...) = { x = false; y = true; }.

Weird. What if the argument isn’t a set?

“Formal argument” here refers to the attributes pattern-matched by the function. Plain lambdas are not included, e.g. functionArgs (x: ...) = { }.

Gotcha.

We have getEnv to read environment variables, and a caveat to be careful with it. Makes sense.

Interestingly, there are functions like isList and isFunction but no isSet. Instead that function is called isAttrs. Sorta gross. There’s also isNull, but it is deprecated in favor of x == null. This is a little weird to me – it’s kind of nice to be able to write like builtins.all isNull [null null null] instead of builtins.all (x: x == null) [null null null]. I assume Nix doesn’t support operator sections because there has been no mention of it.

We got regexes! Regex stuff. Alright.

We can parseDrvName:

builtins.parseDrvName "nix-0.12pre12876" = { name = "nix"; version = "0.12pre12876"; }

Okay. Seems to be using Drv as an abbreviation, and not to refer to the .drv files we’ve seen? Cool cool.

builtins.path is weird – it seems that “paths” are not just “paths” but “paths plus a little metadata,” and builtins.path allows me to return new paths with that metadata set. Okay. And it seems like that’s how filterSource works – all paths have a filter function that is apparently run during the act of copying files into the store, and filterSource changes that while leaving all of the other fancy properties alone.

builtins.placeholder:

Return a placeholder string for the specified output that will be substituted by the corresponding output path at build time. Typical outputs would be "out", "bin" or "dev".

So presumably:

builtins.placeholder "foo" = "/nix/store/g2dwvn98qciaj087nf82xn99qidhv87m-my-hello-1.0-foo"

But I don’t really understand how this works – what if I try to call this function outside of a derivation? How does… what does that mean? There’s no type signature; I must be misunderstanding what it’s saying.

I wish we had gone over, like, how to evaluate Nix expressions before all this. It would be nice if I had a repl that I could use, but I don’t know how to do that.

File system stuff: builtins.pathExists, builtins.readDir, builtins.readFile. And builtins.toFile! That’s cool.

Store the string s in a file in the Nix store and return its path.

One day I hope to understand how side effects work in this “pure” language. Is this like… is this another kind of derivation? I have no idea.

The manual has a nice example here of using this to “inline” a file:

{ stdenv, fetchurl, perl }:

stdenv.mkDerivation {
  name = "hello-2.1.1";

  builder = builtins.toFile "builder.sh" "
    source $stdenv/setup

    PATH=$perl/bin:$PATH

    tar xvfz $src
    cd hello-*
    ./configure --prefix=$out
    make
    make install
  ";

  src = fetchurl {
    url = http://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  inherit perl;
}

Neat.

We have seq and deepSeq, with definitions that are… that you would have no hope of understanding if you did not already know what they did from Haskell.

builtins.toPath:

DEPRECATED. Use /. + "/path" to convert a string into an absolute path. For relative paths, use ./. + "/path".

Is that really better than toPath?

toJSON/fromJSON/toXML. No fromXML. Shocking.

builtins.trace e1 e2:

Evaluate e1 and print its abstract syntax representation on standard error. Then return e2. This function is useful for debugging.

Neat; that’s useful.

builtins.tryEval shallowly evaluates expressions and tells you if they throw. But it’s, you know, shallow, and needs to be combined with deepseq to do what you’d expect it to. This is… this is something that makes perfect sense to me, but I would need to write like a page of words to explain it, and I am tired. I feel like there is very little chance that normal humans ever needs to use tryEval or seq or deepSeq.

Huh; I’m actually curious about that.

I rg tryEval and do not find it used in any actual package files – just in scripts and helpers. So it seems like something Nixpkgs maintainers need to think about, but not Nix package maintainers. Good. Similar for deepSeq. seq seems to get a fair bit of use, but I don’t look into it any further.

That seems like a good place to stop. It’s good to look through and get a sense for what’s out there.

I wonder if this is exhaustive – I still have not learned how to like… just evaluate arbitrary Nix expressions, without writing a package to do it. But it would be nice to be able to just dump builtins and see for myself what’s in there.

Sigh fine. It looks like nix repl can do this. I fire it up.

$ nix repl
Welcome to Nix version 2.3.10. Type :? for help.

nix-repl> builtins
{ abort = «primop»; add = «primop»; addErrorContext = «primop»; all = «primop»; any = «primop»; appendContext = «primop»; attrNames = «primop»; attrValues = «primop»; baseNameOf = «primop»; bitAnd = «primop»; bitOr = «primop»; bitXor = «primop»; builtins = { ... }; catAttrs = «primop»; compareVersions = «primop»; concatLists = «primop»; concatMap = «primop»; concatStringsSep = «primop»; currentSystem = "x86_64-darwin"; currentTime = 1615431925; deepSeq = «primop»; derivation = «lambda @ /nix/store/q0z2kvkgrpvaipa87jl98qh7g5pym5fj-nix-2.3.10/share/nix/corepkgs/derivation.nix:4:1»; derivationStrict = «primop»; dirOf = «primop»; div = «primop»; elem = «primop»; elemAt = «primop»; false = false; fetchGit = «primop»; fetchMercurial = «primop»; fetchTarball = «primop»; fetchurl = «primop»; filter = «primop»; filterSource = «primop»; findFile = «primop»; foldl' = «primop»; fromJSON = «primop»; fromTOML = «primop»; functionArgs = «primop»; genList = «primop»; genericClosure = «primop»; getAttr = «primop»; getContext = «primop»; getEnv = «primop»; hasAttr = «primop»; hasContext = «primop»; hashFile = «primop»; hashString = «primop»; head = «primop»; import = «primop-app»; intersectAttrs = «primop»; isAttrs = «primop»; isBool = «primop»; isFloat = «primop»; isFunction = «primop»; isInt = «primop»; isList = «primop»; isNull = «primop»; isPath = «primop»; isString = «primop»; langVersion = 5; length = «primop»; lessThan = «primop»; listToAttrs = «primop»; map = «primop»; mapAttrs = «primop»; match = «primop»; mul = «primop»; nixPath = [ ... ]; nixVersion = "2.3.10"; null = null; parseDrvName = «primop»; partition = «primop»; path = «primop»; pathExists = «primop»; placeholder = «primop»; readDir = «primop»; readFile = «primop»; removeAttrs = «primop»; replaceStrings = «primop»; scopedImport = «primop»; seq = «primop»; sort = «primop»; split = «primop»; splitVersion = «primop»; storeDir = "/nix/store"; storePath = «primop»; stringLength = «primop»; sub = «primop»; substring = «primop»; tail = «primop»; throw = «primop»; toFile = «primop»; toJSON = «primop»; toPath = «primop»; toString = «primop»; toXML = «primop»; trace = «primop»; true = true; tryEval = «primop»; typeOf = «primop»; unsafeDiscardOutputDependency = «primop»; unsafeDiscardStringContext = «primop»; unsafeGetAttrPos = «primop»; valueSize = «primop»; }

My builtins has 105 things in it, many of which are not documented. Most of these stringify to «primop» which I assume means “primitive operation,” i.e. actual language builtins. But not all of them!

Let’s look at just the things that are interesting – I’ll filter out primops that the manual already covered. And we are left with:

addErrorContext = «primop»;
appendContext = «primop»;
catAttrs = «primop»;
currentTime = 1614488750; # this is real; i did not change this
derivation = «lambda @ /nix/store/q0z2kvkgrpvaipa87jl98qh7g5pym5fj-nix-2.3.10/share/nix/corepkgs/derivation.nix:4:1»;
derivationStrict = «primop»;
false = false;
fetchMercurial = «primop»;
findFile = «primop»;
fromTOML = «primop»;
genericClosure = «primop»;
getContext = «primop»;
hasContext = «primop»;
hashString = «primop»;
langVersion = 5;
mapAttrs = «primop»;
nixPath = [ ... ];
nixVersion = "2.3.10";
null = null;
partition = «primop»;
scopedImport = «primop»;
storeDir = "/nix/store";
storePath = «primop»;
true = true;
unsafeDiscardOutputDependency = «primop»;
unsafeDiscardStringContext = «primop»;
unsafeGetAttrPos = «primop»;
valueSize = «primop»;

Kind of a surprising number of things! Some of these are pretty self explanatory. No idea what “context” is, though. Some of these seem to be helpers that were just not added to the manual when they were added to the language. Or maybe these are deprecated, and that’s why they aren’t documented? But other deprecated functions are documented. Maybe these are more deprecated. I don’t know.

I narrow this down to the following list of “interesting” functions – things that do not seem self explanatory.

addErrorContext = «primop»;
appendContext = «primop»;
derivation = «lambda @ /nix/store/q0z2kvkgrpvaipa87jl98qh7g5pym5fj-nix-2.3.10/share/nix/corepkgs/derivation.nix:4:1»;
derivationStrict = «primop»;
findFile = «primop»;
genericClosure = «primop»;
getContext = «primop»;
hasContext = «primop»;
nixPath = [ ... ];
scopedImport = «primop»;
storePath = «primop»;
unsafeDiscardOutputDependency = «primop»;
unsafeDiscardStringContext = «primop»;
unsafeGetAttrPos = «primop»;
valueSize = «primop»;

Obviously I know derivation, but the fact that it’s a lambda and not a built-in is super interesting to me. I look in that file, and the comment explains:

/* This is the implementation of the ‘derivation’ builtin function.
   It's actually a wrapper around the ‘derivationStrict’ primop. */

Who puts fancy quotes in source code? That’s crazy. Get outta here.

nix-repl> builtins.getContext {}
error: value is a set while a string was expected, at (string):1:1

nix-repl> builtins.getContext ""
{ }

nix-repl> builtins.getContext "hello"
{ }

Hmmm. No idea.

nix-repl> builtins.appendContext "hello"
«primop-app»

nix-repl> builtins.appendContext "hello" "context"
error: value is a string while a set was expected, at (string):1:1

nix-repl> builtins.appendContext "hello" { context = "context"; }
error: Context key 'context' is not a store path, at 0x7fc1b3006f48

Okay; still no idea. I try genericClosure, and am able to follow the type errors long enough to get something that does nothing:

nix-repl> :p builtins.genericClosure { startSet = [{ key = "value"; }]; operator = (x: [x x]); }
[ { key = "value"; } ]

No idea. I give up on that one.

nix-repl> builtins.nixPath
[ { ... } { ... } ]

nix-repl> :p builtins.nixPath
[ { path = "/Users/ian/.nix-defexpr/channels"; 
    prefix = ""; }
  { path = "/nix/store/q0z2kvkgrpvaipa87jl98qh7g5pym5fj-nix-2.3.10/share/nix/corepkgs";
    prefix = "nix"; } ]

(Formatting mine, so you won’t have to scroll as much.)

Okay. I was kind of expecting just a list of strings. That doesn’t match my NIX_PATH. What is prefix? I have no idea.

nix-repl> builtins.valueSize 1
24

nix-repl> builtins.valueSize true
24

nix-repl> builtins.valueSize 1.5
24

nix-repl> builtins.valueSize ""
25

nix-repl> builtins.valueSize "a"
26

nix-repl> builtins.valueSize {}
269296

nix-repl> builtins.valueSize { a = 1; }
269296

Okay. I assume values have 16 bytes of header, and ints/floats are each 8 bytes, because those are nice round numbers. I assume strings are null-terminated. 269296 is a big ol' number. I do not worry further.

I can’t figure out anything else. Hopefully those functions aren’t too important?

Why doesn’t fetchGit return a fixed-output derivation?
Does toString have the same behavior as Nix’s stringification of derivation attributes?
What is builtins.placeholder?
What is “context”?
What do the unsafe functions do?
What is scopedImport?

How to Learn Nix, Part 14:Built-in Functions

15.5. Built-in Functions

How to Learn Nix, Part 14:
Built-in Functions