I’ve been using Nix for a few months now, and one of my largest outstanding pain points is that any time I’m in a nix-shell, none of my stuff works the way I want it to.

On a scale from “uses the stock PS1” to “wrote their own shell,” I’m prettttty far to the right. I haven’t actually written my own shell, but, you know, I’ve started writing my own shell, and I have lots of aliases and a fancy custom fzf-based autocomplete thing and weird tmux integrations and strong feelings about how my shell should behave.

And I cannot tell you the number of times I have seen this error message:

I opened a PR a few days ago to try to fix the issue I ran into with nix-env -u trying to upgrade to a weird alpha pre-release version of Python.

In my head, I was making the world a better place. It was an issue that bit me; it was an issue that bit someone who emailed me; it seemed like an issue worth fixing. But the maintainer of Python responded to that PR with:

I don’t think we should be doing this. Yes, it resolves the issue for python3, but there are others out there as well. “It is known” that nix-env -i should not be used.

My gut reaction to that comment was: well, sure, but we shouldn’t let perfect be the enemy of good. python3 is such a common, important package! Who cares if there are some random little packages sprinkled here and there that have the same problem. It’s worth fixing for big packages like Python.

But this person knows a lot more about Nix than I do. And a lot more about Nixpkgs. So if they believe that this is not worth fixing, there is a very good chance that this is not worth fixing, regardless of what my gut has to say about it.

So: let’s find out? Let’s look into this, and figure out how widespread this issue actually is.

I had assumed that it was a rare issue, because python3 is the only problematic package I’ve encountered, and I have quite a few packages installed in my environment at this point. Add to that the fact that it’s a relatively recent problem – introduced some time between NixOS 20.09 and NixOS 21.05 – and it seems like this isn’t typical. python3 is a weird outlier, right?

Let’s find out!

Over the course of reading the Nix manual, reading the Nixpkgs manual, reading the Nix Pills, and performing my own experiments, I have accumulated quite a few “open questions.”

I leave them at the bottom of each post, and they help ease my mind: writing them down allows me to forget them. They are not lurking in the darker corners of my brain, distracting me as I try to go about my day. They will be there for me when I need them, and that is a calming thought.

Now, I’m not done learning Nix. But I have read all of the official documentation, at this point, and it is time to enter into a new chapter of my Nix journey: I am leaving the warm embrace of the manuals, and setting off on my own. And what better place to start by going through my open questions?

All 160 of them.

So this isn’t great:

$ nix-env -i python3
installing 'python3-3.10.0a5'

Why isn’t this great? Well, because in the Python versioning scheme, python3-3.10.0a5 is short for “Python 3.10.0 alpha version 5.” The latest stable release of the Python reference implementation, right now, is actually 3.9.5. 3.10 is currently considered the “pre” branch, and 3.11 is currently the “dev” branch. I think; I’m not really a Python person.

So it’s not great that a new Nix user might try to install python3 and wind up with some random alpha version of Python (the latest in the 3.10 branch is actually 3.10.0b2, for whatever that’s worth).

But it’s even less great because – at the time that I am writing these words right now – the python3-3.10.0a5 derivation is broken, and will not build.

I don’t know about the word “pills.”

The word “pills” makes me think of medicine; it makes me think of the idiom “a bitter pill to swallow;” it makes me think of sweaters that just came out of the wash. It does not make me think of… documentation, or learning.

I assume that the title is meant to evoke “bite-sized” or something like that. I don’t know. It’s weird a weird choice, for a series.

One of my very first open questions, all the way back in the quick start guide, was this:

How do I collect garbage without making my next invocation of nix-shell -p hello redownload all the stuff it needs to be a shell?

Since the time I wrote that question, a lot has changed. I read the entire Nix manual. I read the entire Nixpkgs manual. I’ve been using Nix for months.

And I still have no idea what the answer is.

I read the Nixpkgs manual.

It took a long time. It was a very long manual.

And I learned a lot. But not quite as much as I wanted to.

Now I thought I’d do another little summary of my thoughts and feelings, for myself and others.

This is it. We’ve reached the end. After a month and a half of reading the Nixpkgs manual in excruciating detail, I have finally defeated it.

Or, I’m about to defeat it. We’re about to defeat it. We’re going to do this together.

Chapter 16 is a curious beast.

It is a list of 17 packages, and information about how to use or configure them.

Some of them make a lot of sense to me: Nginx, Emacs, X.org. Complicated packages that, I imagine, there is a lot of configuring to do.

Some of them do not make sense to me. Some of them I have never heard of. ibus-engines.typing-booster. I don’t really think of that as on the same level as, you know, Eclipse. “Cataclysm: Dark Days Ahead.” That’s just… a game. It is the only game listed. Look, I do not know. I’m not sure what the inclusion criteria here is, or if it’s anything beyond “someone put it here.”

Here’s the complete list:

Okay, we’re going to talk about Chapter 15 now.

Except that, by one crude estimate, Chapter 15 accounts for more than one third of the entire Nixpkgs manual.

It’s really 26 chapters in one, although they aren’t called chapters, they’re just called sections.

It’s a very, very long chapter.

So… we’re only actually going to talk about parts of Chapter 15.

Yes, I took a long break from reading Nix documentation.

But I did not give up on Nix, just as Nix has not given up on me. I’ve simply been busy, and finding myself less intrinsically motivated to read the Nixpkgs manual than I was when I was reading the Nix manual.

I know you were worried. The number of messages I got encouraging me to keep going, not to give up when I’m so close to the home stretch – it was almost nonzero.

So here we go. Let’s get right back into it.

You know, this is a really short chapter.

And it’s the last chapter of Part II.

And the last five posts have taken multiple days to write, and have all come out far too long for any human to reasonably read.

And the mess that was cross-compilation left me kind of drained, and not very excited to keep reading.

So I’m gonna take it easy this time.

Back in Chapter 6, we got a sort of whirlwind introduction to cross-compilation.

No; perhaps “introduction” is the wrong word. It was more of a detailed description of an algorithm used somewhere within the dependency resolution phase of cross-compilation, presented without context or motivation, in a section that mostly left us with more questions than answers.

So hopefully now we’ll get an actual introduction.

The next few chapters take place in the standard environment “part,” but we have left the standard environment “chapter.” I suppose that that these are details only relevant to stdenv.mkDerivation? Or I shouldn’t read too much into the grouping here.

Okay. I remember stdenv. I remember it adding a lot of nice things to my PATH in a builder script, although I couldn’t tell you what exactly it included. I remember that it absolutely insists that you have an output named out or dev, and if you don’t (because, say, the example in the manual didn’t) you’ll get a horrible error message from deep in the bowels of some shell script.

Let’s see if we can get more specific.

Okay. We aren’t actually done with Chapter 5 yet. We got through the bulk of it, but we still have a few short sections to go.

Oh gosh. I’ve been really putting off chapter five because, well, because look at it. It’s massive. It’s intimidating.

But as I recall from the Nix manual, the reference sections do not make the most exciting reading, but they are usually very educational. So I can’t just skip them.

Even if I really want to skip them.

We’ve had two chapters covering two different ways to override packages, but we still haven’t actually been able to override any packages. But this chapter is about overriding, so maybe it will tell us how to use the features from chapters 2 and 3?

This is a very curious presentation order.

This is my first encounter with the term “overlays.” The manual describes it thus:

Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages.

Any questions?

This is a very long guide about setting up tmux “from scratch,” which is to say without any of the default keybindings or any default behavior. It’s mostly an excuse to demonstrate a simple ~/.tmux.conf, and to explain how you can set up tmux to do exactly what you want it to, and nothing more.

I like this, because it turns my ~/.tmux.conf into documentation about what my tmux can do, and how it works. If something isn’t in that file, it doesn’t exist. If I forget how to do something, I can just look at that file. I don’t need to remember any tmux defaults; I don’t need anything except my little file.

This is a fancy thing that iTerm can do, somewhat invasively, and Terminal.app can do, somewhat transparently, and if you already know how to do this then just keep doing your thing and don’t worry about it.

This post is going to show you how to do it in tmux, because tmux is great and it’s not coupled to your choice of terminal emulator and it’s a cool superpower to add to your toolkit.

But it’s not trivial to do in tmux. It took me a while to figure it out. Which is why you’re looking at a blog post, and not a code snippet.

Look, you already know that tmux lets you split your terminals or whatever. You know it lets you maintain remote sessions like a supercharged nohup. Neither of these features are very interesting, if you’re using a modern terminal emulator (or a tiling window manager) and doing your development locally.

So you’ve never tried tmux. People raved about it, but you didn’t listen: you’re not in the target audience. tmux is not for you.

I am projecting, obviously: that was my impression of tmux, a few years ago. Before I tried it. Before I learned how wrong I was.

Well, I took a bit of a break from learning Nix, but I’m back to continue my journey through the documentation.

I already read the whole Nix manual, so now it’s time read through the Nixpkgs manual. Wish me luck.

Given the contents of my bar:

$ cat facts/bar
brandy
dry vermouth
lemon
light rum
lime
orange liqueur
reposado tequila
sugar

And my cocktail recipe book:

$ head facts/recipes
martini <- london dry gin
martini <- dry vermouth
daiquiri <- light rum
daiquiri <- lime juice
daiquiri <- simple syrup
margarita <- blanco tequila or reposado tequila
margarita <- lime juice
margarita <- orange liqueur
margarita <- lime wedge
...

What cocktails can I mix?

$ cat results/mixable
daiquiri
margarita
sidecar

Gosh, that’s not very many. What could I add to my bar to expand my options?

$ cat results/shopping-list
london dry gin -> gimlet
london dry gin -> martini
champagne -> airmail
cognac -> between-the-sheets
sherry -> sherry-cobbler
rhum agricole -> ti-punch

We open on a chat conversation that I had with my good friend and former colleague “Doug,” in which we were discussing my experience reading through the Nix manual.

 ian: there were certain things that i wanted to learn how to do
 ian: and i still don't really know how to do them
 ian: like
 ian: it seems like
 ian: you should just have a file
 ian: called "ian.nix"
 ian: where you write down the things you want
 ian: and then there should be a way to say "install exactly this set
      of things"
 ian: but there isn't
 ian: so i still want to figure out how to write that
 ian: anyway

It’s been several days since I reinstalled a bunch of stuff with Nix instead of Homebrew, and so far everything has been fine. I mean, the software I installed, like, works the same. As far as I can tell.

But I realize that I should probably update my packages, and see if there are any juicy new bytes to keep my hard drive entertained.

I really didn’t expect that this would be worthy of a blog post but, well, here we are: my first upgrade. Attempt.

In my last post, I tried to install Mercurial, and it didn’t go well.

Solemn music plays over this black and white flashback:

I don’t feel like I know enough about Nix to manage my own build dependencies with it, but I do feel like I know enough to use Nix as a simple package manager. I use Homebrew on my laptop, and at this point I definitely know more about Nix than I do about Homebrew.

Why would I switch away from Homebrew? Mostly because I think it will be educational. I don’t really have anything against Homebrew. It works. It has a lot of the packages I want. It does break more often than I’d like – not, like, often, but any time I go more than a year or so without using a particular computer I know that I’m going to return to some bizarre Ruby error that I have to spend half an hour figuring out before I can do anything.

And there’s no way that Nix would ever play me like that.

Now that I’ve reached the end of the manual, I want to reflect a little bit on my journey so far.

What have I learned?

I have learned a lot, really.

But I feel like I haven’t learned enough to like… use Nix.

Let’s talk about nix.conf.

This is another reference section – seems like we’re going through all the options. I’m going to read this list and make notes in anything that sounds interesting.

Oooookay.

I said I was gonna read the manual, right? I’ll do it. I’ll read it. I already read all those functions, didn’t I? It’s time for Part VI: Command Reference.

~/sd is my script directory. It looks like this:

$ tree ~/sd
/Users/ian/sd
├── blog
│   ├── preview
│   └── publish
├── book
│   ├── open
│   ├── pdf
│   ├── progress
│   └── typeset
├── cat
├── dim
│   └── visualize
├── edit
├── git
│   └── ugh
├── help
├── new
├── nix
│   ├── diff
│   ├── info
│   └── sync
├── tmux
│   ├── init
│   └── restore
└── which

sd is a command on my PATH. It dispatches its arguments to my script directory.

$ sd blog publish

Will run the script ~/sd/blog/publish.

This sounds like a dumb way to save myself having to type a few characters. Which it is. But it’s also slightly more than that:

Oh gosh oh gosh. I don’t know if I’m ready.

It’s time for Part V: Advanced Topics.

Ooooookay. This is a really, really long section. This is like… a reference. Dang. Okay.

The section we’ve all been waiting for…

Okay! Chapter 15 is finally going to teach us the language we’ve been using for the past several chapters. I’m excited.

The Nix expression language is a pure, lazy, functional language.

I didn’t know it was a pure language.

We got to Part IV! We have learned everything we need to know about Package Management and now get to learn about Writing Nix Expressions.

I have to point out that this “part” basically starts out with a link to a whole other manual. My goodness. I don’t know if I can type up every thought that goes through my head as I read two manuals. I’ve gotta sleep at some point.

So apparently you can move packages around from one machine to another very easily.

Neat? I guess?

I turn my eyes to Chapter 12, and learn:

A Nix channel is just a URL that points to a place that contains a set of Nix expressions and a manifest.

Neat. What does that mean?

I remember from using Nix long ago that when you “uninstall” a package, you don’t really “uninstall” it – you just remove the symlinks to it in your ~/.nix-profile/bin directory, so it no longer appears on your PATH.

Now I know that it doesn’t “remove symlinks” but rather “builds a new user environment and changes the ~/.nix-profile symlink to point to the new generation’s symlink to that user environment.” But you know, effectively the same thing. So there is a separate operation to actually delete those files, and remove the old generations.

Let’s find out more.

Chapter 10 describes the directory structure of ~/.nix-profile, and explains how rollbacks work.

The chapter starts with a sort of complicated drawing of multi-user directories with like default profiles and custom ones and it’s way too much for me to follow and I don’t really care. I think it’s easier to just look at some examples on my current system, and imagine how I would generalize them to multiple users.

Part III deals with using Nix as a package manager. The quick start guide already gave us a brief introduction to this, but now we get to go into detail.

I start reading through the Nix manual. I won’t quote it in its entirety like I did in the glossary. But I will any time I want to highlight something or I have a question or I don’t understand something.

I decide to start learning Nix by reading through the glossary. I don’t expect to fully understand every definition – I expect it will take first-hand experience to really understand these terms – but I think it will be a useful primer before I read the documentation. At least I will recognize when I encounter a Nix-specific term.

As I said in the introduction to this series, I’m not coming to Nix completely blind. I used Nix for a brief period five years ago, but I basically haven’t touched it since then.

So the first thing I did was try to write down everything I could remember about Nix, as I think that it’s pretty important context that will give me an advantage understanding some concepts. Mostly I did this for myself: it was nice to think back on what I had had trouble with back then, and what I wanted to make sure I understood this time around.

Five years ago, I migrated my VPS from Arch Linux to NixOS.

I had no idea what I was doing. I had never used Nix before. I thought I could figure it out.

Five years later, I’m still running NixOS. And I still have no idea how it works.

This is an adaptation of a talk that I gave a few times to graduates of the Flatiron School as part of Trello’s fellowship program.

This is a softer post than my “usual” fare. This post is not about a particular challenge or task; it’s about programming for the sake of programming. It’s a self-indulgent affair. You have been warned.

Look at this program:

main = do
  contents <- readFile "foo.txt"
  writeFile "foo.txt" ('a':contents)

What does it do?

This post started out as “Applicative Functors for Non-Haskell Developers.”

That was an ambitious title.

Instead, it’s turned out to be something more like “Applicative Functors for Curious Haskell Novices,” or, more accurately, “Applicative Functors for Ian Henry, Two Years Ago.” But that’s not as catchy.

Specifically, this post assumes the following prior Haskell knowledge:

We all know the absolute minimum every software developer must know about Unicode. We aren’t bad citizens of the internet, spreading malice and encoding errors willy-nilly around the world. We’re good people, just trying to get by.

But knowing the minimum is starting to bother us. And the more we think about UTF-8 and text encoding, the more we realize that we don’t “get it” at a messy, squishy level. We’ve read about it, sure, but we’ve never actually wrapped our hands around it and felt the blood pump through its veins.

So that’s what we’re going to do in this post: grab UTF-8 by the bytes and squeeze some real understanding out of it.

Let’s do something extremely simple:

We’ve got a view controller, it has a user, the user’s got a username, and we want to show that in our navbar title. With an @ in front, because that’s what all the cool kids do these days. Simple, right?

- (void)viewDidLoad {
    [super viewDidLoad];
    self.title = [NSString stringWithFormat:@"@%@", self.user.username];
}

Yep! That’s all it takes. As long as the username never changes.