Adventures with pf, nix darwin, and Tailscale on macOS Ventura

I recently obtained a bleeding edge M2 Ultra Mac Studio with 128GB unified RAM (affectionately named brick), mainly so I can feel like I'm participating in the dev explosion ongoing in the local LLM space. True to my interests, I have done practically zero LLM software development (I fixed a print statement in llama.cpp!) or language model training, but have spent an inordinate amount of time configuring and tweaking my new overpriced and overpowerful gadget to suit my technical eccentricities.

My most coveted feature has been a working remote desktop that is accessible securely from all of my devices, from anywhere in the world. (If I am out and about, I am not likely to need to work on anything tech related. Why do I want this power? Because it's basically magic.)

Now, it's not exactly hard to get this working. Just set up Tailscale and turn on Remote Management, right? (This actually works.)

However, when you've got a "cute" login password (because who really wants to type a serious password every time you reboot or need sudo? I rebooted probably 100 times configuring the firewall down the page) keeping that exposed to my home LAN made me anxious. You're just begging to be brute-forced by the malware on your shitty TP-Link router, off-brand smart bulb, or theoretical porn-addicted family-member who doesn't know how to use an ad-blocker. Unlike with ssh, there's no way to lock Remote Management (or File Sharing for that matter) behind a cryptographic key rather than your login password.

The reasonable thing to do is just use a stronger login password.

The next most reasonable thing to do is to block everything except your Tailscale VPN, so at least your terrible opsec is swept under the rug. Hopefully any would-be criminals will trip on the lumps.

That sounds nice and simple in theory, but the problem is that the macOS firewall world is a complicated place, especially for somebody who's green to Macs but is still determined to configure it with additional "help" from the nix programming language.

What follows is the configuration I ended up with that finally worked, after a night and a half of trying to grok the details of how macOS's firewalls, nix darwin, and launchctl all converge.

Configuration

Starting pf at launch

You can check if pf is running with $ sudo pfctl -s info. When it's running the output includes Status: Enabled.

First, you need to enable pf at boot. You need a LaunchDaemon. A LaunchDaemon runs as root during startup, whereas a LaunchAgent is user-level. In your darwin-configuration.nix, add:

launchd.daemons = {
	"pfctl" = {
		command = "/sbin/pfctl -e";
		serviceConfig = {
		  RunAtLoad = true;
		};
	 };
};

After a $ darwin-rebuild switch, you will have a /Library/LaunchDaemons/org.nixos.pfctl.plist.

What's great about this is that now you can use pf without having to mess with the GUI settings for firewall. In my case, I really didn't want the Application Level Firewall, because it conflicts with mosh, which is much nicer than pure ssh. (I can start a mosh session over tailnet to brick from my phone and switch to data with hardly an interruption. Another useless IT superpower.)

(While you can whitelist applications and binaries in the GUI, I don't think there's a reasonable way to do this with mosh installed via nix, which lives in a hashed derivation directory in /nix/store that changes frequently:)

Configuring pf

Make sure that the default /etc/pf.conf is being loaded by running $ sudo pfctl -s rules. You should see something like:

No ALTQ support in kernel
ALTQ related functions disabled
scrub-anchor "com.apple/*" all fragment reassemble
anchor "com.apple/*" all

(You would not believe how long it took me to get to the point where pf started at boot and actually read the config file.)

Next, edit /etc/pf.conf to add a reference to your custom "anchor":

...
load anchor "com.apple" from "/etc/pf.anchors/com.apple"

#
# My anchor
#
anchor "us.vczf.tailscale"
load anchor "us.vczf.tailscale" from "/etc/pf.anchors/us.vczf.tailscale"

The reverse domain name notation is probably just a convention to avoid namespace conflicts. If you're feeling creative, you can paint outside the lines.

In your anchor:

## Block all incoming traffic
block in all

## Allow loopback interface and Tailscale CGNAT IP range
pass in from { lo0 100.64.0.0/10 } to any

## Allow DHCP traffic to your router
#  Mine is 192.168.32.1, but check your network because it's
#  unlikely to be the same.
#
#  "quick" stops further processing.
pass in quick proto { udp } from 192.168.32.1 to any port { 67 68 }

## Allow DNS traffic to your router
pass in quick proto { tcp udp } from 192.168.32.1 to any port { 53 }

## Allow all outgoing traffic
pass out all

Check your configuration syntax with $ pfctl -vnf /etc/pf.conf.

Now, reboot and check if it works. Assuming you already have Tailscale set up, it should.

If it's not, you can also use something like $ sudo pfctl -a us.vczf.tailscale -s rules' to check that your anchor's rules have been loaded.

Adding your pf config to nix

Copy your pf.conf and your anchor to your ~/.nixpkgs/. Then add something like this to your darwin-configuration.nix:

environment.etc = {
	 "pf.conf" = {
		copy = true;
		source = ./pf.conf;
	 };

	 "pf.anchors/us.vczf.tailscale" = {
		copy = true;
		source = ./pf.anchors/us.vczf.tailscale;
	 };
};

It's important to use copy = true;, because symlinks kept interfering with pf loading the configuration for me. You'll also have to delete your /etc/pf.conf and anchor, otherwise $ darwin-rebuild switch will fail to place the files there.

Closing thoughts

The above is probably doing what it's supposed to. I've done some basic port scanning with nmap and ad-hoc testing with Screens 4 (LAN vs tailnet) and RealVNC. I don't really understand pf though, and GPT4 wrote half of my configs. You should test your config before assuming it's working.

~ vczf @ Sun Jul 30 04:35:52 EDT 2023