headscale: move the tailnet to orbit.sol, route all DNS through pihole

Three connected changes, all triggered by the same outage.

base_domain leaves mgaction.town. That zone has a wildcard A+AAAA pointing
at neptun, and DNS wildcards match multi-label names, so
jupiter.hosts.mgaction.town resolved publicly to NEPTUN and Caddy proxied
to itself -- a silent loop rather than a lookup failure. Nesting the
tailnet inside the LAN domain as orbit.sol keeps the theme and resolves
unambiguously, since tailscale matches routes by longest suffix.

override_local_dns = true with pihole as the only global nameserver, so
roaming devices get ad blocking and .sol names off-LAN. With it false,
globalResolvers land in the netmap's FallbackResolvers, which a phone
with carrier DNS never consults. No public fallback is listed on purpose:
tailscale treats the list as a set, so a second entry would let queries
slip past the filter whenever mercury is slow. The cost is that mercury
is now a single point of failure for tailnet DNS.

neptun and mercury opt out individually. mercury would otherwise resolve
through itself. neptun must not depend on a Pi behind a domestic line to
renew the certificates for the control server every other node needs --
and it is circular besides, since tailscaled has to resolve
vpn.mgaction.town to connect at all. Instead neptun runs a dnsmasq stub
forwarding just orbit.sol to MagicDNS on 100.100.100.100, which tailscaled
answers whenever it is running regardless of --accept-dns. That resolves
jupiter live, so the hardcoded /etc/hosts pin is gone.

Also sets dns.nameservers.split explicitly: nixpkgs renders its own
dns.split option one level too high, but headscale reads
dns.nameservers.split (hscontrol/types/config.go:722) and so does
headplane, whose DNS page dies on the missing key with "Cannot convert
undefined or null to object". The module's option is dead as written.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -226,11 +226,24 @@ another way in.
 - mercury's own `resolv.conf` is deliberately public resolvers, not its own
   pihole (`resolveLocalQueries = false`, see `CLAUDE.md`) — so `.sol` names do
   not resolve *on mercury itself*. That is expected, not a fault.
-- LAN `.sol` names resolve on tailnet members only because headscale sets
+- **mercury is load-bearing for the whole tailnet's DNS.** headscale sets
-  `override_local_dns = false`. With upstream's default of `true`, every node's
+  `override_local_dns = true` with pihole as the only global nameserver, so
-  `resolv.conf` is replaced with MagicDNS and `.sol` returns NXDOMAIN
+  every node — including a phone on mobile data — resolves through it and gets
-  everywhere — along with losing ad blocking, since queries stop reaching
+  ad blocking and `.sol` names anywhere. The flip side is that mercury (or the
-  pihole at all.
+  home connection) going down costs name resolution on every device, not just
+  `.sol`. Recovery on a stranded device is turning Tailscale off.
+  There is deliberately no public fallback in `nameservers.global`: tailscale
+  treats that list as a set, so a second entry would let queries slip past the
+  filter whenever mercury is slow.
+  neptun and mercury opt out with `--accept-dns=false` — mercury because it
+  would otherwise resolve through itself, neptun because a public reverse
+  proxy must not depend on a Pi at home to renew its certificates.
+- Tailnet names are `*.orbit.sol`, LAN names are `*.sol`. Both work everywhere
+  on the tailnet because tailscale matches DNS routes by **longest suffix**, so
+  `orbit.sol` reaches MagicDNS even though everything else goes to pihole.
+  Never name a LAN host `orbit`: pihole's `address=/<host>.sol/<ip>` lines match
+  a name *and everything beneath it*, which would swallow the entire tailnet
+  zone.
 
 ## Adding a service
 

common.nix

@@ -47,7 +47,7 @@
     options = "--delete-older-than 30d";
   };
 
-  environment.systemPackages = with pkgs; [ vim git htop tmux curl ];
+  environment.systemPackages = with pkgs; [ vim git htop tmux curl wget ];
 
   # ---- Locale / firewall base ----
   time.timeZone = "Europe/Berlin";

hosts/mercury/configuration.nix

@@ -31,6 +31,12 @@
   networking.defaultGateway = { address = "10.0.0.1"; interface = "eth0"; };
   networking.nameservers = [ "1.1.1.1" "9.9.9.9" ];
 
+  # Never take the tailnet's DNS on THIS host: headscale points every node at
+  # pihole, which runs here — mercury would be resolving through itself. Keep
+  # the public resolvers above for the Pi's own lookups, exactly as the
+  # unbound resolveLocalQueries note in CLAUDE.md requires.
+  services.tailscale.extraUpFlags = [ "--accept-dns=false" ];
+
   # ---- pihole web admin password (from sops) ----
   # The pihole container reads FTLCONF_* env vars. Render an env file from the
   # sops secret and feed it to the container — password stays out of repo/store.

hosts/neptun/configuration.nix

@@ -43,6 +43,29 @@
   networking.defaultGateway6 = { address = "fe80::1"; interface = "eth0"; };
   networking.nameservers = [ "9.9.9.9" "1.1.1.1" "2620:fe::fe" ];
 
+  # ---- Local split-DNS stub ----
+  # neptun must NOT take the tailnet's DNS: headscale points every node at
+  # pihole on mercury, and making a public reverse proxy's name resolution
+  # depend on a Pi behind a domestic line would take ACME renewals — and so
+  # the certs for the control server every node needs — down with it. It is
+  # also circular, since tailscaled has to resolve vpn.mgaction.town to
+  # connect in the first place.
+  #
+  # So neptun opts out with --accept-dns=false and does its own split DNS.
+  # tailscaled still answers MagicDNS on 100.100.100.100 whenever it is
+  # running (--accept-dns only governs whether it rewrites resolv.conf), so
+  # dnsmasq forwards just the tailnet suffix there and everything else to the
+  # public resolvers above. jupiter's address is therefore resolved live and
+  # never pinned — nothing to update when the tailnet is rebuilt.
+  #
+  # resolveLocalQueries (default) points resolv.conf at 127.0.0.1 and feeds
+  # networking.nameservers to dnsmasq as upstreams via resolvconf.
+  services.tailscale.extraUpFlags = [ "--accept-dns=false" ];
+  services.dnsmasq = {
+    enable = true;
+    settings.server = [ "/orbit.sol/100.100.100.100" ];
+  };
+
   # firewall (enable + 22), caddy (80/443), tailscale (trust tailscale0 + join
   # headscale) come from ../../common.nix and ../../services/{caddy,tailscale}.nix.
 
@@ -63,24 +86,24 @@
 
   # ---- Audiobookshelf ----
   services.caddy.virtualHosts."abs.mgaction.town".extraConfig = ''
-    reverse_proxy http://jupiter.hosts.mgaction.town:8000
+    reverse_proxy http://jupiter.orbit.sol:8000
   '';
 
   # ---- Seerr ----
   services.caddy.virtualHosts."seerr.mgaction.town".extraConfig = ''
-    reverse_proxy http://jupiter.hosts.mgaction.town:5055
+    reverse_proxy http://jupiter.orbit.sol:5055
   '';
 
   # ---- Jellyfin ----
   services.caddy.virtualHosts."jellyfin.mgaction.town".extraConfig = ''
-    reverse_proxy http://jupiter.hosts.mgaction.town:8096
+    reverse_proxy http://jupiter.orbit.sol:8096
   '';
 
   # ---- Gitea WebUI ----
   # Gitea's web UI and HTTPS clones (services/dev/gitea.nix, HTTP_PORT 3000).
   # Its SSH side is the separate :2222 forward further down.
   services.caddy.virtualHosts."git.mgaction.town".extraConfig = ''
-    reverse_proxy http://jupiter.hosts.mgaction.town:3000
+    reverse_proxy http://jupiter.orbit.sol:3000
   '';
 
   # ---- Gitea SSH forward ----
@@ -95,7 +118,7 @@
     wantedBy = [ "multi-user.target" ];
     serviceConfig = {
       DynamicUser = true;
-      ExecStart = "${pkgs.socat}/bin/socat TCP-LISTEN:2222,fork,reuseaddr TCP:jupiter.hosts.mgaction.town:2222";
+      ExecStart = "${pkgs.socat}/bin/socat TCP-LISTEN:2222,fork,reuseaddr TCP:jupiter.orbit.sol:2222";
       Restart = "always";
     };
   };

services/vpn/headscale.nix

@@ -2,9 +2,10 @@
 
 # Headscale — self-hosted control server for the tailnet. Every host's
 # services/vpn/tailscale.nix points --login-server at https://vpn.mgaction.town
-# (this host). MagicDNS base_domain "hosts.mgaction.town" matches the
+# (this host). MagicDNS base_domain "orbit.sol" matches the
-# "jupiter.hosts.mgaction.town" names used in this repo's Caddy vhosts
+# "jupiter.orbit.sol" names used in this repo's Caddy vhosts
-# (e.g. hosts/neptun/configuration.nix) — don't change one without the other.
+# (hosts/neptun/configuration.nix) — changing base_domain means changing
+# those too, and re-pointing neptun's dnsmasq stub at the new suffix.
 #
 # TLS terminates at Caddy (see the host's configuration.nix); headscale
 # itself only listens on localhost.
@@ -17,18 +18,54 @@
       server_url = "https://vpn.mgaction.town";
 
       dns = {
-        base_domain = "hosts.mgaction.town";
+        # Deliberately OUTSIDE mgaction.town. That zone has a wildcard A+AAAA
-        nameservers.global = [ "1.1.1.1" "9.9.9.9" ];
+        # pointing at neptun, and DNS wildcards match multi-label names — so
+        # with base_domain = hosts.mgaction.town, `jupiter.hosts.mgaction.town`
+        # resolved publicly to NEPTUN and Caddy proxied to itself: a silent
+        # loop rather than a lookup failure.
+        #
+        # `.sol` is the LAN domain pihole serves, so this nests the tailnet
+        # inside it: planets sit on the LAN as jupiter.sol, and reach each
+        # other in orbit as jupiter.orbit.sol. Resolution is unambiguous
+        # because tailscale matches routes by LONGEST suffix, so orbit.sol
+        # goes to MagicDNS even when everything else funnels to pihole.
+        #
+        # Never give a LAN host the name `orbit`: pihole's
+        # `address=/<host>.sol/<ip>` lines match a name AND everything under
+        # it, so an `orbit` host would swallow this entire zone.
+        base_domain = "orbit.sol";
+        # pihole on mercury, over the tailnet — so every roaming device gets
+        # ad blocking and .sol names wherever it is, not just on the LAN.
+        # Deliberately NO public fallback: tailscale treats the list as a set,
+        # so adding 9.9.9.9 here would let queries slip past the filter
+        # whenever mercury is briefly slow. Strict blocking, at the cost of
+        # mercury being a single point of failure for tailnet DNS.
+        #
+        # ⚠️  A hardcoded tailnet address, so it changes if mercury re-enrols
+        #     — check `headscale nodes list` if DNS dies tailnet-wide.
+        nameservers.global = [ "100.64.0.7" ];
 
-        # Leave each client's own resolvers alone; only route base_domain to
+        # Must be set, and must be HERE rather than via the module's
-        # MagicDNS. Upstream defaults this to true, which replaces resolv.conf
+        # `dns.split` option. nixpkgs renders that option one level too high
-        # with 100.100.100.100 on every node — that silently breaks the LAN's
+        # (a sibling of `nameservers:`), but headscale reads
-        # `.sol` names (pihole on mercury serves those, and the global
+        # dns.nameservers.split (hscontrol/types/config.go:722) and so does
-        # nameservers above return NXDOMAIN for them) and takes ad blocking
+        # headplane. So the module's option is dead, and the missing key makes
-        # with it. It also makes a node's entire DNS depend on tailscaled
+        # headplane's DNS page die with
-        # being up, which is what forced --accept-dns=false onto neptun and
+        #   TypeError: Cannot convert undefined or null to object
-        # mercury individually.
+        # from Object.keys(config.dns.nameservers.split).
-        override_local_dns = false;
+        nameservers.split = { };
+
+        # Point every node's resolver at MagicDNS, which forwards on to the
+        # global nameserver above. That is the only way to get pihole onto a
+        # roaming device: with this false, globalResolvers land in the
+        # netmap's FallbackResolvers (hscontrol/types/config.go:826-830) and a
+        # phone with carrier DNS never consults them.
+        #
+        # The cost is that every node's DNS now depends on mercury and on the
+        # home connection, so mercury going down costs name resolution
+        # everywhere, not just `.sol`. neptun and mercury opt out of this
+        # individually with --accept-dns=false — see their configuration.nix.
+        override_local_dns = true;
       };
 
       # Authentik as the login provider, so `tailscale up --login-server ...`