refactor!: replace ZeroTier with Tailscale and Wireguard

- Tailscale UX is better, and the Headscale control server is also
  easier to self-host than ZeroTier (although Headscale is not the
  official control server, the author now works at Tailscale)
- Wireguard is also added as an alternative to avoid relying on a
  third-party service, however it requires port-forwarding

README.md

@@ -48,7 +48,7 @@ More information can be found in [the roadmap](#roadmap) below.
 - [x] Modular architecture, easy to add or remove features/components
 - [x] Automated certificate management
 - [x] Automatically update DNS records for exposed services
-- [x] VPN without port forwarding
+- [x] VPN (Tailscale or Wireguard)
 - [x] Expose services to the internet securely with [Cloudflare Tunnel](https://www.cloudflare.com/products/tunnel/)
 - [x] CI/CD platform
 - [x] Private container registry
@@ -207,16 +207,21 @@ They can't capture all the project's features, but they are sufficient to get a
         <td><a href="https://rook.io">Rook Ceph</a></td>
         <td>Cloud-Native Storage for Kubernetes</td>
     </tr>
+    <tr>
+        <td><img width="32" src="https://avatars.githubusercontent.com/u/48932923?s=200&v=4"></td>
+        <td><a href="https://tailscale.com">Tailscale</a></td>
+        <td>VPN without port forwarding</td>
+    </tr>
+    <tr>
+        <td><img width="32" src="https://avatars.githubusercontent.com/u/13991055?s=200&v=4"></td>
+        <td><a href="https://www.wireguard.com">Wireguard</a></td>
+        <td>Fast, modern, secure VPN tunnel</td>
+    </tr>
     <tr>
         <td><img width="32" src="https://avatars.githubusercontent.com/u/84780935?s=200&v=4"></td>
         <td><a href="https://woodpecker-ci.org">Woodpecker CI</a></td>
         <td>Simple yet powerful CI/CD engine with great extensibility</td>
     </tr>
-    <tr>
-        <td><img width="32" src="https://docs.zerotier.com/img/ZeroTierIcon.png"></td>
-        <td><a href="https://zerotier.com">ZeroTier</a></td>
-        <td>VPN without port forwarding</td>
-    </tr>
     <tr>
         <td><img width="32" src="https://zotregistry.dev/v2.0.2/assets/images/logo.svg"></td>
         <td><a href="https://zotregistry.dev">Zot Registry</a></td>

apps/homepage/values.yaml

@@ -142,9 +142,6 @@ app-template:
             - Terraform Cloud:
               - href: https://app.terraform.io
                 icon: terraform.svg
-            - Zerotier:
-              - href: https://my.zerotier.com
-                icon: zerotier.svg
           - Infrastructure:
             - Router:
               - href: https://192.168.1.1

docs/getting-started/vpn-setup.md

@@ -0,0 +1,61 @@
+# VPN setup
+
+You can choose between [Tailscale](https://tailscale.com), [Wireguard](https://www.wireguard.com), or use both like me.
+
+## Tailscale (requires third-party account)
+
+Get an [auth key](https://tailscale.com/kb/1085/auth-keys) from [Tailscale admin console](https://login.tailscale.com/admin/authkeys):
+
+- Description: homelab
+- Reusable: optionally set this to true
+
+Add it to `external/terraform.tfvars` as an extra secret:
+
+```hcl
+extra_secrets = {
+  tailscale-auth-key = "tskey-auth-myauthkeyhere"
+}
+```
+
+Apply the secret:
+
+```sh
+make external
+```
+
+Finally, [enable subnet routes](https://tailscale.com/kb/1019/subnets#step-3-enable-subnet-routes-from-the-admin-console) for `homelab-router`
+from the [admin console](https://login.tailscale.com/admin/machines).
+
+You can now connect to your homelab via Tailscale and [invite user to your Tailscale network](https://tailscale.com/kb/1371/invite-users).
+
+## Wireguard (requires port-forwarding)
+
+Update the peer list in `apps/wireguard/values.yaml`:
+
+```yaml
+PEERS: |
+  UserDevice
+  FooPhone
+  FooLaptop
+  BarDesktop
+```
+
+Go to your router settings and forward the Wireguard service.
+Each router is different, here's mine for reference:
+
+- Protocol: `UDP`
+- Start Port: `51820`
+- End Port: `51820`
+- Local IP Address: `192.168.1.226` (find it with `kubectl get service -n wireguard wireguard`)
+- Start Port Local: `51820`
+- End Port Local: `51820`
+
+To get the QR code (for mobile) and config (for desktop), run:
+
+!!! warning
+
+    This command will print sensitive secrets to the terminal.
+
+```sh
+./scripts/get-wireguard-config FooPhone
+```

docs/installation/production/external-resources.md

@@ -12,7 +12,6 @@ Below is a list of external resources and why we need them (also see some [alter
 | Terraform Cloud | Workspace       | Terraform state backend                                                                                                     |
 | Cloudflare      | DNS             | DNS and [DNS-01 challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) for certificates                 |
 | Cloudflare      | Tunnel          | Public services to the internet without port forwarding                                                                     |
-| ZeroTier        | Virtual network | Use as VPN to access home network from anywhere (with [UDP hole punching](https://en.wikipedia.org/wiki/UDP_hole_punching)) |
 | ntfy            | Topic           | External notification service to receive alerts                                                                             |
 <!-- | Minio           | Bucket     | Onsite backup                                                                                               | -->
 <!-- | AWS             | S3 Glacier | Offsite backup                                                                                              | -->
@@ -51,11 +50,6 @@ If you decide to use a [different Terraform backend](https://www.terraform.io/la
 <!-- └── Is in - 117.xxx.xxx.xxx, 2402:xxx:xxx:xxx:xxx:xxx:xxx:xxx -->
 <!-- ``` -->
 
-### ZeroTier
-
-- Create a ZeroTier account <https://my.zerotier.com>
-- Generate a new API Token at <https://my.zerotier.com/account>
-
 <!-- ### Create Minio keys -->
 
 <!-- TODO: skip this for now -->
@@ -82,11 +76,6 @@ To avoid vendor lock-in, each external provider must have an equivalent alternat
     - Create a small VPS in the cloud and utilize Wireguard to route traffic via it
     - Access everything via VPN
     - See also [awesome tunneling](https://github.com/anderspitman/awesome-tunneling)
-- ZeroTier virtual network:
-    - [Host your own ZeroTier](https://docs.zerotier.com/self-hosting/introduction)
-    - [Tailscale](https://tailscale.com) (closed source, but you can use [Headscale](https://github.com/juanfont/headscale) to host your own Tailscale control server)
-    - [Netmaker](https://www.netmaker.org) (there's no hosted version, you'll need to host your own server)
-    - Wireguard server (requires port forwarding)
 - ntfy:
     - [Self-host your own ntfy server](https://docs.ntfy.sh/install)
     - Any other [integration supported by Grafana Alerting](https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/integrations/#list-of-supported-integrations)

external/main.tf

@@ -5,14 +5,6 @@ module "cloudflare" {
   cloudflare_api_key    = var.cloudflare_api_key
 }
 
-module "zerotier" {
-  source                 = "./modules/zerotier"
-  zerotier_central_token = var.zerotier_central_token
-  bridged_routes = [
-    "192.168.1.224/27"
-  ]
-}
-
 module "ntfy" {
   source = "./modules/ntfy"
   auth   = var.ntfy

external/modules/zerotier/main.tf

@@ -1,57 +0,0 @@
-locals {
-  router_ip = cidrhost(var.managed_route, 1) # Use the second IP in the VPN subnet as the router
-}
-
-resource "zerotier_network" "network" {
-  name        = var.name
-  description = var.description
-  private     = true
-
-  route {
-    target = var.managed_route
-  }
-
-  dynamic "route" {
-    for_each = var.bridged_routes
-
-    content {
-      target = route.value
-      via    = local.router_ip
-    }
-  }
-
-  assignment_pool {
-    start = cidrhost(var.managed_route, 0)
-    end   = cidrhost(var.managed_route, -1)
-  }
-}
-
-resource "zerotier_identity" "router" {}
-
-resource "zerotier_member" "router" {
-  network_id              = zerotier_network.network.id
-  name                    = "router"
-  member_id               = zerotier_identity.router.id
-  allow_ethernet_bridging = true
-  no_auto_assign_ips      = true
-  ip_assignments = [
-    local.router_ip
-  ]
-}
-
-resource "kubernetes_secret" "router" {
-  metadata {
-    name      = "zerotier-router"
-    namespace = "zerotier"
-
-    annotations = {
-      "app.kubernetes.io/managed-by" = "Terraform"
-    }
-  }
-
-  data = {
-    ZEROTIER_NETWORK_ID      = zerotier_network.network.id
-    ZEROTIER_IDENTITY_PUBLIC = zerotier_identity.router.public_key
-    ZEROTIER_IDENTITY_SECRET = zerotier_identity.router.private_key
-  }
-}

external/modules/zerotier/variables.tf

@@ -1,34 +0,0 @@
-variable "zerotier_central_url" {
-  description = "ZeroTier Central API endpoint"
-  type        = string
-  default     = "https://my.zerotier.com/api" # https://github.com/zerotier/go-ztcentral/blob/4d397d1e82c043a5376789177ad55536044d69ce/client.go#L44
-}
-
-variable "zerotier_central_token" {
-  description = "ZeroTier Central API Token"
-  type        = string
-  sensitive   = true
-}
-
-variable "name" {
-  description = "Network name"
-  type        = string
-  default     = "homelab"
-}
-
-variable "description" {
-  description = "Network description"
-  type        = string
-  default     = "Homelab network"
-}
-
-variable "managed_route" {
-  description = "ZeroTier managed route"
-  type        = string
-  default     = "10.147.17.0/24"
-}
-
-variable "bridged_routes" {
-  description = "List of bridged routes" # TODO
-  type        = list(string)
-}

external/modules/zerotier/versions.tf

@@ -1,13 +0,0 @@
-terraform {
-  required_providers {
-    zerotier = {
-      source  = "zerotier/zerotier"
-      version = "~> 1.4.0"
-    }
-  }
-}
-
-provider "zerotier" {
-  zerotier_central_url   = var.zerotier_central_url
-  zerotier_central_token = var.zerotier_central_token
-}

external/namespaces.yml

@@ -13,4 +13,3 @@
         - global-secrets
         - k8up-operator
         - monitoring-system
-        - zerotier

external/terraform.tfvars.example

@@ -5,9 +5,6 @@ cloudflare_account_id = "foobarid"
 # https://dash.cloudflare.com/profile/api-tokens
 cloudflare_api_key = "foobarkey"
 
-# https://my.zerotier.com/account
-zerotier_central_token = "foobartoken"
-
 ntfy = {
   # https://ntfy.sh or your own instance
   url = "https://ntfy.sh"

external/variables.tf

@@ -11,10 +11,6 @@ variable "cloudflare_account_id" {
   type = string
 }
 
-variable "zerotier_central_token" {
-  type = string
-}
-
 variable "ntfy" {
   type = object({
     url   = string

mkdocs.yml

@@ -47,6 +47,7 @@ nav:
       - installation/production/deployment.md
     - installation/post-installation.md
   - Getting started:
+    - getting-started/vpn-setup.md
     - getting-started/user-onboarding.md
     - getting-started/install-pre-commit-hooks.md
   - Concepts:

platform/zerotier/deployment.yaml

@@ -1,65 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: zerotier-router
-  namespace: zerotier
-  labels:
-    app: zerotier-router
-spec:
-  selector:
-    matchLabels:
-      app: zerotier-router
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        app: zerotier-router
-    spec:
-      containers:
-        - name: zerotier-router
-          image: zerotier/zerotier:latest  # TODO use tag
-          command:
-            - "sh"
-            - "-c"
-          args: # TODO optimize this
-            - |
-              # TODO install this on upstream image?
-              apt-get install -y iptables
-
-              # TODO is there a better way to get the interface name?
-              export PHY_IFACE="$(ip route | grep ${POD_IP} | cut -d ' ' -f 3)"
-              export ZT_IFACE=zt0
-
-              # Override the default random interface name
-              mkdir -p /var/lib/zerotier-one
-              echo "${ZEROTIER_NETWORK_ID}=${ZT_IFACE}" >> /var/lib/zerotier-one/devicemap
-
-              iptables -t nat -A POSTROUTING -o $PHY_IFACE -j MASQUERADE
-              iptables -A FORWARD -i $PHY_IFACE -o $ZT_IFACE -m state --state RELATED,ESTABLISHED -j ACCEPT
-              iptables -A FORWARD -i $ZT_IFACE -o $PHY_IFACE -j ACCEPT
-
-              /entrypoint.sh "${ZEROTIER_NETWORK_ID}"
-          resources:
-            requests:
-              cpu: 100m
-              memory: 128Mi
-          env:
-            - name: POD_IP
-              valueFrom:
-                fieldRef:
-                  fieldPath: status.podIP
-          envFrom:
-            - secretRef:
-                name: zerotier-router
-          securityContext:
-            capabilities:
-              add:
-                - NET_ADMIN
-          volumeMounts:
-            - name: dev-net-tun
-              mountPath: /dev/net/tun
-      volumes:
-        - name: dev-net-tun
-          hostPath:
-            path: /dev/net/tun
-            type: CharDevice

platform/zerotier/kustomization.yaml

@@ -1,5 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - deployment.yaml

scripts/get-wireguard-config

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+set -eu
+
+PEER="${1}"
+
+export KUBECONFIG=./metal/kubeconfig.yaml
+
+kubectl -n wireguard exec -it deployment/wireguard -- /app/show-peer "${PEER}"
+kubectl -n wireguard exec -it deployment/wireguard -- cat "/config/peer_${PEER}/peer_${PEER}.conf"