mirror of
https://github.com/khuedoan/homelab.git
synced 2025-07-04 15:27:52 +07:00
refactor(docs): migrate to mkdocs (#68)
* refactor(docs): migrate to mkdocs * More markdown * Admonitions
This commit is contained in:
Khue Doan
9
Makefile
9
Makefile
@ -34,3 +34,12 @@ docs:
|
|||||||
dev:
|
dev:
|
||||||
make -C metal cluster env=dev
|
make -C metal cluster env=dev
|
||||||
make -C bootstrap
|
make -C bootstrap
|
||||||
|
|
||||||
|
docs:
|
||||||
|
docker run \
|
||||||
|
--rm \
|
||||||
|
--interactive \
|
||||||
|
--tty \
|
||||||
|
--publish 8000:8000 \
|
||||||
|
--volume $(shell pwd):/docs \
|
||||||
|
squidfunk/mkdocs-material
|
||||||
|
|||||||
344
README.md
344
README.md
@ -1,10 +1,7 @@
|
|||||||
<div align="center">
|
|
||||||
|
|
||||||
# Khue's Homelab
|
# Khue's Homelab
|
||||||
|
|
||||||
<!-- ANCHOR: introduction -->
|
**[Features](#features) • [Get Started](#get-started) • [Documentation](https://homelab.khuedoan.com)**
|
||||||
|
|
||||||
[](https://matrix.to/#/#homelab:matrix.khuedoan.com)
|
|
||||||
[](https://github.com/khuedoan/homelab/tags)
|
[](https://github.com/khuedoan/homelab/tags)
|
||||||
[](https://homelab.khuedoan.com)
|
[](https://homelab.khuedoan.com)
|
||||||
[](https://www.gnu.org/licenses/gpl-3.0.html)
|
[](https://www.gnu.org/licenses/gpl-3.0.html)
|
||||||
@ -13,33 +10,34 @@
|
|||||||
This project utilizes [Infrastructure as Code](https://en.wikipedia.org/wiki/Infrastructure_as_code) and [GitOps](https://www.weave.works/technologies/gitops) to automate provisioning, operating, and updating self-hosted services in my homelab.
|
This project utilizes [Infrastructure as Code](https://en.wikipedia.org/wiki/Infrastructure_as_code) and [GitOps](https://www.weave.works/technologies/gitops) to automate provisioning, operating, and updating self-hosted services in my homelab.
|
||||||
It can be used as a highly customizable framework to build your own homelab.
|
It can be used as a highly customizable framework to build your own homelab.
|
||||||
|
|
||||||
<!-- ANCHOR_END: introduction -->
|
> **What is a homelab?**
|
||||||
|
>
|
||||||
Current status: **ALPHA**
|
> Homelab is a laboratory at home where you can self-host, experiment with new technologies, practice for certifications, and so on.
|
||||||
|
> For more information about homelab in general, see the [r/homelab introduction](https://www.reddit.com/r/homelab/wiki/introduction).
|
||||||
</div>
|
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This section provides a high level overview of the project.
|
Project status: **ALPHA**
|
||||||
For further information, please see the [documentation](https://homelab.khuedoan.com).
|
|
||||||
|
This project is still in the experimental stage, and I don't use anything critical on it.
|
||||||
|
Expect breaking changes that may require a complete redeployment.
|
||||||
|
A proper upgrade path is planned for the stable release.
|
||||||
|
More information can be found in [the roadmap](#roadmap) below.
|
||||||
|
|
||||||
### Hardware
|
### Hardware
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
- 4 × NEC SFF `PC-MK26ECZDR` (Japanese version of the ThinkCentre M700):
|
- 4 × NEC SFF `PC-MK26ECZDR` (Japanese version of the ThinkCentre M700):
|
||||||
- CPU: `Intel Core i5-6600T @ 2.70GHz`
|
- CPU: `Intel Core i5-6600T @ 2.70GHz`
|
||||||
- RAM: `16GB`
|
- RAM: `16GB`
|
||||||
- SSD: `128GB`
|
- SSD: `128GB`
|
||||||
- TP-Link `TL-SG108` switch:
|
- TP-Link `TL-SG108` switch:
|
||||||
- Ports: `8`
|
- Ports: `8`
|
||||||
- Speed: `1000Mbps`
|
- Speed: `1000Mbps`
|
||||||
|
|
||||||
### Features
|
### Features
|
||||||
|
|
||||||
Project status: **Alpha** (see [roadmap](#roadmap) below)
|
|
||||||
|
|
||||||
- [x] Common applications: Gitea, Seafile, Jellyfin, Paperless...
|
- [x] Common applications: Gitea, Seafile, Jellyfin, Paperless...
|
||||||
- [x] Automated bare metal provisioning with PXE boot
|
- [x] Automated bare metal provisioning with PXE boot
|
||||||
- [x] Automated Kubernetes installation and management
|
- [x] Automated Kubernetes installation and management
|
||||||
@ -58,198 +56,172 @@ Project status: **Alpha** (see [roadmap](#roadmap) below)
|
|||||||
- [ ] Automated offsite backups 🚧
|
- [ ] Automated offsite backups 🚧
|
||||||
- [ ] Single sign-on 🚧
|
- [ ] Single sign-on 🚧
|
||||||
|
|
||||||
Some demo videos and screenshots are shown here.
|
Some demo videos and screenshots are shown here (click to enlarge).
|
||||||
They can't capture all of the project's features, but they are sufficient to get a concept of it.
|
They can't capture all the project's features, but they are sufficient to get a concept of it.
|
||||||
|
|
||||||
| [](https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7) |
|
| [![][screenshot-01]](https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7) | [![][screenshot-02]](https://www.youtube.com/watch?v=y-d7btNNAT8) |
|
||||||
| :--: |
|
| :--: | :--: |
|
||||||
| Deploy with a single command (after updating the config files of course) |
|
| Deploy with a single command (after updating the configuration files) | PXE boot |
|
||||||
|
| [![][screenshot-03]][screenshot-03] | [![][screenshot-04]][screenshot-04] |
|
||||||
|
| Homepage with Ingress discovery powered by [Hajimari](https://github.com/toboshii/hajimari) | Monitoring dashboard powered by [Grafana](https://grafana.com/) |
|
||||||
|
| [![][screenshot-05]][screenshot-05] | [![][screenshot-06]][screenshot-06] |
|
||||||
|
| Git server powered by [Gitea](https://gitea.io/en-us/) | [Matrix](https://matrix.org/) chat server |
|
||||||
|
| [![][screenshot-07]][screenshot-07] | [![][screenshot-08]][screenshot-08] |
|
||||||
|
| Continuous integration with [Tekton](https://tekton.dev/) | Continuous deployment with [ArgoCD](https://argoproj.github.io/cd/) |
|
||||||
|
| [![][screenshot-09]][screenshot-09] | [![][screenshot-10]][screenshot-10] |
|
||||||
|
| Cluster management using [Lens](https://k8slens.dev/) | Secret management with [Vault](https://www.vaultproject.io/) |
|
||||||
|
|
||||||
| [](https://www.youtube.com/watch?v=y-d7btNNAT8) |
|
[screenshot-01]: https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7.svg
|
||||||
| :--: |
|
[screenshot-02]: https://user-images.githubusercontent.com/27996771/157303477-df2e7410-8f02-4648-a86c-71e6b7e89e35.png
|
||||||
| PXE boot |
|
[screenshot-03]: https://user-images.githubusercontent.com/27996771/149445807-0f869eb7-d8f5-4fef-ab97-ac281df91a06.png
|
||||||
|
[screenshot-04]: https://user-images.githubusercontent.com/27996771/149446631-1c5d056b-1fdc-48e6-96ba-e1abe1762be0.png
|
||||||
|  |
|
[screenshot-05]: https://user-images.githubusercontent.com/27996771/149444871-38889c9d-862f-41ff-8c05-8ece21da3e9c.png
|
||||||
| :--: |
|
[screenshot-06]: https://user-images.githubusercontent.com/27996771/149448510-7163310c-2049-4ccd-901d-f11f605bfc32.png
|
||||||
| Homepage with Ingress discovery powered by [Hajimari](https://github.com/toboshii/hajimari) |
|
[screenshot-07]: https://user-images.githubusercontent.com/27996771/149445374-58fd0605-bb9a-46e4-81d6-5e584d2b94a9.png
|
||||||
|
[screenshot-08]: https://user-images.githubusercontent.com/27996771/149444716-fc0d7282-4cf7-4ddb-97a4-1a3fb47ff2b8.png
|
||||||
|  |
|
[screenshot-09]: https://user-images.githubusercontent.com/27996771/149448896-9d79947d-468c-45c6-a81d-b43654e8ab6b.png
|
||||||
| :--: |
|
[screenshot-10]: https://user-images.githubusercontent.com/27996771/149452309-de4a893b-e94c-4ba8-9119-ea87449cf77e.png
|
||||||
| Git server powered by [Gitea](https://gitea.io/en-us/) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| Continuous integration with [Tekton](https://tekton.dev/) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| Continuous deployment with [ArgoCD](https://argoproj.github.io/cd/) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| Monitoring dashboard powered by [Grafana](https://grafana.com/) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| [Matrix](https://matrix.org/) chat server powered by [Element](https://matrix.org/docs/projects/client/element) and [Dendrite](https://matrix.org/docs/projects/server/dendrite) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| Cluster management using [Lens](https://k8slens.dev/) (or you can just use `kubectl`) |
|
|
||||||
|
|
||||||
|  |
|
|
||||||
| :--: |
|
|
||||||
| Secret management with [Vault](https://www.vaultproject.io/) |
|
|
||||||
|
|
||||||
### Tech stack
|
### Tech stack
|
||||||
|
|
||||||
<table>
|
<table>
|
||||||
<tr>
|
<tr>
|
||||||
<th>Logo</th>
|
<th>Logo</th>
|
||||||
<th>Name</th>
|
<th>Name</th>
|
||||||
<th>Description</th>
|
<th>Description</th>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://simpleicons.org/icons/ansible.svg"></td>
|
<td><img width="32" src="https://simpleicons.org/icons/ansible.svg"></td>
|
||||||
<td><a href="https://www.ansible.com">Ansible</a></td>
|
<td><a href="https://www.ansible.com">Ansible</a></td>
|
||||||
<td>Automate bare metal provisioning and configuration</td>
|
<td>Automate bare metal provisioning and configuration</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/argo/icon/color/argo-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/argo/icon/color/argo-icon-color.svg"></td>
|
||||||
<td><a href="https://argoproj.github.io/cd">ArgoCD</a></td>
|
<td><a href="https://argoproj.github.io/cd">ArgoCD</a></td>
|
||||||
<td>GitOps tool built to deploy applications to Kubernetes</td>
|
<td>GitOps tool built to deploy applications to Kubernetes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://github.com/jetstack/cert-manager/raw/master/logo/logo.png"></td>
|
<td><img width="32" src="https://github.com/jetstack/cert-manager/raw/master/logo/logo.png"></td>
|
||||||
<td><a href="https://cert-manager.io">cert-manager</a></td>
|
<td><a href="https://cert-manager.io">cert-manager</a></td>
|
||||||
<td>Cloud native certificate management</td>
|
<td>Cloud native certificate management</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://avatars.githubusercontent.com/u/314135?s=200&v=4"></td>
|
<td><img width="32" src="https://avatars.githubusercontent.com/u/314135?s=200&v=4"></td>
|
||||||
<td><a href="https://www.cloudflare.com">Cloudflare</a></td>
|
<td><a href="https://www.cloudflare.com">Cloudflare</a></td>
|
||||||
<td>DNS and Tunnel</td>
|
<td>DNS and Tunnel</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://www.docker.com/wp-content/uploads/2022/03/Moby-logo.png"></td>
|
<td><img width="32" src="https://www.docker.com/wp-content/uploads/2022/03/Moby-logo.png"></td>
|
||||||
<td><a href="https://www.docker.com">Docker</a></td>
|
<td><a href="https://www.docker.com">Docker</a></td>
|
||||||
<td>Ephermeral PXE server and convenient tools container</td>
|
<td>Ephermeral PXE server and convenient tools container</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://github.com/kubernetes-sigs/external-dns/raw/master/docs/img/external-dns.png"></td>
|
<td><img width="32" src="https://github.com/kubernetes-sigs/external-dns/raw/master/docs/img/external-dns.png"></td>
|
||||||
<td><a href="https://github.com/kubernetes-sigs/external-dns">ExternalDNS</a></td>
|
<td><a href="https://github.com/kubernetes-sigs/external-dns">ExternalDNS</a></td>
|
||||||
<td>Synchronizes exposed Kubernetes Services and Ingresses with DNS providers</td>
|
<td>Synchronizes exposed Kubernetes Services and Ingresses with DNS providers</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://upload.wikimedia.org/wikipedia/commons/b/bb/Gitea_Logo.svg"></td>
|
<td><img width="32" src="https://upload.wikimedia.org/wikipedia/commons/b/bb/Gitea_Logo.svg"></td>
|
||||||
<td><a href="https://gitea.com">Gitea</a></td>
|
<td><a href="https://gitea.com">Gitea</a></td>
|
||||||
<td>Self-hosted Git service</td>
|
<td>Self-hosted Git service</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://grafana.com/static/img/menu/grafana2.svg"></td>
|
<td><img width="32" src="https://grafana.com/static/img/menu/grafana2.svg"></td>
|
||||||
<td><a href="https://grafana.com">Grafana</a></td>
|
<td><a href="https://grafana.com">Grafana</a></td>
|
||||||
<td>Operational dashboards</td>
|
<td>Operational dashboards</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/helm/icon/color/helm-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/helm/icon/color/helm-icon-color.svg"></td>
|
||||||
<td><a href="https://helm.sh">Helm</a></td>
|
<td><a href="https://helm.sh">Helm</a></td>
|
||||||
<td>The package manager for Kubernetes</td>
|
<td>The package manager for Kubernetes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/k3s/icon/color/k3s-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/k3s/icon/color/k3s-icon-color.svg"></td>
|
||||||
<td><a href="https://k3s.io">K3s</a></td>
|
<td><a href="https://k3s.io">K3s</a></td>
|
||||||
<td>Lightweight distribution of Kubernetes</td>
|
<td>Lightweight distribution of Kubernetes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/kubernetes/icon/color/kubernetes-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/kubernetes/icon/color/kubernetes-icon-color.svg"></td>
|
||||||
<td><a href="https://kubernetes.io">Kubernetes</a></td>
|
<td><a href="https://kubernetes.io">Kubernetes</a></td>
|
||||||
<td>Container-orchestration system, the backbone of this project</td>
|
<td>Container-orchestration system, the backbone of this project</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://github.com/grafana/loki/blob/main/docs/sources/logo.png?raw=true"></td>
|
<td><img width="32" src="https://github.com/grafana/loki/blob/main/docs/sources/logo.png?raw=true"></td>
|
||||||
<td><a href="https://grafana.com/oss/loki">Loki</a></td>
|
<td><a href="https://grafana.com/oss/loki">Loki</a></td>
|
||||||
<td>Log aggregation system</td>
|
<td>Log aggregation system</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/longhorn/icon/color/longhorn-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/longhorn/icon/color/longhorn-icon-color.svg"></td>
|
||||||
<td><a href="https://longhorn.io">Longhorn</a></td>
|
<td><a href="https://longhorn.io">Longhorn</a></td>
|
||||||
<td>Cloud native distributed block storage for Kubernetes</td>
|
<td>Cloud native distributed block storage for Kubernetes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://avatars.githubusercontent.com/u/60239468?s=200&v=4"></td>
|
<td><img width="32" src="https://avatars.githubusercontent.com/u/60239468?s=200&v=4"></td>
|
||||||
<td><a href="https://metallb.org">MetalLB</a></td>
|
<td><a href="https://metallb.org">MetalLB</a></td>
|
||||||
<td>Bare metal load-balancer for Kubernetes</td>
|
<td>Bare metal load-balancer for Kubernetes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://avatars.githubusercontent.com/u/1412239?s=200&v=4"></td>
|
<td><img width="32" src="https://avatars.githubusercontent.com/u/1412239?s=200&v=4"></td>
|
||||||
<td><a href="https://www.nginx.com">NGINX</a></td>
|
<td><a href="https://www.nginx.com">NGINX</a></td>
|
||||||
<td>Kubernetes Ingress Controller</td>
|
<td>Kubernetes Ingress Controller</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/prometheus/icon/color/prometheus-icon-color.svg"></td>
|
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/prometheus/icon/color/prometheus-icon-color.svg"></td>
|
||||||
<td><a href="https://prometheus.io">Prometheus</a></td>
|
<td><a href="https://prometheus.io">Prometheus</a></td>
|
||||||
<td>Systems monitoring and alerting toolkit</td>
|
<td>Systems monitoring and alerting toolkit</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://docs.renovatebot.com/assets/images/logo.png"></td>
|
<td><img width="32" src="https://docs.renovatebot.com/assets/images/logo.png"></td>
|
||||||
<td><a href="https://www.whitesourcesoftware.com/free-developer-tools/renovate">Renovate</a></td>
|
<td><a href="https://www.whitesourcesoftware.com/free-developer-tools/renovate">Renovate</a></td>
|
||||||
<td>Automatically update dependencies</td>
|
<td>Automatically update dependencies</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://avatars.githubusercontent.com/u/75713131?s=200&v=4"></td>
|
<td><img width="32" src="https://avatars.githubusercontent.com/u/75713131?s=200&v=4"></td>
|
||||||
<td><a href="https://rockylinux.org">Rocky Linux</a></td>
|
<td><a href="https://rockylinux.org">Rocky Linux</a></td>
|
||||||
<td>Base OS for Kubernetes nodes</td>
|
<td>Base OS for Kubernetes nodes</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://avatars.githubusercontent.com/u/47602533?s=200&v=4"></td>
|
<td><img width="32" src="https://avatars.githubusercontent.com/u/47602533?s=200&v=4"></td>
|
||||||
<td><a href="https://tekton.dev">Tekton</a></td>
|
<td><a href="https://tekton.dev">Tekton</a></td>
|
||||||
<td>Cloud native solution for building CI/CD systems</td>
|
<td>Cloud native solution for building CI/CD systems</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://trow.io/trow.png"></td>
|
<td><img width="32" src="https://trow.io/trow.png"></td>
|
||||||
<td><a href="https://trow.io">Trow</a></td>
|
<td><a href="https://trow.io">Trow</a></td>
|
||||||
<td>Private container registry</td>
|
<td>Private container registry</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><img width="32" src="https://simpleicons.org/icons/vault.svg"></td>
|
<td><img width="32" src="https://simpleicons.org/icons/vault.svg"></td>
|
||||||
<td><a href="https://www.vaultproject.io">Vault</a></td>
|
<td><a href="https://www.vaultproject.io">Vault</a></td>
|
||||||
<td>Secrets and encryption management system</td>
|
<td>Secrets and encryption management system</td>
|
||||||
</tr>
|
</tr>
|
||||||
</table>
|
</table>
|
||||||
|
|
||||||
## Get Started
|
## Get Started
|
||||||
|
|
||||||
- [Try it out locally](https://homelab.khuedoan.com/try-locally.html) without any hardware
|
- [Try it out locally](https://homelab.khuedoan.com/installation/development.md) without any hardware
|
||||||
- [Deploy on real hardware](https://homelab.khuedoan.com/deployment) for real workload
|
- [Deploy on real hardware](https://homelab.khuedoan.com/installation/production/prerequisites.md) for production workload
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
See [roadmap](https://homelab.khuedoan.com/roadmap.html) and [open issues](https://github.com/khuedoan/homelab/issues) for a list of proposed features and known issues.
|
See [roadmap](https://homelab.khuedoan.com/reference/roadmap) and [open issues](https://github.com/khuedoan/homelab/issues) for a list of proposed features and known issues.
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
Any contributions you make, either big or small, are greatly appreciated.
|
Any contributions you make, either big or small, are greatly appreciated.
|
||||||
|
|
||||||
|
Please see [contributing guide](https://homelab.khuedoan.com/reference/contributing) for more information.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
> Copyright (c) 2020, 2021, 2022 Khue Doan
|
Copyright © 2020 - 2022 Khue Doan
|
||||||
|
|
||||||
<details>
|
Distributed under the GPLv3 License.
|
||||||
|
See [license page](https://homelab.khuedoan.com/reference/license) or `LICENSE.md` file for more information.
|
||||||
<summary>Distributed under the GPLv3 License.</summary>
|
|
||||||
|
|
||||||
This project is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
|
|
||||||
|
|
||||||
This project is distributed in the hope that it will be useful, but **WITHOUT ANY WARRANTY**; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
|
||||||
See the GNU General Public License for more details.
|
|
||||||
|
|
||||||
You should have received a copy of the GNU General Public License along with this project (`LICENSE.md`).
|
|
||||||
If not, see <https://www.gnu.org/licenses>.
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
## Acknowledgements
|
## Acknowledgements
|
||||||
|
|
||||||
- [ArgoCD usage in my coworker's homelab](https://github.com/locmai/humble)
|
- [ArgoCD usage and monitoring configuration in locmai/humble](https://github.com/locmai/humble)
|
||||||
- [README template](https://github.com/othneildrew/Best-README-Template)
|
- [README template](https://github.com/othneildrew/Best-README-Template)
|
||||||
- [Run the same Cloudflare Tunnel across many `cloudflared` processes](https://developers.cloudflare.com/cloudflare-one/tutorials/many-cfd-one-tunnel)
|
- [Run the same Cloudflare Tunnel across many `cloudflared` processes](https://developers.cloudflare.com/cloudflare-one/tutorials/many-cfd-one-tunnel)
|
||||||
- [MAC address environment variable in GRUB config](https://askubuntu.com/questions/1272400/how-do-i-automate-network-installation-of-many-ubuntu-18-04-systems-with-efi-and)
|
- [MAC address environment variable in GRUB config](https://askubuntu.com/questions/1272400/how-do-i-automate-network-installation-of-many-ubuntu-18-04-systems-with-efi-and)
|
||||||
|
|||||||
@ -1,10 +0,0 @@
|
|||||||
.POSIX:
|
|
||||||
|
|
||||||
default: book
|
|
||||||
|
|
||||||
mermaid*.js:
|
|
||||||
mdbook-mermaid install .
|
|
||||||
|
|
||||||
.PHONY: book
|
|
||||||
book: mermaid*.js
|
|
||||||
mdbook build .
|
|
||||||
@ -1,11 +0,0 @@
|
|||||||
# Documents
|
|
||||||
|
|
||||||
Documents can be viewed at <https://homelab.khuedoan.com>.
|
|
||||||
It's running on my other cluster in the [khuedoan/horus](https://github.com/khuedoan/horus) project
|
|
||||||
(so if the homelab goes down I can still read the documentation).
|
|
||||||
|
|
||||||
To view locally, install [`mdbook`](https://github.com/rust-lang/mdBook#installation) and run:
|
|
||||||
|
|
||||||
```
|
|
||||||
mdbook serve
|
|
||||||
```
|
|
||||||
@ -1,18 +0,0 @@
|
|||||||
[book]
|
|
||||||
authors = ["Khue Doan"]
|
|
||||||
language = "en"
|
|
||||||
multilingual = false
|
|
||||||
src = "src"
|
|
||||||
title = "Khue's Homelab"
|
|
||||||
|
|
||||||
[preprocessor.mermaid]
|
|
||||||
command = "mdbook-mermaid"
|
|
||||||
|
|
||||||
[output.html]
|
|
||||||
additional-js = ["mermaid.min.js", "mermaid-init.js"]
|
|
||||||
git-repository-url = "https://github.com/khuedoan/homelab"
|
|
||||||
edit-url-template = "https://github.com/khuedoan/homelab/edit/master/docs/{path}"
|
|
||||||
|
|
||||||
# TODO deprecate this after 6 months
|
|
||||||
[output.html.redirect]
|
|
||||||
"/try-on-a-vm.html" = "/try-locally.html"
|
|
||||||
3
docs/index.md
Normal file
3
docs/index.md
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
--8<--
|
||||||
|
README.md
|
||||||
|
--8<--
|
||||||
@ -1,4 +1,4 @@
|
|||||||
# Try locally
|
# Development environment
|
||||||
|
|
||||||
## Caveats compare to production environment
|
## Caveats compare to production environment
|
||||||
|
|
||||||
@ -17,15 +17,15 @@ Host machine:
|
|||||||
|
|
||||||
- OS: Linux (Windows and macOS will not work due to networking limitations, you can use a Linux VM)
|
- OS: Linux (Windows and macOS will not work due to networking limitations, you can use a Linux VM)
|
||||||
- Recommended hardware specifications:
|
- Recommended hardware specifications:
|
||||||
- CPU: 4 cores
|
- CPU: 4 cores
|
||||||
- RAM: 16 GiB
|
- RAM: 16 GiB
|
||||||
|
|
||||||
Install the following packages:
|
Install the following packages:
|
||||||
|
|
||||||
- `docker`
|
- `docker`
|
||||||
- `make`
|
- `make`
|
||||||
|
|
||||||
Clone the repository (follow the [configuration guide](./deployment/configuration.md) if you want to customize it):
|
Clone the repository and checkout the development branch:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
git clone https://github.com/khuedoan/homelab
|
git clone https://github.com/khuedoan/homelab
|
||||||
44
docs/installation/production/configuration.md
Normal file
44
docs/installation/production/configuration.md
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# Configuration
|
||||||
|
|
||||||
|
Open the tools container if you haven't already:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make tools
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! note
|
||||||
|
|
||||||
|
It will take a while to build the tools container on the first time
|
||||||
|
|
||||||
|
Run the following script to configure the homelab:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make configure
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! example
|
||||||
|
|
||||||
|
<!-- TODO update example input -->
|
||||||
|
|
||||||
|
```
|
||||||
|
Text editor (nvim):
|
||||||
|
Enter seed repo (github.com/khuedoan/homelab): github.com/example/homelab
|
||||||
|
Enter your domain (khuedoan.com): example.com
|
||||||
|
```
|
||||||
|
|
||||||
|
It will prompt you to edit the inventory:
|
||||||
|
|
||||||
|
- IP address: the desired one, not the current one, since your servers have no operating system installed yet
|
||||||
|
- Disk: based on `/dev/$DISK`, in my case it's `sda`, but yours can be `sdb`, `nvme0n1`...
|
||||||
|
- Network interface: usually it's `eth0`, mine is `eno1`
|
||||||
|
- MAC address: the **lowercase, colon separated** MAC address of the above network interface
|
||||||
|
|
||||||
|
!!! example
|
||||||
|
|
||||||
|
```yaml title="metal/inventories/prod.yml"
|
||||||
|
--8<--
|
||||||
|
metal/inventories/prod.yml
|
||||||
|
--8<--
|
||||||
|
```
|
||||||
|
|
||||||
|
At the end it will show what has changed. After examining the diff, commit and push the changes.
|
||||||
21
docs/installation/production/deployment.md
Normal file
21
docs/installation/production/deployment.md
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
# Deployment
|
||||||
|
|
||||||
|
Open the tools container, which includes all the tools needed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make tools
|
||||||
|
```
|
||||||
|
|
||||||
|
Build the lab:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! note
|
||||||
|
|
||||||
|
It will take a while to download Rocky Linux ISO on the first time
|
||||||
|
|
||||||
|
Yes it's that simple!
|
||||||
|
|
||||||
|
You can read the [architecture document](../../reference/architecture.md) while waiting for the deployment to complete.
|
||||||
@ -1,8 +1,8 @@
|
|||||||
# External resources
|
# External resources
|
||||||
|
|
||||||
**WIP documents**
|
!!! info
|
||||||
|
|
||||||
> These resources are optional, the homelab still works without them but will lack some features like trusted certificates and offsite backup
|
These resources are optional, the homelab still works without them but will lack some features like trusted certificates and offsite backup
|
||||||
|
|
||||||
Although I try to keep the amount of external resources to the minimum, there's still need for a few of them.
|
Although I try to keep the amount of external resources to the minimum, there's still need for a few of them.
|
||||||
Below is a list of external resources and why we need them (also see some [alternatives](#alternatives) below).
|
Below is a list of external resources and why we need them (also see some [alternatives](#alternatives) below).
|
||||||
@ -12,15 +12,17 @@ Below is a list of external resources and why we need them (also see some [alter
|
|||||||
| Terraform Cloud | Workspace | Terraform state backend |
|
| Terraform Cloud | Workspace | Terraform state backend |
|
||||||
| Cloudflare | DNS | DNS and [DNS-01 challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) for certificates |
|
| Cloudflare | DNS | DNS and [DNS-01 challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) for certificates |
|
||||||
| Cloudflare | Tunnel | Public services to the internet without port-forwarding |
|
| Cloudflare | Tunnel | Public services to the internet without port-forwarding |
|
||||||
| Minio | Bucket | Onsite backup |
|
<!-- | Minio | Bucket | Onsite backup | -->
|
||||||
| AWS | S3 Glacier | Offsite backup |
|
<!-- | AWS | S3 Glacier | Offsite backup | -->
|
||||||
|
|
||||||
This layer will:
|
This layer will:
|
||||||
|
|
||||||
- Create external resources
|
- Create external resources
|
||||||
- Add external secrets to namespaces
|
- Add external secrets to namespaces
|
||||||
|
|
||||||
## Prerequisites
|
## Create credentials
|
||||||
|
|
||||||
|
You'll be asked to provide these credentials on first build.
|
||||||
|
|
||||||
### Create Terraform workspace
|
### Create Terraform workspace
|
||||||
|
|
||||||
@ -52,25 +54,17 @@ If you decide to use a [different Terraform backend](https://www.terraform.io/la
|
|||||||
<!-- └── Is in - 117.xxx.xxx.xxx, 2402:xxx:xxx:xxx:xxx:xxx:xxx:xxx -->
|
<!-- └── Is in - 117.xxx.xxx.xxx, 2402:xxx:xxx:xxx:xxx:xxx:xxx:xxx -->
|
||||||
<!-- ``` -->
|
<!-- ``` -->
|
||||||
|
|
||||||
### Create Minio keys
|
<!-- ### Create Minio keys -->
|
||||||
|
|
||||||
TODO: skip this for now
|
<!-- TODO: skip this for now -->
|
||||||
|
|
||||||
### Create AWS API key
|
<!-- ### Create AWS API key -->
|
||||||
|
|
||||||
TODO: skip this for now
|
<!-- TODO: skip this for now -->
|
||||||
|
|
||||||
## Deploy
|
|
||||||
|
|
||||||
Apply Terraform (you will be prompted to log in to Terraform Cloud and enter API keys from the previous steps):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# From the project root
|
|
||||||
make external
|
|
||||||
```
|
|
||||||
|
|
||||||
## Alternatives
|
## Alternatives
|
||||||
|
|
||||||
- Terraform Cloud: any other [Terraform backends](https://www.terraform.io/language/settings/backends)
|
- Terraform Cloud: any other [Terraform backends](https://www.terraform.io/language/settings/backends)
|
||||||
|
- Cloudflare DNS: see [manual DNS setup](../../tutorials/manual-dns-setup.md)
|
||||||
- Cloudflare Tunnel: you can create a small VPS in the cloud and utilize Wireguard and HAProxy to route traffic via it, or just use simple port-forwarding if it's available (see also [awesome tunneling](https://github.com/anderspitman/awesome-tunneling))
|
- Cloudflare Tunnel: you can create a small VPS in the cloud and utilize Wireguard and HAProxy to route traffic via it, or just use simple port-forwarding if it's available (see also [awesome tunneling](https://github.com/anderspitman/awesome-tunneling))
|
||||||
- Minio and S3 Glacier: any S3 compatible object storage, such as Backblaze B2, Minio...
|
<!-- - Minio and S3 Glacier: any S3 compatible object storage, such as Backblaze B2, Minio... -->
|
||||||
@ -14,4 +14,4 @@ Save the following files to a safe location (like a password manager):
|
|||||||
|
|
||||||
## Next steps
|
## Next steps
|
||||||
|
|
||||||
TODO
|
- [User onboarding](../../user-guide/onboarding.md)
|
||||||
@ -1,25 +1,46 @@
|
|||||||
# Prerequisites
|
# Prerequisites
|
||||||
|
|
||||||
|
## Fork this repository
|
||||||
|
|
||||||
|
Because [this project](https://github.com/khuedoan/homelab) applies GitOps practices,
|
||||||
|
it's the source of truth for _my_ homelab, so you'll need to fork it to make it yours:
|
||||||
|
|
||||||
|
[:fontawesome-solid-code-fork: Fork khuedoan/homelab](https://github.com/khuedoan/homelab/fork){ .md-button }
|
||||||
|
|
||||||
|
By using this project you agree to [the license](/license).
|
||||||
|
|
||||||
|
|
||||||
|
!!! summary "License TL;DR"
|
||||||
|
|
||||||
|
- This project is free to use for any purpose, but it comes with no warranty
|
||||||
|
- You must use the same [GPLv3 license](https://www.gnu.org/licenses/gpl-3.0.en.html) in `LICENSE.md`
|
||||||
|
- You must keep the copy right notice and/or include an acknowledgement
|
||||||
|
- Your project must remain open-source
|
||||||
|
|
||||||
## Hardware requirements
|
## Hardware requirements
|
||||||
|
|
||||||
### Initial controller
|
### Initial controller
|
||||||
|
|
||||||
> The initial controller is the machine used to bootstrap the cluster, we only need it once, you can use your laptop or desktop
|
!!! info
|
||||||
|
|
||||||
- Any machine that can run Docker with the `host` networking driver ([which means only Docker on Linux hosts](https://docs.docker.com/network/host/), you can use a Linux virtual machine with bridged networking if you're on macOS or Windows)
|
The initial controller is the machine used to bootstrap the cluster, we only need it once, you can use your laptop or desktop
|
||||||
|
|
||||||
|
- A Linux machine that can run Docker (because the `host` networking driver used for PXE boot [only supports Linux](https://docs.docker.com/network/host/), you can use a Linux virtual machine with bridged networking if you're on macOS or Windows).
|
||||||
|
|
||||||
### Servers
|
### Servers
|
||||||
|
|
||||||
Any modern `x86_64` computer(s) should work, you can use old PCs, laptops or servers.
|
Any modern `x86_64` computer(s) should work, you can use old PCs, laptops or servers.
|
||||||
|
|
||||||
> This is the requirements for _each_ node
|
!!! info
|
||||||
|
|
||||||
|
This is the requirements for _each_ node
|
||||||
|
|
||||||
| Component | Minimum | Recommended |
|
| Component | Minimum | Recommended |
|
||||||
| :-- | :-- | :-- |
|
| :-- | :-- | :-- |
|
||||||
| CPU | 2 cores | 4 cores |
|
| CPU | 2 cores | 4 cores |
|
||||||
| RAM | 8 GB | 16 GB |
|
| RAM | 8 GB | 16 GB |
|
||||||
| Hard drive | 128 GB | 512 GB (depending on your storage usage, the base installation will not use more than 128GB) |
|
| Hard drive | 128 GB | 512 GB (depending on your storage usage, the base installation will not use more than 128GB) |
|
||||||
| Node count | 1 (checkout the [single node cluster adjustments](../tutorials/single-node-cluster-adjustments.md) tutorial) | 3 or more for high availability |
|
| Node count | 1 (checkout the [single node cluster adjustments](../../tutorials/single-node-cluster-adjustments.md) tutorial) | 3 or more for high availability |
|
||||||
|
|
||||||
Additional capabilities:
|
Additional capabilities:
|
||||||
|
|
||||||
@ -28,18 +49,20 @@ Additional capabilities:
|
|||||||
|
|
||||||
### Network setup
|
### Network setup
|
||||||
|
|
||||||
- All servers must be connected to the same **wired** network with the initial controller (Wifi is untested, please let me know if it works)
|
- All servers must be connected to the same **wired** network with the initial controller
|
||||||
- You have the access to change DNS config (on your router or at your domain registrar)
|
- You have the access to change DNS config (on your router or at your domain registrar)
|
||||||
|
|
||||||
## Domain
|
## Domain
|
||||||
|
|
||||||
Buying a domain is highly recommended, but if you don't have one, you can also update your router config and point [`*.home.arpa`](https://datatracker.ietf.org/doc/html/rfc8375) to the load balancer (more on that later).
|
Buying a domain is highly recommended, but if you don't have one, see [manual DNS setup](../../tutorials/manual-dns-setup.md).
|
||||||
|
|
||||||
## BIOS setup
|
## BIOS setup
|
||||||
|
|
||||||
> You need to do it once per machine if the default config is not sufficent,
|
!!! info
|
||||||
> usually for consumer hardware this can not be automated
|
|
||||||
> (it requires something like [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) to automate).
|
You need to do it once per machine if the default config is not sufficent,
|
||||||
|
usually for consumer hardware this can not be automated
|
||||||
|
(it requires something like [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) to automate).
|
||||||
|
|
||||||
Common settings:
|
Common settings:
|
||||||
|
|
||||||
@ -52,24 +75,26 @@ Boot order options (select one, each has their pros and cons):
|
|||||||
1. Only boot from the network if no operating system found: works on most hardware but you need to manually wipe your hard drive or delete the existing boot record for the current OS
|
1. Only boot from the network if no operating system found: works on most hardware but you need to manually wipe your hard drive or delete the existing boot record for the current OS
|
||||||
2. Prefer booting from the network if turned on via WoL: more convenience but your BIOS must support it, and you must test it throughly to ensure you don't accidentally wipe your servers
|
2. Prefer booting from the network if turned on via WoL: more convenience but your BIOS must support it, and you must test it throughly to ensure you don't accidentally wipe your servers
|
||||||
|
|
||||||
Below is my BIOS setup for reference. Your motherboard may have a different name for the options, so you'll need to adapt it to your hardware.
|
!!! example
|
||||||
|
|
||||||
```yaml
|
Below is my BIOS setup for reference. Your motherboard may have a different name for the options, so you'll need to adapt it to your hardware.
|
||||||
Devices:
|
|
||||||
NetworkSetup:
|
```yaml
|
||||||
PXEIPv4: true
|
Devices:
|
||||||
PXEIPv6: false
|
NetworkSetup:
|
||||||
Advanced:
|
PXEIPv4: true
|
||||||
CPUSetup:
|
PXEIPv6: false
|
||||||
VT-d: true
|
Advanced:
|
||||||
Power:
|
CPUSetup:
|
||||||
AutomaticPowerOn:
|
VT-d: true
|
||||||
WoL: Automatic # Use network boot if Wake-on-LAN
|
Power:
|
||||||
Security:
|
AutomaticPowerOn:
|
||||||
SecureBoot: false
|
WoL: Automatic # Use network boot if Wake-on-LAN
|
||||||
Startup:
|
Security:
|
||||||
CSM: false
|
SecureBoot: false
|
||||||
```
|
Startup:
|
||||||
|
CSM: false
|
||||||
|
```
|
||||||
|
|
||||||
## Gather information
|
## Gather information
|
||||||
|
|
||||||
177
docs/reference/architecture.md
Normal file
177
docs/reference/architecture.md
Normal file
@ -0,0 +1,177 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
```
|
||||||
|
+--------------+
|
||||||
|
| ./apps |
|
||||||
|
|--------------|
|
||||||
|
| ./platform |
|
||||||
|
|--------------| +------------+
|
||||||
|
| ./system |- - - -| ./external |
|
||||||
|
|--------------| +------------+
|
||||||
|
| ./bootstrap |
|
||||||
|
|--------------|
|
||||||
|
| ./metal |
|
||||||
|
|--------------|
|
||||||
|
| HARDWARE |
|
||||||
|
+--------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
Main components:
|
||||||
|
|
||||||
|
- `./metal`: bare metal management, install Linux and Kubernetes
|
||||||
|
- `./bootstrap`: GitOps bootstrap with ArgoCD
|
||||||
|
- `./system`: critical system components for the cluster (load balancer, storage, ingress, operation tools...)
|
||||||
|
- `./platform`: essential components for service hosting platform (vault, git...)
|
||||||
|
- `./apps`: user facing applications
|
||||||
|
- `./external` (optional): externally managed services
|
||||||
|
|
||||||
|
Support components:
|
||||||
|
|
||||||
|
- `./tools`: tools container, includes all the tools you'll need
|
||||||
|
- `./docs`: all documentation go here, this will generate a searchable web UI
|
||||||
|
- `./scripts`: scripts to automate common tasks
|
||||||
|
|
||||||
|
## Provisioning flow
|
||||||
|
|
||||||
|
Everything is automated, after you edit the configuration files, you just need to run a single `make` command and it will:
|
||||||
|
|
||||||
|
- (1) Build the `./metal` layer:
|
||||||
|
- Create an ephemeral, stateless PXE server
|
||||||
|
- Install Linux on all servers in parallel
|
||||||
|
- Build a Kubernetes cluster (based on k3s)
|
||||||
|
- (2) Build the `./bootstrap` layer:
|
||||||
|
- Install ArgoCD
|
||||||
|
- Configure the root app to manage other layers (and also manage itself)
|
||||||
|
|
||||||
|
From now on, ArgoCD will do the rest:
|
||||||
|
|
||||||
|
- (3) Build the `./system` layer (storage, networking, monitoring, etc)
|
||||||
|
- (4) Build the `./platform` layer (Gitea, Vault, SSO, etc)
|
||||||
|
- (5) Build the `./apps` layer: (Syncthing, Jellyfin, etc)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
subgraph metal[./metal]
|
||||||
|
pxe[PXE Server] -.-> linux[Rocky Linux] --> k3s
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph bootstrap[./bootstrap]
|
||||||
|
argocd[ArgoCD] --> rootapp[Root app]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph system[./system]
|
||||||
|
metallb[MetalLB]
|
||||||
|
nginx[NGINX]
|
||||||
|
longhorn[Longhorn]
|
||||||
|
cert-manager
|
||||||
|
external-dns[External DNS]
|
||||||
|
cloudflared
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph external[./external]
|
||||||
|
letsencrypt[Let's Encrypt]
|
||||||
|
cloudflare[Cloudflare]
|
||||||
|
end
|
||||||
|
|
||||||
|
letsencrypt -.-> cert-manager
|
||||||
|
cloudflare -.-> cert-manager
|
||||||
|
cloudflare -.-> external-dns
|
||||||
|
cloudflare -.-> cloudflared
|
||||||
|
|
||||||
|
subgraph platform
|
||||||
|
gitea[Gitea]
|
||||||
|
tekton[Tekton]
|
||||||
|
vault[Vault]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph apps
|
||||||
|
jellyfin[Jellyfin]
|
||||||
|
matrix[Matrix]
|
||||||
|
paperless[Paperless]
|
||||||
|
seafile[Seafile]
|
||||||
|
end
|
||||||
|
|
||||||
|
make[Run make] -- 1 --> metal -- 2 --> bootstrap -. 3 .-> system -. 4 .-> platform -. 5 .-> apps
|
||||||
|
```
|
||||||
|
|
||||||
|
Below is the pseudo code for the entire process, you don't have to read it right now, but it will be handy for debugging.
|
||||||
|
|
||||||
|
??? detailed "Detailed provisioning flow"
|
||||||
|
|
||||||
|
```
|
||||||
|
Human run make:
|
||||||
|
build ./metal:
|
||||||
|
install the OS:
|
||||||
|
download the installer image and extract it
|
||||||
|
create a PXE server on the controller using Docker Compose:
|
||||||
|
DHCP server
|
||||||
|
TFTP server
|
||||||
|
HTTP server
|
||||||
|
create init config for each machine
|
||||||
|
turn the machines on via WoL
|
||||||
|
the machines boot:
|
||||||
|
select network boot automatically
|
||||||
|
broadcast DHCP request
|
||||||
|
DHCP server reply:
|
||||||
|
machine IP
|
||||||
|
TFTP server (next-server) IP
|
||||||
|
get boot files from TFTP server
|
||||||
|
GRUB
|
||||||
|
GRUB config with URL to init config based on MAC address
|
||||||
|
kernel
|
||||||
|
initrd
|
||||||
|
boot to the kernel
|
||||||
|
download from HTTP server:
|
||||||
|
init config from the URL in GRUB config
|
||||||
|
remaining files required to boot
|
||||||
|
install the OS based on the init config:
|
||||||
|
configure the system
|
||||||
|
remaining files required to install
|
||||||
|
reboot to the new OS
|
||||||
|
controller see the machines are ready
|
||||||
|
build a Kubernetes cluster:
|
||||||
|
download k3s binary
|
||||||
|
generate cluster token
|
||||||
|
copy k3s config files
|
||||||
|
enable k3s service and form a cluster
|
||||||
|
create KUBECONFIG file
|
||||||
|
create MetalLB config:
|
||||||
|
use the last /27 subnet of the network
|
||||||
|
apply the config
|
||||||
|
build ./bootstrap:
|
||||||
|
install ArgoCD:
|
||||||
|
apply helm chart
|
||||||
|
wait for status
|
||||||
|
install root app:
|
||||||
|
select values file:
|
||||||
|
if Gitea unreachable (first install):
|
||||||
|
get data from GitHub
|
||||||
|
else:
|
||||||
|
get data from Gitea
|
||||||
|
apply helm chart
|
||||||
|
wait for status
|
||||||
|
ArgoCD apply the rest:
|
||||||
|
clone git repo
|
||||||
|
install components based on directories:
|
||||||
|
./bootstrap (it manages itself):
|
||||||
|
argocd
|
||||||
|
root
|
||||||
|
./system:
|
||||||
|
storage
|
||||||
|
loadbalancer
|
||||||
|
ingress
|
||||||
|
etc
|
||||||
|
./platform (depends on ./system):
|
||||||
|
git:
|
||||||
|
migrate the homelab repository from GitHub
|
||||||
|
ArgoCD switch the source from GitHub to Gitea
|
||||||
|
ci
|
||||||
|
vault
|
||||||
|
etc
|
||||||
|
./apps (depends on ./system and ./platform):
|
||||||
|
homepage
|
||||||
|
jellyfin
|
||||||
|
etc
|
||||||
|
```
|
||||||
@ -24,9 +24,9 @@
|
|||||||
|
|
||||||
- Upgrade to Kubernetes 1.23
|
- Upgrade to Kubernetes 1.23
|
||||||
- Support external resources:
|
- Support external resources:
|
||||||
- Cloudflare DNS and Tunnel
|
- Cloudflare DNS and Tunnel
|
||||||
- Backblaze for backup
|
- Backblaze for backup
|
||||||
- Auto inject secrets to required namespaces
|
- Auto inject secrets to required namespaces
|
||||||
- Replace self-signed certificates with Let's Encrypt production (with API token injected from the `external` layer)
|
- Replace self-signed certificates with Let's Encrypt production (with API token injected from the `external` layer)
|
||||||
- Add DNS records automatically using external-dns
|
- Add DNS records automatically using external-dns
|
||||||
- Easy Cloudflare Tunnel configuration with annotations
|
- Easy Cloudflare Tunnel configuration with annotations
|
||||||
33
docs/reference/contributing.md
Normal file
33
docs/reference/contributing.md
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
# Contributing
|
||||||
|
|
||||||
|
## How to contribute
|
||||||
|
|
||||||
|
### Bug report
|
||||||
|
|
||||||
|
TODO
|
||||||
|
|
||||||
|
### Merge request
|
||||||
|
|
||||||
|
TODO
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
|
||||||
|
Documents can be viewed at <https://homelab.khuedoan.com>.
|
||||||
|
It's running on my other cluster in the [khuedoan/horus](https://github.com/khuedoan/horus) project
|
||||||
|
(so if the homelab goes down I can still read the documentation).
|
||||||
|
|
||||||
|
To edit and view locally, run:
|
||||||
|
|
||||||
|
```
|
||||||
|
make docs
|
||||||
|
```
|
||||||
|
|
||||||
|
Then visit [localhost:8000](http://localhost:8000)
|
||||||
|
|
||||||
|
## Contributors
|
||||||
|
|
||||||
|
Here is a list of the contributors who have helped improving my homelab. Big shout-out to them!
|
||||||
|
|
||||||
|
- Loc Mai ([@locmai](https://github.com/locmai))
|
||||||
|
|
||||||
|
If you feel you're missing from this list, feel free to add yourself in a PR.
|
||||||
5
docs/reference/license.md
Normal file
5
docs/reference/license.md
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
Copyright © 2020 - 2022 Khue Doan
|
||||||
|
|
||||||
|
--8<--
|
||||||
|
LICENSE.md
|
||||||
|
--8<--
|
||||||
91
docs/reference/roadmap.md
Normal file
91
docs/reference/roadmap.md
Normal file
@ -0,0 +1,91 @@
|
|||||||
|
# Roadmap
|
||||||
|
|
||||||
|
!!! info
|
||||||
|
|
||||||
|
Current status: **ALPHA**
|
||||||
|
|
||||||
|
## Alpha requirements
|
||||||
|
|
||||||
|
Literally anything that works.
|
||||||
|
|
||||||
|
## Beta requirements
|
||||||
|
|
||||||
|
Good enough for tinkering and personal usage, and reasonably secure.
|
||||||
|
|
||||||
|
- [x] Automated bare metal provisioning
|
||||||
|
- [x] Controller set up (Docker)
|
||||||
|
- [x] OS installation (PXE boot)
|
||||||
|
- [x] Automated cluster creation (k3s)
|
||||||
|
- [x] Automated application deployment (ArgoCD)
|
||||||
|
- [x] Automated DNS management
|
||||||
|
- [x] Initialize GitOps repository on Gitea automatically
|
||||||
|
- [x] Observability
|
||||||
|
- [x] Monitoring
|
||||||
|
- [x] Logging
|
||||||
|
- [ ] Alerting
|
||||||
|
- [ ] SSO
|
||||||
|
- [ ] Reasonably secure
|
||||||
|
- [x] Automated certificate management
|
||||||
|
- [x] Declarative secret management
|
||||||
|
- [ ] Replace all default passwords with randomly generated ones
|
||||||
|
- [x] Expose services to the internet securely with Cloudflare Tunnel
|
||||||
|
- [x] Only use open-source technologies (except external managed services in `./external`)
|
||||||
|
- [x] Everything is defined as code
|
||||||
|
- [ ] Backup solution (3 copies, 2 seperate devices, 1 offsite)
|
||||||
|
- [ ] Define [SLOs](https://en.wikipedia.org/wiki/Service-level_objective):
|
||||||
|
- [ ] 70% availability (might break in the weekend due to new experimentation)
|
||||||
|
- [x] Core applications
|
||||||
|
- [x] Gitea
|
||||||
|
- [x] Tekton
|
||||||
|
- [x] Vault
|
||||||
|
- [x] Private container registry
|
||||||
|
- [x] Homepage
|
||||||
|
|
||||||
|
## Stable requirements
|
||||||
|
|
||||||
|
Can be used in "production" (for family or even small scale businesses).
|
||||||
|
|
||||||
|
- [x] A single command to deploy everything
|
||||||
|
- [x] Fast deployment time (from empty hard drive to running services in under 1 hour)
|
||||||
|
- [ ] Fully _automatic_, not just _automated_
|
||||||
|
- [x] Bare-metal OS rolling upgrade
|
||||||
|
- [x] Kubernetes version rolling upgrade
|
||||||
|
- [x] Application version upgrade
|
||||||
|
- [ ] Encrypted backups
|
||||||
|
- [ ] Secrets rotation
|
||||||
|
- [x] Self healing
|
||||||
|
- [ ] Secure by default
|
||||||
|
- [ ] SELinux
|
||||||
|
- [ ] Network policies
|
||||||
|
- [ ] Static code analysis
|
||||||
|
- [ ] Chaos testing
|
||||||
|
- [x] Minimal dependency on external services
|
||||||
|
- [ ] Complete documentation
|
||||||
|
- [x] Diagram as code
|
||||||
|
- [x] Book (this book)
|
||||||
|
- [ ] Walkthrough tutorial and feature demo (video)
|
||||||
|
- [x] Configuration script for new users
|
||||||
|
- [ ] SLOs:
|
||||||
|
- [ ] 99,9% availability (less than 9 hours of downtime per year)
|
||||||
|
- [ ] 99,99% data durability
|
||||||
|
- [ ] Clear upgrade path
|
||||||
|
- [ ] Additional applications
|
||||||
|
- [ ] Matrix with bridges
|
||||||
|
- [ ] VPN server
|
||||||
|
- [ ] PeerTube
|
||||||
|
- [x] Seafile
|
||||||
|
- [x] Blog
|
||||||
|
- [ ] [Development dashboard](https://github.com/khuedoan/homelab-backstage)
|
||||||
|
|
||||||
|
## Unplanned
|
||||||
|
|
||||||
|
Nice to have
|
||||||
|
|
||||||
|
- [ ] Addition applications
|
||||||
|
- [ ] Mail server
|
||||||
|
- [ ] Air-gap install
|
||||||
|
- [ ] Automated testing
|
||||||
|
- [ ] Security audit
|
||||||
|
- [ ] Serverless ([Knative](https://knative.dev))
|
||||||
|
- [ ] Cluster API ([last attempt](https://github.com/khuedoan/homelab/pull/2))
|
||||||
|
- [ ] Split DNS (requires a better router)
|
||||||
1
docs/runbooks/argocd.md
Normal file
1
docs/runbooks/argocd.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# ArgoCD
|
||||||
1
docs/runbooks/cert-manager.md
Normal file
1
docs/runbooks/cert-manager.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# cert-manager
|
||||||
1
docs/runbooks/gitea.md
Normal file
1
docs/runbooks/gitea.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Gitea
|
||||||
1
docs/runbooks/longhorn.md
Normal file
1
docs/runbooks/longhorn.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Longhorn
|
||||||
@ -1,4 +1,4 @@
|
|||||||
# Secret management
|
# Vault
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
@ -7,8 +7,10 @@
|
|||||||
- Secrets that can be generated are automatically generated and stored in Vault.
|
- Secrets that can be generated are automatically generated and stored in Vault.
|
||||||
- Integrate with GitOps using [External Secrets Operator](https://external-secrets.io)
|
- Integrate with GitOps using [External Secrets Operator](https://external-secrets.io)
|
||||||
|
|
||||||
> Despite the name "_External_ Secrets Operator", our Vault is deployed on the same cluster.
|
!!! info
|
||||||
> HashiCorp Vault can be replaced with AWS Secret Manager, Google Cloud Secret Manager, Azure Key Vault, etc.
|
|
||||||
|
Despite the name _External_ Secrets Operator, our Vault is deployed on the same cluster.
|
||||||
|
HashiCorp Vault can be replaced with AWS Secret Manager, Google Cloud Secret Manager, Azure Key Vault, etc.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TD
|
||||||
@ -1 +0,0 @@
|
|||||||
homelab.khuedoan.com
|
|
||||||
@ -1,35 +0,0 @@
|
|||||||
# Summary
|
|
||||||
|
|
||||||
- [Introduction](introduction.md)
|
|
||||||
- [Try locally](try-locally.md)
|
|
||||||
- [Deployment](./deployment/README.md)
|
|
||||||
- [Provisioning flow](./deployment/provisioning-flow.md)
|
|
||||||
- [Prerequisites](./deployment/prerequisites.md)
|
|
||||||
- [Configuration](./deployment/configuration.md)
|
|
||||||
- [Deploy the homelab](./deployment/deployment.md)
|
|
||||||
- [DNS setup](./deployment/dns.md)
|
|
||||||
- [External resources (optional)](./deployment/external-resources.md)
|
|
||||||
- [Post-installation](./deployment/post-installation.md)
|
|
||||||
- [Troubleshooting](./troubleshooting.md)
|
|
||||||
- [Tutorials]()
|
|
||||||
- [Use both GitHub and Gitea](./tutorials/use-both-github-and-gitea.md)
|
|
||||||
- [Single node cluster adjustments](./tutorials/single-node-cluster-adjustments.md)
|
|
||||||
- [Add or remove nodes (scale up or down)](./tutorials/add-or-remove-nodes.md)
|
|
||||||
- [Run commands on multiple nodes](./tutorials/run-commands-on-multiple-nodes.md)
|
|
||||||
- [Install new applications]()
|
|
||||||
- [Expose services to the internet](tutorials/expose-services-to-the-internet.md)
|
|
||||||
- [Runbooks]()
|
|
||||||
- [ArgoCD]()
|
|
||||||
- [Gitea]()
|
|
||||||
- [Vault]()
|
|
||||||
- [Reference](./reference/README.md)
|
|
||||||
- [Architecture](./reference/architecture.md)
|
|
||||||
- [Secret management](./reference/secret-management.md)
|
|
||||||
- [FAQ](./reference/faq.md)
|
|
||||||
- [Contributors](./reference/contributors.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
- [Changelog](./changelog.md)
|
|
||||||
- [Roadmap](./roadmap.md)
|
|
||||||
- [License](./license.md)
|
|
||||||
@ -1,10 +0,0 @@
|
|||||||
# Deployment
|
|
||||||
|
|
||||||
In this section you will learn how to:
|
|
||||||
|
|
||||||
- Prepare your hardware
|
|
||||||
- Update the configurations to fit your needs (hardware info, domain name...)
|
|
||||||
- Build the cluster and bootstrap it
|
|
||||||
- Perform some addtional steps depending on your set up (DNS, external secrets...)
|
|
||||||
|
|
||||||
If you encounter any issue, please see the [Troubleshooting](../troubleshooting.md) section.
|
|
||||||
@ -1,59 +0,0 @@
|
|||||||
# Configuration
|
|
||||||
|
|
||||||
## Fork this repository
|
|
||||||
|
|
||||||
Because this repository ([khuedoan/homelab](https://github.com/khuedoan/homelab)) applies GitOps practices,
|
|
||||||
it's the source of truth for my homelab, so you'll need to fork it to make it yours.
|
|
||||||
|
|
||||||
## Choose the environment
|
|
||||||
|
|
||||||
| Environment | Branch | Recommended setup |
|
|
||||||
| ----------- | -------- | ------------------------------------- |
|
|
||||||
| Production | `master` | Real hardware |
|
|
||||||
| Development | `dev` | A local [k3d](https://k3d.io) cluster |
|
|
||||||
|
|
||||||
For example, if you're trying out the dev VM, use the development environment:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
git checkout dev
|
|
||||||
```
|
|
||||||
|
|
||||||
<!-- TODO show complete workflow -->
|
|
||||||
|
|
||||||
## Run the configure script
|
|
||||||
|
|
||||||
Open the tools container if you haven't already:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
make tools
|
|
||||||
```
|
|
||||||
|
|
||||||
Run the following script to configure the homelab:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
make configure
|
|
||||||
```
|
|
||||||
|
|
||||||
Example input:
|
|
||||||
|
|
||||||
<!-- TODO update example input -->
|
|
||||||
|
|
||||||
```
|
|
||||||
Text editor (nvim):
|
|
||||||
Enter seed repo (github.com/khuedoan/homelab): github.com/example/homelab
|
|
||||||
Enter your domain (khuedoan.com): example.com
|
|
||||||
```
|
|
||||||
|
|
||||||
It will prompt you to edit the inventory, for example:
|
|
||||||
|
|
||||||
> - The IP addresses are the desired ones, not the current one, since your servers have no operating system installed yet.
|
|
||||||
> - Disk: based on `/dev/$DISK`, in my case it's `sda`, but yours can be `sdb`, `nvme0n1`...
|
|
||||||
> - Network interface: usually it's `eth0`, mine is `eno1`
|
|
||||||
> - MAC address: the **lowercase, colon separated** MAC address of the above network interface
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# metal/inventories/prod.yml
|
|
||||||
{{#include ../../../metal/inventories/prod.yml:3:}}
|
|
||||||
```
|
|
||||||
|
|
||||||
At the end it will show what has changed. After examining the diff, commit and push the changes.
|
|
||||||
@ -1,23 +0,0 @@
|
|||||||
# Deploy the homelab
|
|
||||||
|
|
||||||
> It will take a while to build the tools container and download Rocky Linux ISO on the first time
|
|
||||||
|
|
||||||
Open the tools container, which includes all the tools needed:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
make tools
|
|
||||||
```
|
|
||||||
|
|
||||||
Build the lab:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
make
|
|
||||||
```
|
|
||||||
|
|
||||||
Yes it's that simple! Check the status using the following command:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
watch ./scripts/get-status
|
|
||||||
```
|
|
||||||
|
|
||||||
While waiting for all services to become healthy, continue to the next section to update your DNS.
|
|
||||||
@ -1 +0,0 @@
|
|||||||
{{#include ../../../external/README.md}}
|
|
||||||
@ -1,144 +0,0 @@
|
|||||||
# Provisioning flow
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Everything is automated, after you edit the configuration files, you just need to run a single `make` command and it will:
|
|
||||||
|
|
||||||
- (1) Build the `./metal` layer:
|
|
||||||
- Create an ephemeral, stateless PXE server
|
|
||||||
- Install Linux on all servers in parallel
|
|
||||||
- Build a Kubernetes cluster (based on k3s)
|
|
||||||
- (2) Build the `./bootstrap` layer:
|
|
||||||
- Install ArgoCD
|
|
||||||
- Configure the root app to manage other layers (and also manage itself)
|
|
||||||
|
|
||||||
From now on, ArgoCD will do the rest:
|
|
||||||
|
|
||||||
- (3) Build the `./system` layer (storage, networking, monitoring, etc)
|
|
||||||
- (4) Build the `./platform` layer (Gitea, Vault, SSO, etc)
|
|
||||||
- (5) Build the `./apps` layer: (Syncthing, Jellyfin, etc)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
subgraph metal[./metal]
|
|
||||||
pxe[PXE Server] -.-> linux[Rocky Linux] --> k3s
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph bootstrap[./bootstrap]
|
|
||||||
argocd[ArgoCD] --> rootapp[Root app]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph system[./system]
|
|
||||||
metallb[MetalLB]
|
|
||||||
nginx[NGINX]
|
|
||||||
longhorn[Longhorn]
|
|
||||||
cert-manager
|
|
||||||
external-dns[External DNS]
|
|
||||||
cloudflared
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph external[./external]
|
|
||||||
letsencrypt[Let's Encrypt]
|
|
||||||
cloudflare[Cloudflare]
|
|
||||||
end
|
|
||||||
|
|
||||||
letsencrypt -.-> cert-manager
|
|
||||||
cloudflare -.-> cert-manager
|
|
||||||
cloudflare -.-> external-dns
|
|
||||||
cloudflare -.-> cloudflared
|
|
||||||
|
|
||||||
subgraph platform
|
|
||||||
gitea[Gitea]
|
|
||||||
tekton[Tekton]
|
|
||||||
vault[Vault]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph apps
|
|
||||||
jellyfin[Jellyfin]
|
|
||||||
matrix[Matrix]
|
|
||||||
paperless[Paperless]
|
|
||||||
seafile[Seafile]
|
|
||||||
end
|
|
||||||
|
|
||||||
make[Run make] -- 1 --> metal -- 2 --> bootstrap -. 3 .-> system -. 4 .-> platform -. 5 .-> apps
|
|
||||||
```
|
|
||||||
|
|
||||||
## Detailed steps
|
|
||||||
|
|
||||||
Below is the pseudo code for the entire process, you don't have to read it right now, but it will be handy for debugging.
|
|
||||||
|
|
||||||
```
|
|
||||||
Human run make:
|
|
||||||
build ./metal:
|
|
||||||
install the OS:
|
|
||||||
download the installer image and extract it
|
|
||||||
create a PXE server on the controller using Docker Compose:
|
|
||||||
DHCP server
|
|
||||||
TFTP server
|
|
||||||
HTTP server
|
|
||||||
create init config for each machine
|
|
||||||
turn the machines on via WoL
|
|
||||||
the machines boot:
|
|
||||||
select network boot automatically
|
|
||||||
broadcast DHCP request
|
|
||||||
DHCP server reply:
|
|
||||||
machine IP
|
|
||||||
TFTP server (next-server) IP
|
|
||||||
get boot files from TFTP server
|
|
||||||
GRUB
|
|
||||||
GRUB config with URL to init config based on MAC address
|
|
||||||
kernel
|
|
||||||
initrd
|
|
||||||
boot to the kernel
|
|
||||||
download from HTTP server:
|
|
||||||
init config from the URL in GRUB config
|
|
||||||
remaining files required to boot
|
|
||||||
install the OS based on the init config:
|
|
||||||
configure the system
|
|
||||||
remaining files required to install
|
|
||||||
reboot to the new OS
|
|
||||||
controller see the machines are ready
|
|
||||||
build a Kubernetes cluster:
|
|
||||||
download k3s binary
|
|
||||||
generate cluster token
|
|
||||||
copy k3s config files
|
|
||||||
enable k3s service and form a cluster
|
|
||||||
create KUBECONFIG file
|
|
||||||
create MetalLB config:
|
|
||||||
use the last /27 subnet of the network
|
|
||||||
apply the config
|
|
||||||
build ./bootstrap:
|
|
||||||
install ArgoCD:
|
|
||||||
apply helm chart
|
|
||||||
wait for status
|
|
||||||
install root app:
|
|
||||||
select values file:
|
|
||||||
if Gitea unreachable (first install):
|
|
||||||
get data from GitHub
|
|
||||||
else:
|
|
||||||
get data from Gitea
|
|
||||||
apply helm chart
|
|
||||||
wait for status
|
|
||||||
ArgoCD apply the rest:
|
|
||||||
clone git repo
|
|
||||||
install components based on directories:
|
|
||||||
./bootstrap (it manages itself):
|
|
||||||
argocd
|
|
||||||
root
|
|
||||||
./system:
|
|
||||||
storage
|
|
||||||
loadbalancer
|
|
||||||
ingress
|
|
||||||
etc
|
|
||||||
./platform (depends on ./system):
|
|
||||||
git:
|
|
||||||
migrate the homelab repository from GitHub
|
|
||||||
ArgoCD switch the source from GitHub to Gitea
|
|
||||||
ci
|
|
||||||
vault
|
|
||||||
etc
|
|
||||||
./apps (depends on ./system and ./platform):
|
|
||||||
homepage
|
|
||||||
jellyfin
|
|
||||||
etc
|
|
||||||
```
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
# Introduction
|
|
||||||
|
|
||||||
{{#include ../../README.md:introduction}}
|
|
||||||
|
|
||||||
Continue to the next section to try locally, or skip to the deployment guide to deploy on real hardware.
|
|
||||||
@ -1 +0,0 @@
|
|||||||
{{#include ../../LICENSE.md}}
|
|
||||||
@ -1,3 +0,0 @@
|
|||||||
# Reference
|
|
||||||
|
|
||||||
TODO
|
|
||||||
@ -1,30 +0,0 @@
|
|||||||
# Architecture
|
|
||||||
|
|
||||||
TODO
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
```
|
|
||||||
+--------------+
|
|
||||||
| ./apps |
|
|
||||||
|--------------|
|
|
||||||
| ./platform |
|
|
||||||
|--------------| +------------+
|
|
||||||
| ./system |- - - -| ./external |
|
|
||||||
|--------------| +------------+
|
|
||||||
| ./bootstrap |
|
|
||||||
|--------------|
|
|
||||||
| ./metal |
|
|
||||||
|--------------|
|
|
||||||
| HARDWARE |
|
|
||||||
+--------------+
|
|
||||||
```
|
|
||||||
|
|
||||||
- `./metal`: bare metal management, install Linux and Kubernetes
|
|
||||||
- `./bootstrap`: GitOps bootstrap with ArgoCD
|
|
||||||
- `./system`: critical system components for the cluster (load balancer, storage, ingress, operation tools...)
|
|
||||||
- `./platform`: essential components for service hosting platform (vault, git...)
|
|
||||||
- `./apps`: user facing applications
|
|
||||||
- `./external` (optional): externally managed services
|
|
||||||
- `./tools`: tools container, includes all the tools you'll need
|
|
||||||
- `./docs`: all documentation go here, this will generate a searchable web UI
|
|
||||||
@ -1,7 +0,0 @@
|
|||||||
# Contributors
|
|
||||||
|
|
||||||
Here is a list of the contributors who have helped improving my homelab. Big shout-out to them!
|
|
||||||
|
|
||||||
- Loc Mai ([@locmai](https://github.com/locmai))
|
|
||||||
|
|
||||||
If you feel you're missing from this list, feel free to add yourself in a PR.
|
|
||||||
@ -1,86 +0,0 @@
|
|||||||
# Roadmap
|
|
||||||
|
|
||||||
> Current status: **Alpha**
|
|
||||||
|
|
||||||
## Alpha requirements
|
|
||||||
|
|
||||||
Literally anything that works.
|
|
||||||
|
|
||||||
## Beta requirements
|
|
||||||
|
|
||||||
Good enough for tinkering and personal usage, and reasonably secure.
|
|
||||||
|
|
||||||
- [x] Automated bare metal provisioning
|
|
||||||
- [x] Controller set up (Docker)
|
|
||||||
- [x] OS installation (PXE boot)
|
|
||||||
- [x] Automated cluster creation (k3s)
|
|
||||||
- [x] Automated application deployment (ArgoCD)
|
|
||||||
- [x] Automated DNS management
|
|
||||||
- [x] Initialize GitOps repository on Gitea automatically
|
|
||||||
- [x] Observability
|
|
||||||
- [x] Monitoring
|
|
||||||
- [x] Logging
|
|
||||||
- [ ] Alerting
|
|
||||||
- [ ] SSO
|
|
||||||
- [ ] Reasonably secure
|
|
||||||
- [x] Automated certificate management
|
|
||||||
- [ ] Declarative secrets management
|
|
||||||
- [ ] Replace all default passwords with randomly generated ones
|
|
||||||
- [x] Expose services to the internet securely with Cloudflare Tunnel
|
|
||||||
- [x] Only use open-source technologies (except external managed services in `./external`)
|
|
||||||
- [x] Everything is defined as code
|
|
||||||
- [ ] Backup solution (3 copies, 2 seperate devices, 1 offsite)
|
|
||||||
- [ ] 70% availability (might break in the weekend due to new experimentation)
|
|
||||||
- [x] Core applications
|
|
||||||
- [x] Gitea
|
|
||||||
- [x] Tekton
|
|
||||||
- [x] Vault
|
|
||||||
- [x] Private container registry
|
|
||||||
- [x] Homepage
|
|
||||||
|
|
||||||
## Stable requirements
|
|
||||||
|
|
||||||
Can be used in "production" (for family or even small scale businesses).
|
|
||||||
|
|
||||||
- [x] A single command to deploy everything
|
|
||||||
- [x] Fast deployment time (from empty hard drive to running services in under 1 hour)
|
|
||||||
- [ ] Fully _automatic_, not just _automated_
|
|
||||||
- [x] Bare-metal OS rolling upgrade
|
|
||||||
- [x] Kubernetes version rolling upgrade
|
|
||||||
- [ ] Application version upgrade
|
|
||||||
- [ ] Encrypted backups
|
|
||||||
- [ ] Secrets rotation
|
|
||||||
- [x] Self healing
|
|
||||||
- [ ] Secure by default
|
|
||||||
- [ ] SELinux
|
|
||||||
- [ ] Network policy
|
|
||||||
- [ ] Static code analysis
|
|
||||||
- [ ] Chaos testing
|
|
||||||
- [ ] Minimal dependency on external services
|
|
||||||
- [ ] Complete documentation
|
|
||||||
- [x] Diagram as code
|
|
||||||
- [x] Book (this book)
|
|
||||||
- [ ] Walkthrough tutorial and feature demo (video)
|
|
||||||
- [x] Configuration script for new users
|
|
||||||
- [ ] 99,9% availability (less than 9 hours of downtime per year)
|
|
||||||
- [ ] 99,99% data durability
|
|
||||||
- [ ] Additional applications
|
|
||||||
- [ ] Matrix with bridges
|
|
||||||
- [ ] VPN server
|
|
||||||
- [ ] PeerTube
|
|
||||||
- [x] Seafile
|
|
||||||
- [x] Blog
|
|
||||||
- [ ] [Development dashboard](https://github.com/khuedoan/homelab-backstage)
|
|
||||||
|
|
||||||
## Unplanned
|
|
||||||
|
|
||||||
Nice to have
|
|
||||||
|
|
||||||
- [ ] Addition applications
|
|
||||||
- [ ] Mail server
|
|
||||||
- [ ] Air-gap install
|
|
||||||
- [ ] Automated testing
|
|
||||||
- [ ] Security audit
|
|
||||||
- [ ] Serverless (Knative)
|
|
||||||
- [ ] Cluster API (https://github.com/khuedoan/homelab/pull/2)
|
|
||||||
- [ ] Split DNS (requires a better router)
|
|
||||||
@ -1,22 +0,0 @@
|
|||||||
# Run commands on multiple nodes
|
|
||||||
|
|
||||||
Use [ansible-console](https://docs.ansible.com/ansible/latest/cli/ansible-console.html):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd metal
|
|
||||||
make console
|
|
||||||
```
|
|
||||||
|
|
||||||
Then enter the command(s) you want to run, for example:
|
|
||||||
|
|
||||||
```
|
|
||||||
root@all (4)[f:5]$ uptime
|
|
||||||
metal0 | CHANGED | rc=0 >>
|
|
||||||
10:52:02 up 2 min, 1 user, load average: 0.17, 0.15, 0.06
|
|
||||||
metal1 | CHANGED | rc=0 >>
|
|
||||||
10:52:02 up 2 min, 1 user, load average: 0.14, 0.11, 0.04
|
|
||||||
metal3 | CHANGED | rc=0 >>
|
|
||||||
10:52:02 up 2 min, 1 user, load average: 0.03, 0.02, 0.00
|
|
||||||
metal2 | CHANGED | rc=0 >>
|
|
||||||
10:52:02 up 2 min, 1 user, load average: 0.06, 0.06, 0.02
|
|
||||||
```
|
|
||||||
@ -8,12 +8,14 @@ To view PXE server (includes DHCP, TFTP and HTTP server) logs:
|
|||||||
./scripts/pxe-logs
|
./scripts/pxe-logs
|
||||||
```
|
```
|
||||||
|
|
||||||
You can view the logs of one or more containers selectively, for example:
|
!!! tip
|
||||||
|
|
||||||
```sh
|
You can view the logs of one or more containers selectively, for example:
|
||||||
./scripts/pxe-logs dhcp
|
|
||||||
./scripts/pxe-logs tftp http
|
```sh
|
||||||
```
|
./scripts/pxe-logs dhcp
|
||||||
|
./scripts/pxe-logs tftp http
|
||||||
|
```
|
||||||
|
|
||||||
## Nodes not booting from the network
|
## Nodes not booting from the network
|
||||||
|
|
||||||
@ -4,11 +4,13 @@ Or how to scale vertically. To replace the same node with a clean OS, remove it
|
|||||||
|
|
||||||
## Add new nodes
|
## Add new nodes
|
||||||
|
|
||||||
> You can add multiple nodes at the same time
|
!!! tip
|
||||||
|
|
||||||
Ensure that it meets the requirements in [prerequisites](../deployment/prerequisites.md), then add its details to the inventory **at the end of the group** (masters or workers):
|
You can add multiple nodes at the same time
|
||||||
|
|
||||||
```diff
|
Add its details to the inventory **at the end of the group** (masters or workers):
|
||||||
|
|
||||||
|
```diff title="metal/inventories/prod.yml"
|
||||||
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
|
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
|
||||||
index 7f6474a..1bb2cbc 100644
|
index 7f6474a..1bb2cbc 100644
|
||||||
--- a/metal/inventories/prod.yml
|
--- a/metal/inventories/prod.yml
|
||||||
@ -30,11 +32,13 @@ That's it!
|
|||||||
|
|
||||||
## Remove a node
|
## Remove a node
|
||||||
|
|
||||||
> It is recommended to remove nodes one at a time
|
!!! danger
|
||||||
|
|
||||||
|
It is recommended to remove nodes one at a time
|
||||||
|
|
||||||
Remove it from the inventory:
|
Remove it from the inventory:
|
||||||
|
|
||||||
```diff
|
```diff title="metal/inventories/prod.yml"
|
||||||
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
|
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
|
||||||
index 7f6474a..d12b50a 100644
|
index 7f6474a..d12b50a 100644
|
||||||
--- a/metal/inventories/prod.yml
|
--- a/metal/inventories/prod.yml
|
||||||
15
docs/tutorials/create-a-new-user-account.md
Normal file
15
docs/tutorials/create-a-new-user-account.md
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
# Create a new user account
|
||||||
|
|
||||||
|
## Create a new account
|
||||||
|
|
||||||
|
TODO
|
||||||
|
|
||||||
|
## Send initial password
|
||||||
|
|
||||||
|
Choose one of the methods listed below to send the initial password to the user:
|
||||||
|
|
||||||
|
- Share via password manager (if supported)
|
||||||
|
- Encrypt the password file and send it via email or chat
|
||||||
|
- Simply write or print the password on a piece of paper
|
||||||
|
|
||||||
|
On the first login, the user will be required to update their password.
|
||||||
@ -1,6 +1,8 @@
|
|||||||
# Expose services to the internet
|
# Expose services to the internet
|
||||||
|
|
||||||
> This tutorial is for Cloudflare Tunnel users, please skip if you use port-forwarding.
|
!!! info
|
||||||
|
|
||||||
|
This tutorial is for Cloudflare Tunnel users, please skip if you use port-forwarding.
|
||||||
|
|
||||||
Apply the `./external` layer to create a tunnel if you haven't already,
|
Apply the `./external` layer to create a tunnel if you haven't already,
|
||||||
then add the following annotations to your `Ingress` object (replace `example.com` with your domain):
|
then add the following annotations to your `Ingress` object (replace `example.com` with your domain):
|
||||||
@ -1,19 +1,27 @@
|
|||||||
# DNS setup
|
# Manual DNS setup
|
||||||
|
|
||||||
|
!!! info
|
||||||
|
|
||||||
|
Skip this step if you already use the included Cloudflare setup
|
||||||
|
|
||||||
Before you can access the home page at <https://home.example.com>, you'll need to update your DNS config.
|
Before you can access the home page at <https://home.example.com>, you'll need to update your DNS config.
|
||||||
|
|
||||||
Some options for DNS config (choose one):
|
Some options for DNS config (choose one):
|
||||||
|
|
||||||
- Change the DNS config at your domain registrar (easy to automate)
|
- Change the DNS config at your domain registrar (already included and automated)
|
||||||
- Change the DNS config in your router (also works with the [`home.arpa`](https://datatracker.ietf.org/doc/html/rfc8375) domain)
|
- Change the DNS config in your router (also works if you don't own a domain)
|
||||||
- Use [nip.io](https://nip.io) (suitable for a test environment)
|
- Use [nip.io](https://nip.io) (suitable for a test environment)
|
||||||
|
|
||||||
## At your domain registrar (recommended)
|
## At your domain registrar (recommended)
|
||||||
|
|
||||||
I'm using Cloudflare for DNS, continue to the next section for more information.
|
The default configuration is for Cloudflare DNS, but you can change the code to use other providers.
|
||||||
|
|
||||||
## In your router
|
## In your router
|
||||||
|
|
||||||
|
!!! tip
|
||||||
|
|
||||||
|
If you don't have a domain, you can use the `home.arpa` domain (according to [RFC-8375](https://datatracker.ietf.org/doc/html/rfc8375)).
|
||||||
|
|
||||||
You can add each subdomain one by one, or use a wildcard `*.example.com` and point it to the IP address of the load balancer.
|
You can add each subdomain one by one, or use a wildcard `*.example.com` and point it to the IP address of the load balancer.
|
||||||
To acquire a list of subdomains and their addresses, use this command:
|
To acquire a list of subdomains and their addresses, use this command:
|
||||||
|
|
||||||
@ -21,6 +29,6 @@ To acquire a list of subdomains and their addresses, use this command:
|
|||||||
./scripts/get-dns-config
|
./scripts/get-dns-config
|
||||||
```
|
```
|
||||||
|
|
||||||
## Use nip.io
|
## Use [nip.io](https://nip.io)
|
||||||
|
|
||||||
Preconfigured in the `dev` branch.
|
Preconfigured in the `dev` branch.
|
||||||
25
docs/tutorials/run-commands-on-multiple-nodes.md
Normal file
25
docs/tutorials/run-commands-on-multiple-nodes.md
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
# Run commands on multiple nodes
|
||||||
|
|
||||||
|
Use [ansible-console](https://docs.ansible.com/ansible/latest/cli/ansible-console.html):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd metal
|
||||||
|
make console
|
||||||
|
```
|
||||||
|
|
||||||
|
Then enter the command(s) you want to run.
|
||||||
|
|
||||||
|
!!! example
|
||||||
|
|
||||||
|
`root@all (4)[f:5]$ uptime`
|
||||||
|
|
||||||
|
```console
|
||||||
|
metal0 | CHANGED | rc=0 >>
|
||||||
|
10:52:02 up 2 min, 1 user, load average: 0.17, 0.15, 0.06
|
||||||
|
metal1 | CHANGED | rc=0 >>
|
||||||
|
10:52:02 up 2 min, 1 user, load average: 0.14, 0.11, 0.04
|
||||||
|
metal3 | CHANGED | rc=0 >>
|
||||||
|
10:52:02 up 2 min, 1 user, load average: 0.03, 0.02, 0.00
|
||||||
|
metal2 | CHANGED | rc=0 >>
|
||||||
|
10:52:02 up 2 min, 1 user, load average: 0.06, 0.06, 0.02
|
||||||
|
```
|
||||||
@ -6,9 +6,10 @@ Update the following changes, then commit and push.
|
|||||||
|
|
||||||
Set the `defaultClassReplicaCount` to 1:
|
Set the `defaultClassReplicaCount` to 1:
|
||||||
|
|
||||||
```yaml
|
```yaml title="system/longhorn-system/values.yaml" hl_lines="6"
|
||||||
# system/longhorn-system/values.yaml
|
--8<--
|
||||||
{{#include ../../../system/longhorn-system/values.yaml}}
|
system/longhorn-system/values.yaml
|
||||||
|
--8<--
|
||||||
```
|
```
|
||||||
|
|
||||||
## Disable automatic upgrade for OS and k3s
|
## Disable automatic upgrade for OS and k3s
|
||||||
28
docs/user-guide/onboarding.md
Normal file
28
docs/user-guide/onboarding.md
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
# Onboarding
|
||||||
|
|
||||||
|
## Create user
|
||||||
|
|
||||||
|
Ask an admin to [create your account](../tutorials/create-a-new-user-account.md), provide the following information:
|
||||||
|
|
||||||
|
- [ ] Full name
|
||||||
|
- [ ] Select a username
|
||||||
|
- [ ] Email address
|
||||||
|
|
||||||
|
## Install companion apps
|
||||||
|
|
||||||
|
For all users:
|
||||||
|
|
||||||
|
- [ ] [Password manager](#recommended-password-managers)
|
||||||
|
- [ ] [Matrix chat client](https://matrix.org/clients) (optional, you can use the web version)
|
||||||
|
|
||||||
|
For technical users:
|
||||||
|
|
||||||
|
- [ ] [Docker](https://docs.docker.com/engine/install)
|
||||||
|
- [ ] [Lens](https://k8slens.dev) (optional, you can use the included `kubectl` or `k9s` command in the tools container)
|
||||||
|
|
||||||
|
## Appendix
|
||||||
|
|
||||||
|
### Recommended password managers
|
||||||
|
|
||||||
|
- [Bitwarden](https://bitwarden.com/download) (easy to use, but requires an online account)
|
||||||
|
- [KeePassXC](https://keepassxc.org) (completely offline, but you'll need to sync manually)
|
||||||
@ -1,7 +0,0 @@
|
|||||||
# Bare-metal
|
|
||||||
|
|
||||||
- Ansible renders the configuration file for each bare metal machine (like IP, hostname...) and the PXE server from [templates](./roles/pxe_server/templates)
|
|
||||||
- The tools container creates sibling containers to build a PXE server (includes DHCP, TFTP and HTTP server)
|
|
||||||
- Ansible [wake the machines up](./roles/wake/tasks/main.yml) using Wake on LAN
|
|
||||||
- The machine start the boot process, the OS get installed (through PXE server) and the machine reboots to the new operating system
|
|
||||||
- Ansible build a Kubernetes cluster based on k3s
|
|
||||||
68
mkdocs.yml
Normal file
68
mkdocs.yml
Normal file
@ -0,0 +1,68 @@
|
|||||||
|
# yaml-language-server: $schema=https://squidfunk.github.io/mkdocs-material/schema.json
|
||||||
|
|
||||||
|
site_name: Khue's Homelab
|
||||||
|
copyright: Copyright © 2020 - 2022 Khue Doan
|
||||||
|
|
||||||
|
repo_url: https://github.com/khuedoan/homelab
|
||||||
|
|
||||||
|
theme:
|
||||||
|
name: material
|
||||||
|
palette:
|
||||||
|
primary: black
|
||||||
|
features:
|
||||||
|
- navigation.indexes
|
||||||
|
- navigation.expand
|
||||||
|
- search.highlight
|
||||||
|
- search.share
|
||||||
|
|
||||||
|
markdown_extensions:
|
||||||
|
- pymdownx.emoji:
|
||||||
|
emoji_index: !!python/name:materialx.emoji.twemoji
|
||||||
|
emoji_generator: !!python/name:materialx.emoji.to_svg
|
||||||
|
- attr_list
|
||||||
|
- admonition
|
||||||
|
- pymdownx.details
|
||||||
|
- pymdownx.snippets:
|
||||||
|
check_paths: true
|
||||||
|
- def_list
|
||||||
|
- pymdownx.tasklist:
|
||||||
|
- pymdownx.superfences:
|
||||||
|
custom_fences:
|
||||||
|
- name: mermaid
|
||||||
|
class: mermaid
|
||||||
|
format: !!python/name:pymdownx.superfences.fence_code_format
|
||||||
|
|
||||||
|
nav:
|
||||||
|
- Home: index.md
|
||||||
|
- Installation:
|
||||||
|
- installation/development.md
|
||||||
|
- Production:
|
||||||
|
- installation/production/prerequisites.md
|
||||||
|
- installation/production/external-resources.md
|
||||||
|
- installation/production/configuration.md
|
||||||
|
- installation/production/deployment.md
|
||||||
|
- installation/production/post-installation.md
|
||||||
|
- Tutorials:
|
||||||
|
- tutorials/manual-dns-setup.md
|
||||||
|
- tutorials/create-a-new-user-account.md
|
||||||
|
- tutorials/expose-services-to-the-internet.md
|
||||||
|
- tutorials/use-both-github-and-gitea.md
|
||||||
|
- tutorials/add-or-remove-nodes.md
|
||||||
|
- tutorials/run-commands-on-multiple-nodes.md
|
||||||
|
- tutorials/single-node-cluster-adjustments.md
|
||||||
|
- Runbooks:
|
||||||
|
- runbooks/argocd.md
|
||||||
|
- runbooks/cert-manager.md
|
||||||
|
- runbooks/gitea.md
|
||||||
|
- runbooks/longhorn.md
|
||||||
|
- runbooks/vault.md
|
||||||
|
- troubleshooting.md
|
||||||
|
- User guide:
|
||||||
|
- user-guide/onboarding.md
|
||||||
|
- Reference:
|
||||||
|
- reference/architecture.md
|
||||||
|
- reference/license.md
|
||||||
|
- reference/changelog.md
|
||||||
|
- reference/roadmap.md
|
||||||
|
- reference/contributing.md
|
||||||
|
- reference/faq.md
|
||||||
@ -1,3 +0,0 @@
|
|||||||
# Scripts
|
|
||||||
|
|
||||||
Quick (and maybe dirty) scripts to automate common tasks.
|
|
||||||
@ -4,8 +4,6 @@ cloudflared:
|
|||||||
config:
|
config:
|
||||||
tunnel: homelab
|
tunnel: homelab
|
||||||
ingress:
|
ingress:
|
||||||
# It is safe to put a wildcard here
|
|
||||||
# Please see https://homelab.khuedoan.com/reference/faq.html#is-it-safe-to-use-wildcard-in-cloudflare-tunnel-ingress-config
|
|
||||||
- hostname: '*.khuedoan.com'
|
- hostname: '*.khuedoan.com'
|
||||||
service: https://ingress-nginx-controller.ingress-nginx
|
service: https://ingress-nginx-controller.ingress-nginx
|
||||||
originRequest:
|
originRequest:
|
||||||
|
|||||||
@ -1,3 +0,0 @@
|
|||||||
# Tools container
|
|
||||||
|
|
||||||
Includes all necessary command line tools to provision and maintain the homelab.
|
|
||||||
Reference in New Issue
Block a user