mirrors/khuedoan-homelab
1
0
Fork 0
mirror of https://github.com/khuedoan/homelab.git synced 2024-12-22 15:14:31 +07:00
Code Issues Packages Projects Releases Wiki Activity

refactor(docs): migrate to mkdocs (#68)

Browse Source 
* refactor(docs): migrate to mkdocs

* More markdown

* Admonitions
This commit is contained in:
Khue Doan 2022-07-06 12:33:35 +07:00 committed by GitHub
parent a7f91505a5
commit cd41343580
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
51 changed files with 794 additions and 742 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
Makefile
View File

@ -34,3 +34,12 @@ docs:
dev:
make -C metal cluster env=dev
make -C bootstrap
docs:
docker run \
--rm \
--interactive \
--tty \
--publish 8000:8000 \
--volume $(shell pwd):/docs \
squidfunk/mkdocs-material

344
README.md
View File

@ -1,10 +1,7 @@
<div align="center">
# Khue's Homelab
<!-- ANCHOR: introduction -->
**[Features](#features) • [Get Started](#get-started) • [Documentation](https://homelab.khuedoan.com)**
[![chat](https://img.shields.io/matrix/homelab:matrix.khuedoan.com?style=flat-square&logo=matrix&logoColor=white&label=chat)](https://matrix.to/#/#homelab:matrix.khuedoan.com)
[![tag](https://img.shields.io/github/v/tag/khuedoan/homelab?style=flat-square&logo=semver&logoColor=white)](https://github.com/khuedoan/homelab/tags)
[![document](https://img.shields.io/website?label=document&logo=gitbook&logoColor=white&style=flat-square&url=https%3A%2F%2Fhomelab.khuedoan.com)](https://homelab.khuedoan.com)
[![license](https://img.shields.io/github/license/khuedoan/homelab?style=flat-square&logo=gnu&logoColor=white)](https://www.gnu.org/licenses/gpl-3.0.html)
@ -13,33 +10,34 @@
This project utilizes [Infrastructure as Code](https://en.wikipedia.org/wiki/Infrastructure_as_code) and [GitOps](https://www.weave.works/technologies/gitops) to automate provisioning, operating, and updating self-hosted services in my homelab.
It can be used as a highly customizable framework to build your own homelab.
<!-- ANCHOR_END: introduction -->
Current status: **ALPHA**
</div>
> **What is a homelab?**
>
> Homelab is a laboratory at home where you can self-host, experiment with new technologies, practice for certifications, and so on.
> For more information about homelab in general, see the [r/homelab introduction](https://www.reddit.com/r/homelab/wiki/introduction).
## Overview
This section provides a high level overview of the project.
For further information, please see the [documentation](https://homelab.khuedoan.com).
Project status: **ALPHA**
This project is still in the experimental stage, and I don't use anything critical on it.
Expect breaking changes that may require a complete redeployment.
A proper upgrade path is planned for the stable release.
More information can be found in [the roadmap](#roadmap) below.
### Hardware
![Hardware](https://user-images.githubusercontent.com/27996771/98970963-25137200-2543-11eb-8f2d-f9a2d45756ef.JPG)
- 4 × NEC SFF `PC-MK26ECZDR` (Japanese version of the ThinkCentre M700):
- CPU: `Intel Core i5-6600T @ 2.70GHz`
- RAM: `16GB`
- SSD: `128GB`
- CPU: `Intel Core i5-6600T @ 2.70GHz`
- RAM: `16GB`
- SSD: `128GB`
- TP-Link `TL-SG108` switch:
- Ports: `8`
- Speed: `1000Mbps`
- Ports: `8`
- Speed: `1000Mbps`
### Features
Project status: **Alpha** (see [roadmap](#roadmap) below)
- [x] Common applications: Gitea, Seafile, Jellyfin, Paperless...
- [x] Automated bare metal provisioning with PXE boot
- [x] Automated Kubernetes installation and management
@ -58,198 +56,172 @@ Project status: **Alpha** (see [roadmap](#roadmap) below)
- [ ] Automated offsite backups 🚧
- [ ] Single sign-on 🚧
Some demo videos and screenshots are shown here.
They can't capture all of the project's features, but they are sufficient to get a concept of it.
Some demo videos and screenshots are shown here (click to enlarge).
They can't capture all the project's features, but they are sufficient to get a concept of it.
| [![Deployment](https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7.svg)](https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7) |
| :--: |
| Deploy with a single command (after updating the config files of course) |
| [![][screenshot-01]](https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7) | [![][screenshot-02]](https://www.youtube.com/watch?v=y-d7btNNAT8) |
| :--: | :--: |
| Deploy with a single command (after updating the configuration files) | PXE boot |
| [![][screenshot-03]][screenshot-03] | [![][screenshot-04]][screenshot-04] |
| Homepage with Ingress discovery powered by [Hajimari](https://github.com/toboshii/hajimari) | Monitoring dashboard powered by [Grafana](https://grafana.com/) |
| [![][screenshot-05]][screenshot-05] | [![][screenshot-06]][screenshot-06] |
| Git server powered by [Gitea](https://gitea.io/en-us/) | [Matrix](https://matrix.org/) chat server |
| [![][screenshot-07]][screenshot-07] | [![][screenshot-08]][screenshot-08] |
| Continuous integration with [Tekton](https://tekton.dev/) | Continuous deployment with [ArgoCD](https://argoproj.github.io/cd/) |
| [![][screenshot-09]][screenshot-09] | [![][screenshot-10]][screenshot-10] |
| Cluster management using [Lens](https://k8slens.dev/) | Secret management with [Vault](https://www.vaultproject.io/) |
| [![PXE boot](https://user-images.githubusercontent.com/27996771/157303477-df2e7410-8f02-4648-a86c-71e6b7e89e35.png)](https://www.youtube.com/watch?v=y-d7btNNAT8) |
| :--: |
| PXE boot |
| ![](https://user-images.githubusercontent.com/27996771/149445807-0f869eb7-d8f5-4fef-ab97-ac281df91a06.png) |
| :--: |
| Homepage with Ingress discovery powered by [Hajimari](https://github.com/toboshii/hajimari) |
| ![](https://user-images.githubusercontent.com/27996771/149444871-38889c9d-862f-41ff-8c05-8ece21da3e9c.png) |
| :--: |
| Git server powered by [Gitea](https://gitea.io/en-us/) |
| ![](https://user-images.githubusercontent.com/27996771/149445374-58fd0605-bb9a-46e4-81d6-5e584d2b94a9.png) |
| :--: |
| Continuous integration with [Tekton](https://tekton.dev/) |
| ![](https://user-images.githubusercontent.com/27996771/149444716-fc0d7282-4cf7-4ddb-97a4-1a3fb47ff2b8.png) |
| :--: |
| Continuous deployment with [ArgoCD](https://argoproj.github.io/cd/) |
| ![](https://user-images.githubusercontent.com/27996771/149446631-1c5d056b-1fdc-48e6-96ba-e1abe1762be0.png) |
| :--: |
| Monitoring dashboard powered by [Grafana](https://grafana.com/) |
| ![](https://user-images.githubusercontent.com/27996771/149448510-7163310c-2049-4ccd-901d-f11f605bfc32.png) |
| :--: |
| [Matrix](https://matrix.org/) chat server powered by [Element](https://matrix.org/docs/projects/client/element) and [Dendrite](https://matrix.org/docs/projects/server/dendrite) |
| ![](https://user-images.githubusercontent.com/27996771/149448896-9d79947d-468c-45c6-a81d-b43654e8ab6b.png) |
| :--: |
| Cluster management using [Lens](https://k8slens.dev/) (or you can just use `kubectl`) |
| ![](https://user-images.githubusercontent.com/27996771/149452309-de4a893b-e94c-4ba8-9119-ea87449cf77e.png) |
| :--: |
| Secret management with [Vault](https://www.vaultproject.io/) |
[screenshot-01]: https://asciinema.org/a/xkBRkwC6e9RAzVuMDXH3nGHp7.svg
[screenshot-02]: https://user-images.githubusercontent.com/27996771/157303477-df2e7410-8f02-4648-a86c-71e6b7e89e35.png
[screenshot-03]: https://user-images.githubusercontent.com/27996771/149445807-0f869eb7-d8f5-4fef-ab97-ac281df91a06.png
[screenshot-04]: https://user-images.githubusercontent.com/27996771/149446631-1c5d056b-1fdc-48e6-96ba-e1abe1762be0.png
[screenshot-05]: https://user-images.githubusercontent.com/27996771/149444871-38889c9d-862f-41ff-8c05-8ece21da3e9c.png
[screenshot-06]: https://user-images.githubusercontent.com/27996771/149448510-7163310c-2049-4ccd-901d-f11f605bfc32.png
[screenshot-07]: https://user-images.githubusercontent.com/27996771/149445374-58fd0605-bb9a-46e4-81d6-5e584d2b94a9.png
[screenshot-08]: https://user-images.githubusercontent.com/27996771/149444716-fc0d7282-4cf7-4ddb-97a4-1a3fb47ff2b8.png
[screenshot-09]: https://user-images.githubusercontent.com/27996771/149448896-9d79947d-468c-45c6-a81d-b43654e8ab6b.png
[screenshot-10]: https://user-images.githubusercontent.com/27996771/149452309-de4a893b-e94c-4ba8-9119-ea87449cf77e.png
### Tech stack
<table>
<tr>
<th>Logo</th>
<th>Name</th>
<th>Description</th>
</tr>
<tr>
<td><img width="32" src="https://simpleicons.org/icons/ansible.svg"></td>
<td><a href="https://www.ansible.com">Ansible</a></td>
<td>Automate bare metal provisioning and configuration</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/argo/icon/color/argo-icon-color.svg"></td>
<td><a href="https://argoproj.github.io/cd">ArgoCD</a></td>
<td>GitOps tool built to deploy applications to Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/jetstack/cert-manager/raw/master/logo/logo.png"></td>
<td><a href="https://cert-manager.io">cert-manager</a></td>
<td>Cloud native certificate management</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/314135?s=200&v=4"></td>
<td><a href="https://www.cloudflare.com">Cloudflare</a></td>
<td>DNS and Tunnel</td>
</tr>
<tr>
<td><img width="32" src="https://www.docker.com/wp-content/uploads/2022/03/Moby-logo.png"></td>
<td><a href="https://www.docker.com">Docker</a></td>
<td>Ephermeral PXE server and convenient tools container</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/kubernetes-sigs/external-dns/raw/master/docs/img/external-dns.png"></td>
<td><a href="https://github.com/kubernetes-sigs/external-dns">ExternalDNS</a></td>
<td>Synchronizes exposed Kubernetes Services and Ingresses with DNS providers</td>
</tr>
<tr>
<td><img width="32" src="https://upload.wikimedia.org/wikipedia/commons/b/bb/Gitea_Logo.svg"></td>
<td><a href="https://gitea.com">Gitea</a></td>
<td>Self-hosted Git service</td>
</tr>
<tr>
<td><img width="32" src="https://grafana.com/static/img/menu/grafana2.svg"></td>
<td><a href="https://grafana.com">Grafana</a></td>
<td>Operational dashboards</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/helm/icon/color/helm-icon-color.svg"></td>
<td><a href="https://helm.sh">Helm</a></td>
<td>The package manager for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/k3s/icon/color/k3s-icon-color.svg"></td>
<td><a href="https://k3s.io">K3s</a></td>
<td>Lightweight distribution of Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/kubernetes/icon/color/kubernetes-icon-color.svg"></td>
<td><a href="https://kubernetes.io">Kubernetes</a></td>
<td>Container-orchestration system, the backbone of this project</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/grafana/loki/blob/main/docs/sources/logo.png?raw=true"></td>
<td><a href="https://grafana.com/oss/loki">Loki</a></td>
<td>Log aggregation system</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/longhorn/icon/color/longhorn-icon-color.svg"></td>
<td><a href="https://longhorn.io">Longhorn</a></td>
<td>Cloud native distributed block storage for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/60239468?s=200&v=4"></td>
<td><a href="https://metallb.org">MetalLB</a></td>
<td>Bare metal load-balancer for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/1412239?s=200&v=4"></td>
<td><a href="https://www.nginx.com">NGINX</a></td>
<td>Kubernetes Ingress Controller</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/prometheus/icon/color/prometheus-icon-color.svg"></td>
<td><a href="https://prometheus.io">Prometheus</a></td>
<td>Systems monitoring and alerting toolkit</td>
</tr>
<tr>
<td><img width="32" src="https://docs.renovatebot.com/assets/images/logo.png"></td>
<td><a href="https://www.whitesourcesoftware.com/free-developer-tools/renovate">Renovate</a></td>
<td>Automatically update dependencies</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/75713131?s=200&v=4"></td>
<td><a href="https://rockylinux.org">Rocky Linux</a></td>
<td>Base OS for Kubernetes nodes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/47602533?s=200&v=4"></td>
<td><a href="https://tekton.dev">Tekton</a></td>
<td>Cloud native solution for building CI/CD systems</td>
</tr>
<tr>
<td><img width="32" src="https://trow.io/trow.png"></td>
<td><a href="https://trow.io">Trow</a></td>
<td>Private container registry</td>
</tr>
<tr>
<td><img width="32" src="https://simpleicons.org/icons/vault.svg"></td>
<td><a href="https://www.vaultproject.io">Vault</a></td>
<td>Secrets and encryption management system</td>
</tr>
<tr>
<th>Logo</th>
<th>Name</th>
<th>Description</th>
</tr>
<tr>
<td><img width="32" src="https://simpleicons.org/icons/ansible.svg"></td>
<td><a href="https://www.ansible.com">Ansible</a></td>
<td>Automate bare metal provisioning and configuration</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/argo/icon/color/argo-icon-color.svg"></td>
<td><a href="https://argoproj.github.io/cd">ArgoCD</a></td>
<td>GitOps tool built to deploy applications to Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/jetstack/cert-manager/raw/master/logo/logo.png"></td>
<td><a href="https://cert-manager.io">cert-manager</a></td>
<td>Cloud native certificate management</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/314135?s=200&v=4"></td>
<td><a href="https://www.cloudflare.com">Cloudflare</a></td>
<td>DNS and Tunnel</td>
</tr>
<tr>
<td><img width="32" src="https://www.docker.com/wp-content/uploads/2022/03/Moby-logo.png"></td>
<td><a href="https://www.docker.com">Docker</a></td>
<td>Ephermeral PXE server and convenient tools container</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/kubernetes-sigs/external-dns/raw/master/docs/img/external-dns.png"></td>
<td><a href="https://github.com/kubernetes-sigs/external-dns">ExternalDNS</a></td>
<td>Synchronizes exposed Kubernetes Services and Ingresses with DNS providers</td>
</tr>
<tr>
<td><img width="32" src="https://upload.wikimedia.org/wikipedia/commons/b/bb/Gitea_Logo.svg"></td>
<td><a href="https://gitea.com">Gitea</a></td>
<td>Self-hosted Git service</td>
</tr>
<tr>
<td><img width="32" src="https://grafana.com/static/img/menu/grafana2.svg"></td>
<td><a href="https://grafana.com">Grafana</a></td>
<td>Operational dashboards</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/helm/icon/color/helm-icon-color.svg"></td>
<td><a href="https://helm.sh">Helm</a></td>
<td>The package manager for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/k3s/icon/color/k3s-icon-color.svg"></td>
<td><a href="https://k3s.io">K3s</a></td>
<td>Lightweight distribution of Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/kubernetes/icon/color/kubernetes-icon-color.svg"></td>
<td><a href="https://kubernetes.io">Kubernetes</a></td>
<td>Container-orchestration system, the backbone of this project</td>
</tr>
<tr>
<td><img width="32" src="https://github.com/grafana/loki/blob/main/docs/sources/logo.png?raw=true"></td>
<td><a href="https://grafana.com/oss/loki">Loki</a></td>
<td>Log aggregation system</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/longhorn/icon/color/longhorn-icon-color.svg"></td>
<td><a href="https://longhorn.io">Longhorn</a></td>
<td>Cloud native distributed block storage for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/60239468?s=200&v=4"></td>
<td><a href="https://metallb.org">MetalLB</a></td>
<td>Bare metal load-balancer for Kubernetes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/1412239?s=200&v=4"></td>
<td><a href="https://www.nginx.com">NGINX</a></td>
<td>Kubernetes Ingress Controller</td>
</tr>
<tr>
<td><img width="32" src="https://cncf-branding.netlify.app/img/projects/prometheus/icon/color/prometheus-icon-color.svg"></td>
<td><a href="https://prometheus.io">Prometheus</a></td>
<td>Systems monitoring and alerting toolkit</td>
</tr>
<tr>
<td><img width="32" src="https://docs.renovatebot.com/assets/images/logo.png"></td>
<td><a href="https://www.whitesourcesoftware.com/free-developer-tools/renovate">Renovate</a></td>
<td>Automatically update dependencies</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/75713131?s=200&v=4"></td>
<td><a href="https://rockylinux.org">Rocky Linux</a></td>
<td>Base OS for Kubernetes nodes</td>
</tr>
<tr>
<td><img width="32" src="https://avatars.githubusercontent.com/u/47602533?s=200&v=4"></td>
<td><a href="https://tekton.dev">Tekton</a></td>
<td>Cloud native solution for building CI/CD systems</td>
</tr>
<tr>
<td><img width="32" src="https://trow.io/trow.png"></td>
<td><a href="https://trow.io">Trow</a></td>
<td>Private container registry</td>
</tr>
<tr>
<td><img width="32" src="https://simpleicons.org/icons/vault.svg"></td>
<td><a href="https://www.vaultproject.io">Vault</a></td>
<td>Secrets and encryption management system</td>
</tr>
</table>
## Get Started
- [Try it out locally](https://homelab.khuedoan.com/try-locally.html) without any hardware
- [Deploy on real hardware](https://homelab.khuedoan.com/deployment) for real workload
- [Try it out locally](https://homelab.khuedoan.com/installation/development.md) without any hardware
- [Deploy on real hardware](https://homelab.khuedoan.com/installation/production/prerequisites.md) for production workload
## Roadmap
See [roadmap](https://homelab.khuedoan.com/roadmap.html) and [open issues](https://github.com/khuedoan/homelab/issues) for a list of proposed features and known issues.
See [roadmap](https://homelab.khuedoan.com/reference/roadmap) and [open issues](https://github.com/khuedoan/homelab/issues) for a list of proposed features and known issues.
## Contributing
Any contributions you make, either big or small, are greatly appreciated.
Please see [contributing guide](https://homelab.khuedoan.com/reference/contributing) for more information.
## License
> Copyright (c) 2020, 2021, 2022 Khue Doan
Copyright &copy; 2020 - 2022 Khue Doan
<details>
<summary>Distributed under the GPLv3 License.</summary>
This project is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This project is distributed in the hope that it will be useful, but **WITHOUT ANY WARRANTY**; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this project (`LICENSE.md`).
If not, see <https://www.gnu.org/licenses>.
</details>
Distributed under the GPLv3 License.
See [license page](https://homelab.khuedoan.com/reference/license) or `LICENSE.md` file for more information.
## Acknowledgements
- [ArgoCD usage in my coworker's homelab](https://github.com/locmai/humble)
- [ArgoCD usage and monitoring configuration in locmai/humble](https://github.com/locmai/humble)
- [README template](https://github.com/othneildrew/Best-README-Template)
- [Run the same Cloudflare Tunnel across many `cloudflared` processes](https://developers.cloudflare.com/cloudflare-one/tutorials/many-cfd-one-tunnel)
- [MAC address environment variable in GRUB config](https://askubuntu.com/questions/1272400/how-do-i-automate-network-installation-of-many-ubuntu-18-04-systems-with-efi-and)

10
docs/Makefile
View File

@ -1,10 +0,0 @@
.POSIX:
default: book
mermaid*.js:
mdbook-mermaid install .
.PHONY: book
book: mermaid*.js
mdbook build .

11
docs/README.md
View File

@ -1,11 +0,0 @@
# Documents
Documents can be viewed at <https://homelab.khuedoan.com>.
It's running on my other cluster in the [khuedoan/horus](https://github.com/khuedoan/horus) project
(so if the homelab goes down I can still read the documentation).
To view locally, install [`mdbook`](https://github.com/rust-lang/mdBook#installation) and run:
```
mdbook serve
```

18
docs/book.toml
View File

@ -1,18 +0,0 @@
[book]
authors = ["Khue Doan"]
language = "en"
multilingual = false
src = "src"
title = "Khue's Homelab"
[preprocessor.mermaid]
command = "mdbook-mermaid"
[output.html]
additional-js = ["mermaid.min.js", "mermaid-init.js"]
git-repository-url = "https://github.com/khuedoan/homelab"
edit-url-template = "https://github.com/khuedoan/homelab/edit/master/docs/{path}"
# TODO deprecate this after 6 months
[output.html.redirect]
"/try-on-a-vm.html" = "/try-locally.html"

3
docs/index.md Normal file
View File

@ -0,0 +1,3 @@
--8<--
README.md
--8<--

8
docs/src/try-locally.md → docs/installation/development.md
View File

@ -1,4 +1,4 @@
# Try locally
# Development environment
## Caveats compare to production environment
@ -17,15 +17,15 @@ Host machine:
- OS: Linux (Windows and macOS will not work due to networking limitations, you can use a Linux VM)
- Recommended hardware specifications:
- CPU: 4 cores
- RAM: 16 GiB
- CPU: 4 cores
- RAM: 16 GiB
Install the following packages:
- `docker`
- `make`
Clone the repository (follow the [configuration guide](./deployment/configuration.md) if you want to customize it):
Clone the repository and checkout the development branch:
```sh
git clone https://github.com/khuedoan/homelab

44
docs/installation/production/configuration.md Normal file
View File

@ -0,0 +1,44 @@
# Configuration
Open the tools container if you haven't already:
```sh
make tools
```
!!! note
It will take a while to build the tools container on the first time
Run the following script to configure the homelab:
```sh
make configure
```
!!! example
<!-- TODO update example input -->
```
Text editor (nvim):
Enter seed repo (github.com/khuedoan/homelab): github.com/example/homelab
Enter your domain (khuedoan.com): example.com
```
It will prompt you to edit the inventory:
- IP address: the desired one, not the current one, since your servers have no operating system installed yet
- Disk: based on `/dev/$DISK`, in my case it's `sda`, but yours can be `sdb`, `nvme0n1`...
- Network interface: usually it's `eth0`, mine is `eno1`
- MAC address: the **lowercase, colon separated** MAC address of the above network interface
!!! example
```yaml title="metal/inventories/prod.yml"
--8<--
metal/inventories/prod.yml
--8<--
```
At the end it will show what has changed. After examining the diff, commit and push the changes.

21
docs/installation/production/deployment.md Normal file
View File

@ -0,0 +1,21 @@
# Deployment
Open the tools container, which includes all the tools needed:
```sh
make tools
```
Build the lab:
```sh
make
```
!!! note
It will take a while to download Rocky Linux ISO on the first time
Yes it's that simple!
You can read the [architecture document](../../reference/architecture.md) while waiting for the deployment to complete.

32
external/README.md → docs/installation/production/external-resources.md
View File

@ -1,8 +1,8 @@
# External resources
**WIP documents**
!!! info
> These resources are optional, the homelab still works without them but will lack some features like trusted certificates and offsite backup
These resources are optional, the homelab still works without them but will lack some features like trusted certificates and offsite backup
Although I try to keep the amount of external resources to the minimum, there's still need for a few of them.
Below is a list of external resources and why we need them (also see some [alternatives](#alternatives) below).
@ -12,15 +12,17 @@ Below is a list of external resources and why we need them (also see some [alter
| Terraform Cloud | Workspace | Terraform state backend |
| Cloudflare | DNS | DNS and [DNS-01 challenge](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) for certificates |
| Cloudflare | Tunnel | Public services to the internet without port-forwarding |
| Minio | Bucket | Onsite backup |
| AWS | S3 Glacier | Offsite backup |
<!-- | Minio | Bucket | Onsite backup | -->
<!-- | AWS | S3 Glacier | Offsite backup | -->
This layer will:
- Create external resources
- Add external secrets to namespaces
## Prerequisites
## Create credentials
You'll be asked to provide these credentials on first build.
### Create Terraform workspace
@ -52,25 +54,17 @@ If you decide to use a [different Terraform backend](https://www.terraform.io/la
<!-- └── Is in - 117.xxx.xxx.xxx, 2402:xxx:xxx:xxx:xxx:xxx:xxx:xxx -->
<!-- ``` -->
### Create Minio keys
<!-- ### Create Minio keys -->
TODO: skip this for now
<!-- TODO: skip this for now -->
### Create AWS API key
<!-- ### Create AWS API key -->
TODO: skip this for now
## Deploy
Apply Terraform (you will be prompted to log in to Terraform Cloud and enter API keys from the previous steps):
```sh
# From the project root
make external
```
<!-- TODO: skip this for now -->
## Alternatives
- Terraform Cloud: any other [Terraform backends](https://www.terraform.io/language/settings/backends)
- Cloudflare DNS: see [manual DNS setup](../../tutorials/manual-dns-setup.md)
- Cloudflare Tunnel: you can create a small VPS in the cloud and utilize Wireguard and HAProxy to route traffic via it, or just use simple port-forwarding if it's available (see also [awesome tunneling](https://github.com/anderspitman/awesome-tunneling))
- Minio and S3 Glacier: any S3 compatible object storage, such as Backblaze B2, Minio...
<!-- - Minio and S3 Glacier: any S3 compatible object storage, such as Backblaze B2, Minio... -->

2
docs/src/deployment/post-installation.md → docs/installation/production/post-installation.md
View File

@ -14,4 +14,4 @@ Save the following files to a safe location (like a password manager):
## Next steps
TODO
- [User onboarding](../../user-guide/onboarding.md)

77
docs/src/deployment/prerequisites.md → docs/installation/production/prerequisites.md
View File

@ -1,25 +1,46 @@
# Prerequisites
## Fork this repository
Because [this project](https://github.com/khuedoan/homelab) applies GitOps practices,
it's the source of truth for _my_ homelab, so you'll need to fork it to make it yours:
[:fontawesome-solid-code-fork: Fork khuedoan/homelab](https://github.com/khuedoan/homelab/fork){ .md-button }
By using this project you agree to [the license](/license).
!!! summary "License TL;DR"
- This project is free to use for any purpose, but it comes with no warranty
- You must use the same [GPLv3 license](https://www.gnu.org/licenses/gpl-3.0.en.html) in `LICENSE.md`
- You must keep the copy right notice and/or include an acknowledgement
- Your project must remain open-source
## Hardware requirements
### Initial controller
> The initial controller is the machine used to bootstrap the cluster, we only need it once, you can use your laptop or desktop
!!! info
- Any machine that can run Docker with the `host` networking driver ([which means only Docker on Linux hosts](https://docs.docker.com/network/host/), you can use a Linux virtual machine with bridged networking if you're on macOS or Windows)
The initial controller is the machine used to bootstrap the cluster, we only need it once, you can use your laptop or desktop
- A Linux machine that can run Docker (because the `host` networking driver used for PXE boot [only supports Linux](https://docs.docker.com/network/host/), you can use a Linux virtual machine with bridged networking if you're on macOS or Windows).
### Servers
Any modern `x86_64` computer(s) should work, you can use old PCs, laptops or servers.
> This is the requirements for _each_ node
!!! info
This is the requirements for _each_ node
| Component | Minimum | Recommended |
| :-- | :-- | :-- |
| CPU | 2 cores | 4 cores |
| RAM | 8 GB | 16 GB |
| Hard drive | 128 GB | 512 GB (depending on your storage usage, the base installation will not use more than 128GB) |
| Node count | 1 (checkout the [single node cluster adjustments](../tutorials/single-node-cluster-adjustments.md) tutorial) | 3 or more for high availability |
| Node count | 1 (checkout the [single node cluster adjustments](../../tutorials/single-node-cluster-adjustments.md) tutorial) | 3 or more for high availability |
Additional capabilities:
@ -28,18 +49,20 @@ Additional capabilities:
### Network setup
- All servers must be connected to the same **wired** network with the initial controller (Wifi is untested, please let me know if it works)
- All servers must be connected to the same **wired** network with the initial controller
- You have the access to change DNS config (on your router or at your domain registrar)
## Domain
Buying a domain is highly recommended, but if you don't have one, you can also update your router config and point [`*.home.arpa`](https://datatracker.ietf.org/doc/html/rfc8375) to the load balancer (more on that later).
Buying a domain is highly recommended, but if you don't have one, see [manual DNS setup](../../tutorials/manual-dns-setup.md).
## BIOS setup
> You need to do it once per machine if the default config is not sufficent,
> usually for consumer hardware this can not be automated
> (it requires something like [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) to automate).
!!! info
You need to do it once per machine if the default config is not sufficent,
usually for consumer hardware this can not be automated
(it requires something like [IPMI](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) to automate).
Common settings:
@ -52,24 +75,26 @@ Boot order options (select one, each has their pros and cons):
1. Only boot from the network if no operating system found: works on most hardware but you need to manually wipe your hard drive or delete the existing boot record for the current OS
2. Prefer booting from the network if turned on via WoL: more convenience but your BIOS must support it, and you must test it throughly to ensure you don't accidentally wipe your servers
Below is my BIOS setup for reference. Your motherboard may have a different name for the options, so you'll need to adapt it to your hardware.
!!! example
```yaml
Devices:
NetworkSetup:
PXEIPv4: true
PXEIPv6: false
Advanced:
CPUSetup:
VT-d: true
Power:
AutomaticPowerOn:
WoL: Automatic # Use network boot if Wake-on-LAN
Security:
SecureBoot: false
Startup:
CSM: false
```
Below is my BIOS setup for reference. Your motherboard may have a different name for the options, so you'll need to adapt it to your hardware.
```yaml
Devices:
NetworkSetup:
PXEIPv4: true
PXEIPv6: false
Advanced:
CPUSetup:
VT-d: true
Power:
AutomaticPowerOn:
WoL: Automatic # Use network boot if Wake-on-LAN
Security:
SecureBoot: false
Startup:
CSM: false
```
## Gather information

177
docs/reference/architecture.md Normal file
View File

@ -0,0 +1,177 @@
# Architecture
## Components
```
+--------------+
| ./apps |
|--------------|
| ./platform |
|--------------| +------------+
| ./system |- - - -| ./external |
|--------------| +------------+
| ./bootstrap |
|--------------|
| ./metal |
|--------------|
| HARDWARE |
+--------------+
```
Main components:
- `./metal`: bare metal management, install Linux and Kubernetes
- `./bootstrap`: GitOps bootstrap with ArgoCD
- `./system`: critical system components for the cluster (load balancer, storage, ingress, operation tools...)
- `./platform`: essential components for service hosting platform (vault, git...)
- `./apps`: user facing applications
- `./external` (optional): externally managed services
Support components:
- `./tools`: tools container, includes all the tools you'll need
- `./docs`: all documentation go here, this will generate a searchable web UI
- `./scripts`: scripts to automate common tasks
## Provisioning flow
Everything is automated, after you edit the configuration files, you just need to run a single `make` command and it will:
- (1) Build the `./metal` layer:
- Create an ephemeral, stateless PXE server
- Install Linux on all servers in parallel
- Build a Kubernetes cluster (based on k3s)
- (2) Build the `./bootstrap` layer:
- Install ArgoCD
- Configure the root app to manage other layers (and also manage itself)
From now on, ArgoCD will do the rest:
- (3) Build the `./system` layer (storage, networking, monitoring, etc)
- (4) Build the `./platform` layer (Gitea, Vault, SSO, etc)
- (5) Build the `./apps` layer: (Syncthing, Jellyfin, etc)
```mermaid
flowchart TD
subgraph metal[./metal]
pxe[PXE Server] -.-> linux[Rocky Linux] --> k3s
end
subgraph bootstrap[./bootstrap]
argocd[ArgoCD] --> rootapp[Root app]
end
subgraph system[./system]
metallb[MetalLB]
nginx[NGINX]
longhorn[Longhorn]
cert-manager
external-dns[External DNS]
cloudflared
end
subgraph external[./external]
letsencrypt[Let's Encrypt]
cloudflare[Cloudflare]
end
letsencrypt -.-> cert-manager
cloudflare -.-> cert-manager
cloudflare -.-> external-dns
cloudflare -.-> cloudflared
subgraph platform
gitea[Gitea]
tekton[Tekton]
vault[Vault]
end
subgraph apps
jellyfin[Jellyfin]
matrix[Matrix]
paperless[Paperless]
seafile[Seafile]
end
make[Run make] -- 1 --> metal -- 2 --> bootstrap -. 3 .-> system -. 4 .-> platform -. 5 .-> apps
```
Below is the pseudo code for the entire process, you don't have to read it right now, but it will be handy for debugging.
??? detailed "Detailed provisioning flow"
```
Human run make:
build ./metal:
install the OS:
download the installer image and extract it
create a PXE server on the controller using Docker Compose:
DHCP server
TFTP server
HTTP server
create init config for each machine
turn the machines on via WoL
the machines boot:
select network boot automatically
broadcast DHCP request
DHCP server reply:
machine IP
TFTP server (next-server) IP
get boot files from TFTP server
GRUB
GRUB config with URL to init config based on MAC address
kernel
initrd
boot to the kernel
download from HTTP server:
init config from the URL in GRUB config
remaining files required to boot
install the OS based on the init config:
configure the system
remaining files required to install
reboot to the new OS
controller see the machines are ready
build a Kubernetes cluster:
download k3s binary
generate cluster token
copy k3s config files
enable k3s service and form a cluster
create KUBECONFIG file
create MetalLB config:
use the last /27 subnet of the network
apply the config
build ./bootstrap:
install ArgoCD:
apply helm chart
wait for status
install root app:
select values file:
if Gitea unreachable (first install):
get data from GitHub
else:
get data from Gitea
apply helm chart
wait for status
ArgoCD apply the rest:
clone git repo
install components based on directories:
./bootstrap (it manages itself):
argocd
root
./system:
storage
loadbalancer
ingress
etc
./platform (depends on ./system):
git:
migrate the homelab repository from GitHub
ArgoCD switch the source from GitHub to Gitea
ci
vault
etc
./apps (depends on ./system and ./platform):
homepage
jellyfin
etc
```

6
docs/src/changelog.md → docs/reference/changelog.md
View File

@ -24,9 +24,9 @@
- Upgrade to Kubernetes 1.23
- Support external resources:
- Cloudflare DNS and Tunnel
- Backblaze for backup
- Auto inject secrets to required namespaces
- Cloudflare DNS and Tunnel
- Backblaze for backup
- Auto inject secrets to required namespaces
- Replace self-signed certificates with Let's Encrypt production (with API token injected from the `external` layer)
- Add DNS records automatically using external-dns
- Easy Cloudflare Tunnel configuration with annotations

33
docs/reference/contributing.md Normal file
View File

@ -0,0 +1,33 @@
# Contributing
## How to contribute
### Bug report
TODO
### Merge request
TODO
### Documentation
Documents can be viewed at <https://homelab.khuedoan.com>.
It's running on my other cluster in the [khuedoan/horus](https://github.com/khuedoan/horus) project
(so if the homelab goes down I can still read the documentation).
To edit and view locally, run:
```
make docs
```
Then visit [localhost:8000](http://localhost:8000)
## Contributors
Here is a list of the contributors who have helped improving my homelab. Big shout-out to them!
- Loc Mai ([@locmai](https://github.com/locmai))
If you feel you're missing from this list, feel free to add yourself in a PR.

0
docs/src/reference/faq.md → docs/reference/faq.md
View File

5
docs/reference/license.md Normal file
View File

@ -0,0 +1,5 @@
Copyright &copy; 2020 - 2022 Khue Doan
--8<--
LICENSE.md
--8<--

91
docs/reference/roadmap.md Normal file
View File

@ -0,0 +1,91 @@
# Roadmap
!!! info
Current status: **ALPHA**
## Alpha requirements
Literally anything that works.
## Beta requirements
Good enough for tinkering and personal usage, and reasonably secure.
- [x] Automated bare metal provisioning
- [x] Controller set up (Docker)
- [x] OS installation (PXE boot)
- [x] Automated cluster creation (k3s)
- [x] Automated application deployment (ArgoCD)
- [x] Automated DNS management
- [x] Initialize GitOps repository on Gitea automatically
- [x] Observability
- [x] Monitoring
- [x] Logging
- [ ] Alerting
- [ ] SSO
- [ ] Reasonably secure
- [x] Automated certificate management
- [x] Declarative secret management
- [ ] Replace all default passwords with randomly generated ones
- [x] Expose services to the internet securely with Cloudflare Tunnel
- [x] Only use open-source technologies (except external managed services in `./external`)
- [x] Everything is defined as code
- [ ] Backup solution (3 copies, 2 seperate devices, 1 offsite)
- [ ] Define [SLOs](https://en.wikipedia.org/wiki/Service-level_objective):
- [ ] 70% availability (might break in the weekend due to new experimentation)
- [x] Core applications
- [x] Gitea
- [x] Tekton
- [x] Vault
- [x] Private container registry
- [x] Homepage
## Stable requirements
Can be used in "production" (for family or even small scale businesses).
- [x] A single command to deploy everything
- [x] Fast deployment time (from empty hard drive to running services in under 1 hour)
- [ ] Fully _automatic_, not just _automated_
- [x] Bare-metal OS rolling upgrade
- [x] Kubernetes version rolling upgrade
- [x] Application version upgrade
- [ ] Encrypted backups
- [ ] Secrets rotation
- [x] Self healing
- [ ] Secure by default
- [ ] SELinux
- [ ] Network policies
- [ ] Static code analysis
- [ ] Chaos testing
- [x] Minimal dependency on external services
- [ ] Complete documentation
- [x] Diagram as code
- [x] Book (this book)
- [ ] Walkthrough tutorial and feature demo (video)
- [x] Configuration script for new users
- [ ] SLOs:
- [ ] 99,9% availability (less than 9 hours of downtime per year)
- [ ] 99,99% data durability
- [ ] Clear upgrade path
- [ ] Additional applications
- [ ] Matrix with bridges
- [ ] VPN server
- [ ] PeerTube
- [x] Seafile
- [x] Blog
- [ ] [Development dashboard](https://github.com/khuedoan/homelab-backstage)
## Unplanned
Nice to have
- [ ] Addition applications
- [ ] Mail server
- [ ] Air-gap install
- [ ] Automated testing
- [ ] Security audit
- [ ] Serverless ([Knative](https://knative.dev))
- [ ] Cluster API ([last attempt](https://github.com/khuedoan/homelab/pull/2))
- [ ] Split DNS (requires a better router)

1
docs/runbooks/argocd.md Normal file
View File

@ -0,0 +1 @@
# ArgoCD

1
docs/runbooks/cert-manager.md Normal file
View File

@ -0,0 +1 @@
# cert-manager

1
docs/runbooks/gitea.md Normal file
View File

@ -0,0 +1 @@
# Gitea

1
docs/runbooks/longhorn.md Normal file
View File

@ -0,0 +1 @@
# Longhorn

8
docs/src/reference/secret-management.md → docs/runbooks/vault.md
View File

@ -1,4 +1,4 @@
# Secret management
# Vault
## Overview
@ -7,8 +7,10 @@
- Secrets that can be generated are automatically generated and stored in Vault.
- Integrate with GitOps using [External Secrets Operator](https://external-secrets.io)
> Despite the name "_External_ Secrets Operator", our Vault is deployed on the same cluster.
> HashiCorp Vault can be replaced with AWS Secret Manager, Google Cloud Secret Manager, Azure Key Vault, etc.
!!! info
Despite the name _External_ Secrets Operator, our Vault is deployed on the same cluster.
HashiCorp Vault can be replaced with AWS Secret Manager, Google Cloud Secret Manager, Azure Key Vault, etc.
```mermaid
flowchart TD

1
docs/src/CNAME
View File

@ -1 +0,0 @@
homelab.khuedoan.com

35
docs/src/SUMMARY.md
View File

@ -1,35 +0,0 @@
# Summary
- [Introduction](introduction.md)
- [Try locally](try-locally.md)
- [Deployment](./deployment/README.md)
- [Provisioning flow](./deployment/provisioning-flow.md)
- [Prerequisites](./deployment/prerequisites.md)
- [Configuration](./deployment/configuration.md)
- [Deploy the homelab](./deployment/deployment.md)
- [DNS setup](./deployment/dns.md)
- [External resources (optional)](./deployment/external-resources.md)
- [Post-installation](./deployment/post-installation.md)
- [Troubleshooting](./troubleshooting.md)
- [Tutorials]()
- [Use both GitHub and Gitea](./tutorials/use-both-github-and-gitea.md)
- [Single node cluster adjustments](./tutorials/single-node-cluster-adjustments.md)
- [Add or remove nodes (scale up or down)](./tutorials/add-or-remove-nodes.md)
- [Run commands on multiple nodes](./tutorials/run-commands-on-multiple-nodes.md)
- [Install new applications]()
- [Expose services to the internet](tutorials/expose-services-to-the-internet.md)
- [Runbooks]()
- [ArgoCD]()
- [Gitea]()
- [Vault]()
- [Reference](./reference/README.md)
- [Architecture](./reference/architecture.md)
- [Secret management](./reference/secret-management.md)
- [FAQ](./reference/faq.md)
- [Contributors](./reference/contributors.md)
---
- [Changelog](./changelog.md)
- [Roadmap](./roadmap.md)
- [License](./license.md)

10
docs/src/deployment/README.md
View File

@ -1,10 +0,0 @@
# Deployment
In this section you will learn how to:
- Prepare your hardware
- Update the configurations to fit your needs (hardware info, domain name...)
- Build the cluster and bootstrap it
- Perform some addtional steps depending on your set up (DNS, external secrets...)
If you encounter any issue, please see the [Troubleshooting](../troubleshooting.md) section.

59
docs/src/deployment/configuration.md
View File

@ -1,59 +0,0 @@
# Configuration
## Fork this repository
Because this repository ([khuedoan/homelab](https://github.com/khuedoan/homelab)) applies GitOps practices,
it's the source of truth for my homelab, so you'll need to fork it to make it yours.
## Choose the environment
| Environment | Branch | Recommended setup |
| ----------- | -------- | ------------------------------------- |
| Production | `master` | Real hardware |
| Development | `dev` | A local [k3d](https://k3d.io) cluster |
For example, if you're trying out the dev VM, use the development environment:
```sh
git checkout dev
```
<!-- TODO show complete workflow -->
## Run the configure script
Open the tools container if you haven't already:
```sh
make tools
```
Run the following script to configure the homelab:
```sh
make configure
```
Example input:
<!-- TODO update example input -->
```
Text editor (nvim):
Enter seed repo (github.com/khuedoan/homelab): github.com/example/homelab
Enter your domain (khuedoan.com): example.com
```
It will prompt you to edit the inventory, for example:
> - The IP addresses are the desired ones, not the current one, since your servers have no operating system installed yet.
> - Disk: based on `/dev/$DISK`, in my case it's `sda`, but yours can be `sdb`, `nvme0n1`...
> - Network interface: usually it's `eth0`, mine is `eno1`
> - MAC address: the **lowercase, colon separated** MAC address of the above network interface
```yaml
# metal/inventories/prod.yml
{{#include ../../../metal/inventories/prod.yml:3:}}
```
At the end it will show what has changed. After examining the diff, commit and push the changes.

23
docs/src/deployment/deployment.md
View File

@ -1,23 +0,0 @@
# Deploy the homelab
> It will take a while to build the tools container and download Rocky Linux ISO on the first time
Open the tools container, which includes all the tools needed:
```sh
make tools
```
Build the lab:
```sh
make
```
Yes it's that simple! Check the status using the following command:
```sh
watch ./scripts/get-status
```
While waiting for all services to become healthy, continue to the next section to update your DNS.

1
docs/src/deployment/external-resources.md
View File

@ -1 +0,0 @@
{{#include ../../../external/README.md}}

144
docs/src/deployment/provisioning-flow.md
View File

@ -1,144 +0,0 @@
# Provisioning flow
## Overview
Everything is automated, after you edit the configuration files, you just need to run a single `make` command and it will:
- (1) Build the `./metal` layer:
- Create an ephemeral, stateless PXE server
- Install Linux on all servers in parallel
- Build a Kubernetes cluster (based on k3s)
- (2) Build the `./bootstrap` layer:
- Install ArgoCD
- Configure the root app to manage other layers (and also manage itself)
From now on, ArgoCD will do the rest:
- (3) Build the `./system` layer (storage, networking, monitoring, etc)
- (4) Build the `./platform` layer (Gitea, Vault, SSO, etc)
- (5) Build the `./apps` layer: (Syncthing, Jellyfin, etc)
```mermaid
flowchart TD
subgraph metal[./metal]
pxe[PXE Server] -.-> linux[Rocky Linux] --> k3s
end
subgraph bootstrap[./bootstrap]
argocd[ArgoCD] --> rootapp[Root app]
end
subgraph system[./system]
metallb[MetalLB]
nginx[NGINX]
longhorn[Longhorn]
cert-manager
external-dns[External DNS]
cloudflared
end
subgraph external[./external]
letsencrypt[Let's Encrypt]
cloudflare[Cloudflare]
end
letsencrypt -.-> cert-manager
cloudflare -.-> cert-manager
cloudflare -.-> external-dns
cloudflare -.-> cloudflared
subgraph platform
gitea[Gitea]
tekton[Tekton]
vault[Vault]
end
subgraph apps
jellyfin[Jellyfin]
matrix[Matrix]
paperless[Paperless]
seafile[Seafile]
end
make[Run make] -- 1 --> metal -- 2 --> bootstrap -. 3 .-> system -. 4 .-> platform -. 5 .-> apps
```
## Detailed steps
Below is the pseudo code for the entire process, you don't have to read it right now, but it will be handy for debugging.
```
Human run make:
build ./metal:
install the OS:
download the installer image and extract it
create a PXE server on the controller using Docker Compose:
DHCP server
TFTP server
HTTP server
create init config for each machine
turn the machines on via WoL
the machines boot:
select network boot automatically
broadcast DHCP request
DHCP server reply:
machine IP
TFTP server (next-server) IP
get boot files from TFTP server
GRUB
GRUB config with URL to init config based on MAC address
kernel
initrd
boot to the kernel
download from HTTP server:
init config from the URL in GRUB config
remaining files required to boot
install the OS based on the init config:
configure the system
remaining files required to install
reboot to the new OS
controller see the machines are ready
build a Kubernetes cluster:
download k3s binary
generate cluster token
copy k3s config files
enable k3s service and form a cluster
create KUBECONFIG file
create MetalLB config:
use the last /27 subnet of the network
apply the config
build ./bootstrap:
install ArgoCD:
apply helm chart
wait for status
install root app:
select values file:
if Gitea unreachable (first install):
get data from GitHub
else:
get data from Gitea
apply helm chart
wait for status
ArgoCD apply the rest:
clone git repo
install components based on directories:
./bootstrap (it manages itself):
argocd
root
./system:
storage
loadbalancer
ingress
etc
./platform (depends on ./system):
git:
migrate the homelab repository from GitHub
ArgoCD switch the source from GitHub to Gitea
ci
vault
etc
./apps (depends on ./system and ./platform):
homepage
jellyfin
etc
```

5
docs/src/introduction.md
View File

@ -1,5 +0,0 @@
# Introduction
{{#include ../../README.md:introduction}}
Continue to the next section to try locally, or skip to the deployment guide to deploy on real hardware.

1
docs/src/license.md
View File

@ -1 +0,0 @@
{{#include ../../LICENSE.md}}

3
docs/src/reference/README.md
View File

@ -1,3 +0,0 @@
# Reference
TODO

30
docs/src/reference/architecture.md
View File

@ -1,30 +0,0 @@
# Architecture
TODO
## Components
```
+--------------+
| ./apps |
|--------------|
| ./platform |
|--------------| +------------+
| ./system |- - - -| ./external |
|--------------| +------------+
| ./bootstrap |
|--------------|
| ./metal |
|--------------|
| HARDWARE |
+--------------+
```
- `./metal`: bare metal management, install Linux and Kubernetes
- `./bootstrap`: GitOps bootstrap with ArgoCD
- `./system`: critical system components for the cluster (load balancer, storage, ingress, operation tools...)
- `./platform`: essential components for service hosting platform (vault, git...)
- `./apps`: user facing applications
- `./external` (optional): externally managed services
- `./tools`: tools container, includes all the tools you'll need
- `./docs`: all documentation go here, this will generate a searchable web UI

7
docs/src/reference/contributors.md
View File

@ -1,7 +0,0 @@
# Contributors
Here is a list of the contributors who have helped improving my homelab. Big shout-out to them!
- Loc Mai ([@locmai](https://github.com/locmai))
If you feel you're missing from this list, feel free to add yourself in a PR.

86
docs/src/roadmap.md
View File

@ -1,86 +0,0 @@
# Roadmap
> Current status: **Alpha**
## Alpha requirements
Literally anything that works.
## Beta requirements
Good enough for tinkering and personal usage, and reasonably secure.
- [x] Automated bare metal provisioning
- [x] Controller set up (Docker)
- [x] OS installation (PXE boot)
- [x] Automated cluster creation (k3s)
- [x] Automated application deployment (ArgoCD)
- [x] Automated DNS management
- [x] Initialize GitOps repository on Gitea automatically
- [x] Observability
- [x] Monitoring
- [x] Logging
- [ ] Alerting
- [ ] SSO
- [ ] Reasonably secure
- [x] Automated certificate management
- [ ] Declarative secrets management
- [ ] Replace all default passwords with randomly generated ones
- [x] Expose services to the internet securely with Cloudflare Tunnel
- [x] Only use open-source technologies (except external managed services in `./external`)
- [x] Everything is defined as code
- [ ] Backup solution (3 copies, 2 seperate devices, 1 offsite)
- [ ] 70% availability (might break in the weekend due to new experimentation)
- [x] Core applications
- [x] Gitea
- [x] Tekton
- [x] Vault
- [x] Private container registry
- [x] Homepage
## Stable requirements
Can be used in "production" (for family or even small scale businesses).
- [x] A single command to deploy everything
- [x] Fast deployment time (from empty hard drive to running services in under 1 hour)
- [ ] Fully _automatic_, not just _automated_
- [x] Bare-metal OS rolling upgrade
- [x] Kubernetes version rolling upgrade
- [ ] Application version upgrade
- [ ] Encrypted backups
- [ ] Secrets rotation
- [x] Self healing
- [ ] Secure by default
- [ ] SELinux
- [ ] Network policy
- [ ] Static code analysis
- [ ] Chaos testing
- [ ] Minimal dependency on external services
- [ ] Complete documentation
- [x] Diagram as code
- [x] Book (this book)
- [ ] Walkthrough tutorial and feature demo (video)
- [x] Configuration script for new users
- [ ] 99,9% availability (less than 9 hours of downtime per year)
- [ ] 99,99% data durability
- [ ] Additional applications
- [ ] Matrix with bridges
- [ ] VPN server
- [ ] PeerTube
- [x] Seafile
- [x] Blog
- [ ] [Development dashboard](https://github.com/khuedoan/homelab-backstage)
## Unplanned
Nice to have
- [ ] Addition applications
- [ ] Mail server
- [ ] Air-gap install
- [ ] Automated testing
- [ ] Security audit
- [ ] Serverless (Knative)
- [ ] Cluster API (https://github.com/khuedoan/homelab/pull/2)
- [ ] Split DNS (requires a better router)

22
docs/src/tutorials/run-commands-on-multiple-nodes.md
View File

@ -1,22 +0,0 @@
# Run commands on multiple nodes
Use [ansible-console](https://docs.ansible.com/ansible/latest/cli/ansible-console.html):
```sh
cd metal
make console
```
Then enter the command(s) you want to run, for example:
```
root@all (4)[f:5]$ uptime
metal0 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.17, 0.15, 0.06
metal1 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.14, 0.11, 0.04
metal3 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.03, 0.02, 0.00
metal2 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.06, 0.06, 0.02
```

12
docs/src/troubleshooting.md → docs/troubleshooting.md
View File

@ -8,12 +8,14 @@ To view PXE server (includes DHCP, TFTP and HTTP server) logs:
./scripts/pxe-logs
```
You can view the logs of one or more containers selectively, for example:
!!! tip
```sh
./scripts/pxe-logs dhcp
./scripts/pxe-logs tftp http
```
You can view the logs of one or more containers selectively, for example:
```sh
./scripts/pxe-logs dhcp
./scripts/pxe-logs tftp http
```
## Nodes not booting from the network

14
docs/src/tutorials/add-or-remove-nodes.md → docs/tutorials/add-or-remove-nodes.md
View File

@ -4,11 +4,13 @@ Or how to scale vertically. To replace the same node with a clean OS, remove it
## Add new nodes
> You can add multiple nodes at the same time
!!! tip
Ensure that it meets the requirements in [prerequisites](../deployment/prerequisites.md), then add its details to the inventory **at the end of the group** (masters or workers):
You can add multiple nodes at the same time
```diff
Add its details to the inventory **at the end of the group** (masters or workers):
```diff title="metal/inventories/prod.yml"
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
index 7f6474a..1bb2cbc 100644
--- a/metal/inventories/prod.yml
@ -30,11 +32,13 @@ That's it!
## Remove a node
> It is recommended to remove nodes one at a time
!!! danger
It is recommended to remove nodes one at a time
Remove it from the inventory:
```diff
```diff title="metal/inventories/prod.yml"
diff --git a/metal/inventories/prod.yml b/metal/inventories/prod.yml
index 7f6474a..d12b50a 100644
--- a/metal/inventories/prod.yml

15
docs/tutorials/create-a-new-user-account.md Normal file
View File

@ -0,0 +1,15 @@
# Create a new user account
## Create a new account
TODO
## Send initial password
Choose one of the methods listed below to send the initial password to the user:
- Share via password manager (if supported)
- Encrypt the password file and send it via email or chat
- Simply write or print the password on a piece of paper
On the first login, the user will be required to update their password.

4
docs/src/tutorials/expose-services-to-the-internet.md → docs/tutorials/expose-services-to-the-internet.md
View File

@ -1,6 +1,8 @@
# Expose services to the internet
> This tutorial is for Cloudflare Tunnel users, please skip if you use port-forwarding.
!!! info
This tutorial is for Cloudflare Tunnel users, please skip if you use port-forwarding.
Apply the `./external` layer to create a tunnel if you haven't already,
then add the following annotations to your `Ingress` object (replace `example.com` with your domain):

18
docs/src/deployment/dns.md → docs/tutorials/manual-dns-setup.md
View File

@ -1,19 +1,27 @@
# DNS setup
# Manual DNS setup
!!! info
Skip this step if you already use the included Cloudflare setup
Before you can access the home page at <https://home.example.com>, you'll need to update your DNS config.
Some options for DNS config (choose one):
- Change the DNS config at your domain registrar (easy to automate)
- Change the DNS config in your router (also works with the [`home.arpa`](https://datatracker.ietf.org/doc/html/rfc8375) domain)
- Change the DNS config at your domain registrar (already included and automated)
- Change the DNS config in your router (also works if you don't own a domain)
- Use [nip.io](https://nip.io) (suitable for a test environment)
## At your domain registrar (recommended)
I'm using Cloudflare for DNS, continue to the next section for more information.
The default configuration is for Cloudflare DNS, but you can change the code to use other providers.
## In your router
!!! tip
If you don't have a domain, you can use the `home.arpa` domain (according to [RFC-8375](https://datatracker.ietf.org/doc/html/rfc8375)).
You can add each subdomain one by one, or use a wildcard `*.example.com` and point it to the IP address of the load balancer.
To acquire a list of subdomains and their addresses, use this command:
@ -21,6 +29,6 @@ To acquire a list of subdomains and their addresses, use this command:
./scripts/get-dns-config
```
## Use nip.io
## Use [nip.io](https://nip.io)
Preconfigured in the `dev` branch.

25
docs/tutorials/run-commands-on-multiple-nodes.md Normal file
View File

@ -0,0 +1,25 @@
# Run commands on multiple nodes
Use [ansible-console](https://docs.ansible.com/ansible/latest/cli/ansible-console.html):
```sh
cd metal
make console
```
Then enter the command(s) you want to run.
!!! example
`root@all (4)[f:5]$ uptime`
```console
metal0 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.17, 0.15, 0.06
metal1 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.14, 0.11, 0.04
metal3 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.03, 0.02, 0.00
metal2 | CHANGED | rc=0 >>
10:52:02 up 2 min, 1 user, load average: 0.06, 0.06, 0.02
```

7
docs/src/tutorials/single-node-cluster-adjustments.md → docs/tutorials/single-node-cluster-adjustments.md
View File

@ -6,9 +6,10 @@ Update the following changes, then commit and push.
Set the `defaultClassReplicaCount` to 1:
```yaml
# system/longhorn-system/values.yaml
{{#include ../../../system/longhorn-system/values.yaml}}
```yaml title="system/longhorn-system/values.yaml" hl_lines="6"
--8<--
system/longhorn-system/values.yaml
--8<--
```
## Disable automatic upgrade for OS and k3s

0
docs/src/tutorials/use-both-github-and-gitea.md → docs/tutorials/use-both-github-and-gitea.md
View File

28
docs/user-guide/onboarding.md Normal file
View File

@ -0,0 +1,28 @@
# Onboarding
## Create user
Ask an admin to [create your account](../tutorials/create-a-new-user-account.md), provide the following information:
- [ ] Full name
- [ ] Select a username
- [ ] Email address
## Install companion apps
For all users:
- [ ] [Password manager](#recommended-password-managers)
- [ ] [Matrix chat client](https://matrix.org/clients) (optional, you can use the web version)
For technical users:
- [ ] [Docker](https://docs.docker.com/engine/install)
- [ ] [Lens](https://k8slens.dev) (optional, you can use the included `kubectl` or `k9s` command in the tools container)
## Appendix
### Recommended password managers
- [Bitwarden](https://bitwarden.com/download) (easy to use, but requires an online account)
- [KeePassXC](https://keepassxc.org) (completely offline, but you'll need to sync manually)

7
metal/README.md
View File

@ -1,7 +0,0 @@
# Bare-metal
- Ansible renders the configuration file for each bare metal machine (like IP, hostname...) and the PXE server from [templates](./roles/pxe_server/templates)
- The tools container creates sibling containers to build a PXE server (includes DHCP, TFTP and HTTP server)
- Ansible [wake the machines up](./roles/wake/tasks/main.yml) using Wake on LAN
- The machine start the boot process, the OS get installed (through PXE server) and the machine reboots to the new operating system
- Ansible build a Kubernetes cluster based on k3s

68
mkdocs.yml Normal file
View File

@ -0,0 +1,68 @@
# yaml-language-server: $schema=https://squidfunk.github.io/mkdocs-material/schema.json
site_name: Khue's Homelab
copyright: Copyright &copy; 2020 - 2022 Khue Doan
repo_url: https://github.com/khuedoan/homelab
theme:
name: material
palette:
primary: black
features:
- navigation.indexes
- navigation.expand
- search.highlight
- search.share
markdown_extensions:
- pymdownx.emoji:
emoji_index: !!python/name:materialx.emoji.twemoji
emoji_generator: !!python/name:materialx.emoji.to_svg
- attr_list
- admonition
- pymdownx.details
- pymdownx.snippets:
check_paths: true
- def_list
- pymdownx.tasklist:
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
nav:
- Home: index.md
- Installation:
- installation/development.md
- Production:
- installation/production/prerequisites.md
- installation/production/external-resources.md
- installation/production/configuration.md
- installation/production/deployment.md
- installation/production/post-installation.md
- Tutorials:
- tutorials/manual-dns-setup.md
- tutorials/create-a-new-user-account.md
- tutorials/expose-services-to-the-internet.md
- tutorials/use-both-github-and-gitea.md
- tutorials/add-or-remove-nodes.md
- tutorials/run-commands-on-multiple-nodes.md
- tutorials/single-node-cluster-adjustments.md
- Runbooks:
- runbooks/argocd.md
- runbooks/cert-manager.md
- runbooks/gitea.md
- runbooks/longhorn.md
- runbooks/vault.md
- troubleshooting.md
- User guide:
- user-guide/onboarding.md
- Reference:
- reference/architecture.md
- reference/license.md
- reference/changelog.md
- reference/roadmap.md
- reference/contributing.md
- reference/faq.md

3
scripts/README.md
View File

@ -1,3 +0,0 @@
# Scripts
Quick (and maybe dirty) scripts to automate common tasks.

2
system/cloudflared/values.yaml
View File

@ -4,8 +4,6 @@ cloudflared:
config:
tunnel: homelab
ingress:
# It is safe to put a wildcard here
# Please see https://homelab.khuedoan.com/reference/faq.html#is-it-safe-to-use-wildcard-in-cloudflare-tunnel-ingress-config
- hostname: '*.khuedoan.com'
service: https://ingress-nginx-controller.ingress-nginx
originRequest:

3
tools/README.md
View File

@ -1,3 +0,0 @@
# Tools container
Includes all necessary command line tools to provision and maintain the homelab.