Gitops
Architecture
The homelab consists of several more or less interdependent repositories. This approach lets me configure and monitor the whole environment as code. Apart from the Flux repository, everything lives on the local Gitlab instance.
Repositories
Provisioning & Maintenance
proxmox-vm-templates
Used to build virtual machine templates for every Linux distribution that provides cloud-init images. Cloud-init makes it possible to upload SSH keys, install software, and pre-configure services.
homelab-tasks
The repository connects to Netbox to build a dynamic inventory.
- SSH key management on servers (pipeline)
- software updates (
apt dist-upgrade) via a scheduled pipeline. Updates run once a week.
db-backups
Supported systems:
- postgresql
- mysql (mariadb)
- mongodb
- hashicorp vault
The mechanism is used in a Kubernetes cronjob definition.
Infrastructure as Code
angrybits-homelab
Allows:
- creating VMs and docker containers
- configuring DNS, firewall, and other services that have a Terraform provider.
The IaC code in this repository relies on three technologies: Terragrunt, Terraform, and Ansible.
Deployments are run through a pipeline.
More details can be found in the virtual machine creation documentation.
Monitoring & Auto-remediation
proxmox-metrics
API code that collects statistics about completed backups from the Proxmox cluster and exposes them as metrics for Prometheus.
grafana-matrix-api
API that receives alerts from Grafana's alertmanager. It processes the information and sends it to a dedicated channel on the chat server (Matrix).
grafana-alerts-remediate
API that receives alerts from Grafana. It handles NoData alerts and performs a remediation action that determines the current IP address of the node-exporter in Kubernetes and prepares an MR with the change in the angrybits-homelab repository.
Documentation
documentation
The repository containing the code that generates the documentation site.