Introduction

Warning

These docs contain information that relates to my setup. They may or may not work for you.



Lovenet Home Operations Repository

Managed by Flux, Renovate and GitHub Actions 🤖

Renovate

Kubernetes Cluster Information:

Age-Days  Node-Count  Pod-Count  CPU-Usage  Memory-Usage 



Overview

This is the configuration for my GitOps homelab Kubernetes cluster. This cluster runs home software services for my residence. It is quite complex and there are a lot of interdependencies but the declarative nature of GitOps allows me to manage this mesh of code. The software services fall into a few primary categories:

Core Components

Infrastructure

Networking

  • cilium: Kubernetes Container Network Interface (CNI).
  • cert-manager: Creates SSL certificates for services in my Kubernetes cluster.
  • external-dns: Automatically manages DNS records from my cluster in a cloud DNS provider.
  • ingress-nginx: Ingress controller to expose HTTP traffic to pods over DNS.
  • Cloudflared: Cloudflare tunnel client.

Storage

  • Rook-Ceph: Distributed block storage for peristent storage..
  • Minio: S3 Compatible Storage Interface.
  • Longhorn: Cloud native distributed block storage for Kubernetes.
  • NFS: NFS storage.

GitOps


⚙️  Hardware

HostnameDeviceCPURAMOSRoleStorageIOTNetwork
master1Intel NUC7PJYH48 GBCentOS 9k8s Master
master2VM on beast38 GBCentOS 9k8s Master
master3VM on beast38 GBCentOS 9k8s Master
worker1ThinkCentre M910x832 GBCentOS 9k8s Workerlonghorn NVMeCoral USB
worker2ThinkCentre M910x832 GBCentOS 9k8s Workerlonghorn NVMezstick-7
worker3ThinkCentre M910x832 GBCentOS 9k8s Workerlonghorn NVMe, ceph osdsec-vlan
worker4ThinkCentre M910x832 GBCentOS 9k8s Workerlonghorn NVMesec-vlan
worker5VM on beast1024 GBCentOS 9k8s Workerlonghorn NVMe, ceph osd
worker6VM on beast1024 GBCentOS 9k8s Workerlonghorn NVMe, ceph osdskyconnect
worker7VM on beast1024 GBCentOS 9k8s Workerlonghorn NVMe, ceph osdiot-vlan
worker8VM on beast1024 GBCentOS 9k8s Workerlonghorn NVMe, ceph osdiot-vlan

Network

Click to see a high level physical network diagram dns
NameCIDRVLANNotes
Management VLANTBD
Default192.168.0.0/160
IOT VLAN10.10.20.1/2420
Guest VLAN10.10.30.1/2430
Security VLAN10.10.40.1/2440
Kubernetes Pod Subnet (Cilium)10.42.0.0/16N/A
Kubernetes Services Subnet (Cilium)10.43.0.0/16N/A
Kubernetes LB Range (CiliumLoadBalancerIPPool)10.45.0.1/24N/A

☁️ Cloud Dependencies

ServiceUseCost
1PasswordSecrets with External Secrets~$65/yr
CloudflareDomainFree
GitHubHosting this repository and continuous integration/deploymentsFree
MailgunEmail hostingFree (Flex Plan)
PushoverKubernetes Alerts and application notifications$10 (One Time)
Total: ~$5.50/mo

Initialization

./init/create-cluster.sh (on master)

./init/prepare-cluster.sh (on laptop)

./init/initialize-cluster.sh (on laptop)

ssh root@master1 rm /etc/kubernetes/manifests/kube-vip.yaml (on laptop)

Teardown

./init/destroy-cluster.sh (on laptop)

Debugging

  • https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/
  • https://dnschecker.org
  • https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
  • https://github.com/nicolaka/netshoot
  • https://www.redhat.com/sysadmin/using-nfsstat-nfsiostat

Github Webhook

kubectl -n flux-system get receivers.notification.toolkit.fluxcd.io generates token URL to be put into github.com -> Settings -> Webhooks -> Payload URL

  • Content Type: application/json
  • Secret: <token from kubectl -n flux-system describe secrets github-webhook-token>
  • SSL: Enable SSL verification
  • Which events would you like to trigger this webhook?: Just the push event.
  • Active:

Notes

To get metrics-server to work with kubeadm, you need to do the following if it isn't setup with the clusterconfig provided to kubeadm https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#kubelet-serving-certs

  • Need to pull the kubeadm configuration into this repository

Resources: Limits and Requests Philosophy

In short, do set CPU requests, but don't set CPU limits and set the Memory limit to be the same as the Memory requests.

@whazor created this website as a creative way to search Helm Releases across GitHub. You may use it as a means to get ideas on how to configure an applications' Helm values.