Layman Stories

Production-Grade Kubernetes Deployment with Kubespray, HAProxy, Keepalived, and Cilium CNI (Part 1)

Deploying an enterprise-grade, highly available (HA) Kubernetes cluster requires robust load balancing across control plane nodes, automated infrastructure provisioning, and an eBPF-powered Container Network Interface (CNI). In this multi-part series, we walk through building a production-ready Kubernetes cluster from scratch using Kubespray, Ubuntu 24.04 LTS, HAProxy + Keepalived for API server load balancing, and Cilium CNI.

This article is Part 1 of our deployment series, focusing on architecture topology, load balancer setup, Kubespray inventory generation using bash arrays, Cilium CNI configuration, and cluster initialization.

Cluster Architecture & Node Topology

Our cluster layout consists of 8 dedicated virtual or physical nodes running Ubuntu 24.04 LTS:

  • Load Balancers / VIP (2 Nodes): lb-01 (192.168.10.11), lb-02 (192.168.10.12) sharing Virtual IP (VIP): 192.168.10.10
  • Kubernetes Control Plane / Masters (3 Nodes): k8s-master-1 (192.168.10.21), k8s-master-2 (192.168.10.22), k8s-master-3 (192.168.10.23)
  • Kubernetes Worker Nodes (3 Nodes): k8s-worker-1 (192.168.10.31), k8s-worker-2 (192.168.10.32), k8s-worker-3 (192.168.10.33)
  • Ansible Control Node: ansible-deploy (192.168.10.5)

Step 1: System Preparation on All Nodes (Ubuntu 24.04 LTS)

Execute prerequisite configuration across all 8 nodes:

# Update OS packages and disable swap
sudo apt update && sudo apt upgrade -y
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

# Load required kernel modules for Kubernetes & Containerd
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Configure sysctl networking parameters
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

Step 2: Deploy HAProxy + Keepalived Load Balancers

To prevent single points of failure for the Kubernetes API Server, configure Keepalived (VRRP VIP) and HAProxy on lb-01 and lb-02.

1. Install HAProxy and Keepalived:

sudo apt install -y haproxy keepalived

2. Configure Keepalived on lb-01 (Master LB):

cat <<EOF | sudo tee /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
    script "killall -0 haproxy"
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass K8sSecretPass
    }
    virtual_ipaddress {
        192.168.10.10/24
    }
    track_script {
        check_haproxy
    }
}
EOF

3. Configure Keepalived on lb-02 (Backup LB):

Set state BACKUP and priority 100 on lb-02.

4. Configure HAProxy on Both LB Nodes:

cat <<EOF | sudo tee /etc/haproxy/haproxy.cfg
frontend k8s-api
    bind 192.168.10.10:6443
    mode tcp
    option tcplog
    default_backend k8s-masters

backend k8s-masters
    mode tcp
    option tcp-check
    balance roundrobin
    server k8s-master-1 192.168.10.21:6443 check fall 3 rise 2
    server k8s-master-2 192.168.10.22:6443 check fall 3 rise 2
    server k8s-master-3 192.168.10.23:6443 check fall 3 rise 2
EOF

sudo systemctl restart keepalived haproxy
sudo systemctl enable keepalived haproxy

Step 3: Kubespray Setup & Inventory Generation via Bash Array

On your Ansible control node (ansible-deploy), set up Kubespray and generate your inventory file using the bash array method:

# Clone Kubespray repository
git clone https://github.com/kubernetes-sigs/kubespray.git
cd kubespray

# Create Python Virtual Environment
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Copy sample inventory directory
cp -rfp inventory/sample inventory/mycluster

# Declare node IPs array and generate hosts.yaml inventory
declare -a IPS=(192.168.10.21 192.168.10.22 192.168.10.23 192.168.10.31 192.168.10.32 192.168.10.33)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Update inventory/mycluster/hosts.yaml to assign 3 control plane nodes and 3 worker nodes:

all:
  hosts:
    node1:
      ansible_host: 192.168.10.21
      ip: 192.168.10.21
      access_ip: 192.168.10.21
    node2:
      ansible_host: 192.168.10.22
      ip: 192.168.10.22
      access_ip: 192.168.10.22
    node3:
      ansible_host: 192.168.10.23
      ip: 192.168.10.23
      access_ip: 192.168.10.23
    node4:
      ansible_host: 192.168.10.31
      ip: 192.168.10.31
      access_ip: 192.168.10.31
    node5:
      ansible_host: 192.168.10.32
      ip: 192.168.10.32
      access_ip: 192.168.10.32
    node6:
      ansible_host: 192.168.10.33
      ip: 192.168.10.33
      access_ip: 192.168.10.33
  children:
    kube_control_plane:
      hosts:
        node1:
        node2:
        node3:
    kube_node:
      hosts:
        node4:
        node5:
        node6:
    etcd:
      hosts:
        node1:
        node2:
        node3:
    k8s_cluster:
      children:
        kube_control_plane:
        kube_node:

Step 4: Configure External Load Balancer & Cilium CNI

Edit Kubespray cluster variables to route API traffic through our HAProxy VIP and enable Cilium CNI.

1. External Load Balancer Settings (inventory/mycluster/group_vars/all/all.yml):

apiserver_loadbalancer_domain_name: "192.168.10.10"
loadbalancer_apiserver:
  address: 192.168.10.10
  port: 6443

2. Cilium CNI Settings (inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml):

kube_network_plugin: cilium
cilium_enable_netpol: true
cilium_tunnel_mode: "vxlan"
cilium_enable_prometheus: true

Step 5: Run Kubespray Playbook & Verify Cluster

Execute the Kubespray Ansible playbook to build the cluster:

ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml

After deployment completes, verify cluster health from node1 (master):

sudo cp /etc/kubernetes/admin.conf ~/.kube/config
kubectl get nodes -o wide
kubectl get pods -n kube-system
cilium status

Conclusion & What’s Next in Part 2

You now have a production-ready, highly available Kubernetes cluster running on Ubuntu 24.04 LTS with 3 control plane nodes, 3 worker nodes, HAProxy/Keepalived load balancing, and eBPF-powered Cilium CNI.

In Part 2 of this series, we will focus on advanced operational services:

  • Cilium LB IPAM: Setting up dedicated subnet IP pools for Kubernetes LoadBalancer services
  • Cilium Gateway API & HTTPRoute: Configuring eBPF-native ingress routing and HTTPRoute resources
  • Longhorn Storage: Deploying persistent, cloud-native distributed block storage
  • Cert-Manager & Let’s Encrypt ClusterIssuer: Automating SSL/TLS certificate issuance and lifecycle management

admin