Building Kubernetes High Availability Clusters: Easy Guide 101

DevOps teams must deal with problems like bursting workloads, traffic spikes, architecting high availability across multiple regions/multiple availability zones, and much more to deploy cloud-native applications in production environments.

Kubernetes clusters provide a higher level of abstraction for deploying and managing a set of containers that make up a cloud-native application’s microservices. In this article, you will discuss Kubernetes High Availability and how to create a highly available Kubernetes cluster.

Prerequisites

• Knowledge of Containerization

What is Kubernetes?

Kubernetes is an open-source platform designed to manage containerized applications, providing automated deployment, scaling, and management. It consists of worker nodes that run containerized applications within Pods, which are the smallest deployable units in Kubernetes. The control plane, responsible for managing the cluster’s worker nodes and Pods, typically spans multiple machines in production to ensure fault tolerance and high availability. By distributing workloads across nodes and automating tasks like scaling and self-healing, Kubernetes helps maintain reliable and efficient application performance even during failures.

What is Kubernetes High Availability & Why is it Needed?

Running multiple apps in a single container can lead to failures, so Kubernetes uses multiple container clones for high availability. In a Kubernetes High Availability system, critical components like the API server and controller manager are replicated across multiple master nodes (usually two or more). If one master fails, the others keep the cluster running. This multi-master setup eliminates a single point of failure, ensuring that all components, such as the API server, etcd, and Kube-scheduler, remain operational. Adding extra master nodes boosts performance and ensures the cluster’s reliability.

Understanding Scalability

• Scalability is the ability of a system to adjust its performance and cost based on changes in demand. It allows a system to grow or shrink as needed without impacting functionality.
• Kubernetes scalability is influenced by factors like the number of nodes, node pools, and their types.
• Scalability measures how well a system, such as hardware or software, handles increasing demand, like more users or queries.
• Businesses should prioritize scalability when selecting hardware and software, especially as they grow, to avoid performance issues down the road.
• Often, cost is prioritized over scalability, but this can lead to problems in large-scale projects, especially in Big Data, where scaling issues can derail progress.
• Focusing on scalability from the start helps reduce maintenance costs, improve user experience, and boost agility in the long term.

How to Set Up Kubernetes High Availability? 

Choosing a regional or zonal control plane

• Regional clusters are better suited for high availability because they spread control planes across multiple compute zones, unlike zonal clusters which only have one control plane.
• In zonal clusters, upgrading the control plane VM causes downtime, making the Kubernetes API unavailable until the upgrade is complete.
• Regional clusters maintain control plane availability during maintenance, such as rotating IPs or resizing clusters, as at least two of the three control plane VMs stay operational during updates.
• A single-zone outage in a regional cluster won’t cause downtime for the control plane, ensuring continuous availability.

Choosing multi-zonal or single-zone node pools

The Kubernetes control plane and its nodes must be distributed over many zones to provide high availability. Single-zone and multi-zonal node pools are available in GKE. Distribute your workload over many compute zones in a region using multi-zonal node pools, distributing nodes equally across zones to create a highly available application.

How to Create Kubernetes High Availability Scalable Clusters?

In this section you will cover the following points:

• What are Container Images?
• How to Use the Command-Line Interface?
• Method 1: Create Kubernetes High Availability Cluster With Stacked Control Plane Nodes
• Method 2: Create Kubernetes High Availability Cluster With External etcd Nodes 

What are Container Images?

Each server should be able to read and download images from the k8s.gcr.io Kubernetes container image registry. This is viable if you wish to construct a highly available cluster where the hosts do not have access to pull images. You must verify that the necessary container images are already accessible on the required hosts through some other method.

How to Use the Command-Line Interface?

Install kubectl on your PC to administer Kubernetes after your cluster is up and running. The Since kubectl lets you control, maintain, analyze, and troubleshoot Kubernetes clusters. Installing the tool on each control plane node is a good idea because it can be of great help. 

We will cover two distinct ways to use kubeadm to build up a highly available Kubernetes cluster:

1. With Stacked Control Plane Nodes: This method needs minimal infrastructure. etcd members and control plane nodes are both in the exact location.
2. With the help of an External etcd Cluster: This strategy needs greater infrastructure. The nodes of the control plane and the members of etcd are separated.

The first step for both methods is to create a load balancer for kube-apiserver.

1. Make a kube-apiserver load balancer with a DNS-resolvable name.

• In a cloud environment, your control plane nodes should be placed behind a TCP forwarding load balancer. This load balancer sends traffic to all healthy control plane nodes in its target list. An apiserver’s health check is a TCP check on the port that the kube-apiserver listens on (default value:6443).
• In a cloud context, using an IP address directly is not advised.
• The API server port and the load balancer must be able to connect with all control plane nodes. On its listening port, it must also enable incoming traffic.
• Make sure that the load balancer’s IP is always the same as kubeadm’s ControlPlaneEndpoint.

2. Test the connection by adding the first control plane node to the load balancer:

	nc -v <LOAD_BALANCER_IP> <PORT>

A connection rejected error is expected because the API server is not yet up and operating. On the other hand, a timeout indicates that the load balancer cannot interact with the control plane node. Reconfigure the load balancer to interact with the control plane node if a timeout occurs.

      3. To the load balancer target group, add the remaining control plane nodes.

Next, we will talk about the way kubeadm can help build up a Kubernetes High Availability cluster.

Method 1: Create Kubernetes High Availability Cluster With Stacked Control Plane Nodes

Let us discuss the steps for the first control plane node.

1. Initialize the control plane:

sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs

2. Apply the CNI plugin 
3. Type the following to get the control plane components’ pods to start:

kubectl get pod -n kube-system -w

Steps for the remaining Control Plane Nodes

You need to do the following for each extra control plane node:

1. On the first node, run the join command that was previously supplied to you by the kubeadm init output. This is what it should look like:

sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07

You can connect numerous control-plane nodes parallelly.

Method 2: Create Kubernetes High Availability Cluster With External etcd Nodes 

Setting up a cluster with external etcd nodes is identical to setting up a stacked etcd cluster, with the distinction that you should first set up etcd and then send the etcd information to the kubeadm config file.

1. Setting up the etcd cluster

• To set up the etcd cluster, follow these steps.

• Set up SSH according to the instructions given.

• Copy the following files to the first control plane node from an etcd node in the cluster:

export CONTROL_PLANE="ubuntu@10.0.0.7"
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":

Replace the value of CONTROL PLANE with the first control-plane node’s user@host.

Set up the First Control Plane Node

1. Make a kubeadm-config. YAML file with the following contents:

apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" # change this (see below)
etcd:
  external:
    endpoints:
      - https://ETCD_0_IP:2379 # change ETCD_0_IP appropriately
      - https://ETCD_1_IP:2379 # change ETCD_1_IP appropriately
      - https://ETCD_2_IP:2379 # change ETCD_2_IP appropriately
    caFile: /etc/kubernetes/pki/etcd/ca.crt
    certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
    keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key

2. Replace the variables in the config template with the relevant values for your cluster:

• LOAD_BALANCER_DNS
• LOAD_BALANCER_PORT
• ETCD_0_IP
• ETCD_1_IP
• ETCD_2_IP

The steps are identical to those in the stacked etcd setup:

• On this node, Run sudo kubeadm init —config kubeadm-config.yaml —upload-certs
• Write the output join instructions to a text file so they can be used later.
• Select the CNI plugin you want to use.

Advantages of High Availability in Kubernetes

Kubernetes High Availability is not just about Kubernetes stability. It’s about configuring Kubernetes and supporting components like etcd, so that there’s no single point of failure. A single point of failure is a component in a system that, if it fails, makes the whole system stop working. Such events can lead to huge operational disruption as well as revenue losses for any company.

Application Use Cases of Kubernetes High Availability

• If you run an e-commerce website with global traffic, Kubernetes can help you scale, manage, and deploy your applications efficiently while ensuring high availability.
• With Kubernetes High Availability, you can perform critical operations like backups or hardware maintenance without affecting your business or causing downtime.
• Adidas, a major shoe manufacturer, uses Kubernetes on AWS to support their e-commerce operations, achieving a significant boost in performance.
• Thanks to Kubernetes and Prometheus, Adidas reduced website load time by half and sped up release cycles from once every 4-6 weeks to 3-4 times per day.
• They use Kubernetes to run about half of their core systems, managing 4,000 pods and 200 nodes, ensuring fast and reliable operations.

Learn More About:

• Prometheus High Availability and Fault Tolerant Strategies for 2024
• Dynamodb to Databricks

Conclusion

It is clear that Kubernetes High Availability is an essential element of reliability engineering, which focuses on making systems dependable and preventing single points of failure across the system. Although its deployment may appear complicated at first, Kubernetes High Availability provides significant benefits to systems that demand enhanced stability and dependability.

Hevo Data is a No-Code Data Pipeline that offers a faster way to move data from 150+ Data Sources including 60+ Free Sources, into your Data Warehouse to be visualized in a BI tool. Hevo is fully automated and hence does not require you to code.

Want to take Hevo for a spin?

SIGN UP for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.

Share your experience with Kubernetes High Availability in the comments section below!

FAQs