Install & Configure Cilium CNI: A Complete Guide

Introduction

In the dynamic landscape of cloud-native infrastructure, choosing the right Container Network Interface (CNI) is paramount for the performance, security, and observability of your Kubernetes clusters. While Kubernetes provides a robust orchestration layer, it delegates network implementation to CNIs, which dictate how pods communicate with each other, with external services, and with the outside world. Traditional CNIs often rely on `iptables` for network policy enforcement and routing, which can introduce performance bottlenecks and complexity, especially in large-scale environments.

Enter Cilium, a revolutionary CNI powered by eBPF (extended Berkeley Packet Filter). Cilium leverages eBPF programs loaded directly into the Linux kernel, enabling highly efficient and programmable network, security, and observability capabilities. This approach bypasses the limitations of `iptables`, offering superior performance, granular visibility into network traffic, and advanced security features like identity-based policies. If you’re looking to elevate your Kubernetes networking game beyond the basics, Cilium is an indispensable tool, providing a future-proof foundation for your containerized applications.

Prerequisites

Before diving into Cilium installation, ensure you have the following:

• A Kubernetes Cluster: This guide assumes you have a functional Kubernetes cluster (e.g., Kubeadm, Kind, EKS, GKE, AKS). Cilium supports various Kubernetes distributions. For local testing, Kind is an excellent choice.
• `kubectl` Configured: Your `kubectl` command-line tool must be configured to communicate with your cluster.
• `helm` (Optional but Recommended): For advanced configurations and upgrades, Helm is the preferred package manager for Kubernetes. Install it from the official Helm documentation.
• Basic Understanding of Kubernetes Networking: Familiarity with concepts like Pods, Services, Deployments, and Network Policies will be beneficial.
• Linux Kernel Version: Cilium requires a Linux kernel version 4.9.17 or higher for basic functionality, but 5.10 or newer is recommended for full eBPF feature set. You can check your kernel version with `uname -r`.
• Root Privileges or `sudo`: For installing the Cilium CLI and managing system-wide tools.

Step-by-Step Guide: Cilium Installation and Configuration

1. Install the Cilium CLI

The Cilium CLI is an essential tool for interacting with your Cilium installation. It provides commands for installation, status checks, troubleshooting, and managing Hubble (Cilium’s observability layer). Installing it first streamlines the entire process.

# Download the latest Cilium CLI for Linux AMD64
curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}

# Verify the download with the SHA256 checksum
sha256sum --check cilium-linux-amd64.tar.gz.sha256sum

# Extract the executable to /usr/local/bin
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin

# Clean up downloaded files
rm cilium-linux-amd64.tar.gz{,.sha256sum}

Verify

Check if the Cilium CLI is installed correctly and accessible in your PATH.

cilium version

Expected Output:

cilium: v1.14.5 compiled with go1.21.3 on linux/amd64

(Version number may vary based on the latest release)

2. Install Cilium in Your Kubernetes Cluster

With the CLI in place, installing Cilium is remarkably straightforward. The `cilium install` command will automatically detect your cluster environment and deploy the necessary components. This includes the Cilium agent (DaemonSet) on each node, the Cilium operator (Deployment), and relevant ClusterRoles and ServiceAccounts. By default, Cilium will attempt to replace any existing CNI.

# Install Cilium with default settings
cilium install

Verify

After installation, check the status of your Cilium deployment to ensure all components are running and healthy. The `–wait` flag is useful as it will block until Cilium reports a healthy status.

cilium status --wait

Expected Output:

    /¯¯\
 /¯¯\ |    Cilium:         OK
 \__/ |    Operator:       OK
 /¯¯\ |    Hubble:         disabled
 \__/ |    ClusterMesh:    disabled
    \ /    dashboards:     disabled
     ¯

Deployment             cilium-operator    Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet              cilium             Desired: 3, Ready: 3/3, Available: 3/3
Containers:            cilium             Running: 3
                       cilium-operator    Running: 1
Cluster Pods:          10/10 managed by Cilium
Image versions         cilium             v1.14.5
                       cilium-operator    v1.14.5

(Number of ready pods will depend on your cluster size. Hubble will be disabled by default.)

3. Enable Hubble for Observability

Hubble is Cilium’s built-in observability platform, providing unparalleled visibility into network flow, service dependencies, and security policy enforcement. It collects network flow information directly from eBPF, offering rich data that goes beyond typical `netflow` or `IPFIX` solutions. Enabling Hubble is highly recommended for any production or development environment. For more advanced eBPF observability, you can also explore building custom metrics, as discussed in our guide on eBPF Observability with Hubble.

# Enable Hubble and deploy the Hubble UI
cilium hubble enable --ui

Verify

Once Hubble is enabled, you can port-forward the Hubble UI service to your local machine and access it via a web browser. This provides a graphical interface to explore network flows and policies.

# Port-forward the Hubble UI service
cilium hubble ui

Expected Output:

Forwarding from 127.0.0.1:12000 -> 12000
Forwarding from [::1]:12000 -> 12000
Hubble UI available at http://localhost:12000

Open your web browser and navigate to `http://localhost:12000` to see the Hubble UI. You should see a graphical representation of your cluster’s network activity.

4. Implement Cilium Network Policies

Cilium’s network policies are a powerful feature, leveraging identity-based security rather than IP addresses. This means policies can refer to Kubernetes labels directly, making them more resilient to IP changes and easier to manage. CiliumNetworkPolicies (CNP) are custom resources (CRDs) that extend Kubernetes Network Policies. For a deeper dive into securing your cluster, refer to our Kubernetes Network Policies: Complete Security Hardening Guide.

Let’s create a simple `CiliumNetworkPolicy` that allows outbound DNS traffic from pods labeled `app: my-app` to `kube-dns`.

# policy.yaml
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-kube-dns
spec:
  endpointSelector:
    matchLabels:
      app: my-app
  egress:
  - toPorts:
    - port: "53"
      protocol: UDP
    toEndpoints:
    - matchLabels:
        k8s:kube-dns # Label for kube-dns service

Apply this policy to your cluster:

kubectl apply -f policy.yaml

Verify

To verify the policy, you would typically deploy a pod with the `app: my-app` label and try to establish network connections. For this example, we’ll demonstrate using `cilium endpoint get` to see the policies applied to an endpoint. First, deploy a sample application.

# app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["sh", "-c", "while true; do sleep 3600; done"]

kubectl apply -f app.yaml
kubectl get pods -l app=my-app

Expected Output (after pod is running):

NAME                      READY   STATUS    RESTARTS   AGE
my-app-7b8f9c79f4-abcde   1/1     Running   0          1m

Now, get the Cilium endpoint details for `my-app` pod. Replace `my-app-7b8f9c79f4-abcde` with your actual pod name.

POD_NAME=$(kubectl get pod -l app=my-app -o jsonpath='{.items[0].metadata.name}')
cilium endpoint get $POD_NAME -o json | grep 'CiliumNetworkPolicy' -A 5

Expected Output (snippet):

        "policy": {
            "ingress": {},
            "egress": {
                "CiliumNetworkPolicy": [
                    "default/allow-kube-dns"
                ]
            }
        },

This output confirms that the `allow-kube-dns` policy is applied to the pod. You could further test by exec’ing into the pod and trying to resolve a hostname (e.g., `nslookup kubernetes.default.svc.cluster.local`) to ensure DNS works, and then attempting to reach an external IP on a different port to see it blocked.

Production Considerations

Deploying Cilium in production requires careful planning and configuration to ensure optimal performance, reliability, and security.

• Datapath Mode: Cilium supports various datapath modes (e.g., `tunnel`, `direct-routing`). `direct-routing` generally offers better performance but requires specific network configurations (e.g., BGP advertising of pod CIDRs). Choose the mode that best fits your infrastructure. Refer to the Cilium routing documentation for details.
• IP Address Management (IPAM): Cilium can manage IP addresses for pods using various IPAM modes (e.g., `cluster-pool`, `kubernetes`). For large clusters, `cluster-pool` with native routing or BGP is often preferred for efficiency. For cloud providers, integration with cloud provider IPAM (e.g., AWS ENI) is common.
• Resource Requirements: Cilium agents can consume CPU and memory, especially with a large number of policies or high traffic. Monitor resource usage and adjust requests/limits as needed. Consider node sizing and network capacity.
• Observability & Monitoring: Enable Hubble and integrate its metrics with your existing monitoring stack (Prometheus, Grafana). This is crucial for understanding network behavior, troubleshooting, and identifying security incidents. Our article on eBPF Observability: Building Custom Metrics with Hubble provides more insights.
• High Availability: Ensure your Cilium operator is deployed with multiple replicas for high availability. The Cilium agent runs as a DaemonSet, inherently providing high availability per node.
• Security Policies: Design and test your CiliumNetworkPolicies thoroughly. Start with a permissive policy and gradually lock it down. Consider using `CiliumClusterwideNetworkPolicy` for policies that apply across all namespaces.
• Kernel Compatibility: Ensure your nodes run a sufficiently modern Linux kernel (5.10+ recommended) to leverage all eBPF features and performance optimizations.
• Upgrade Strategy: Plan your Cilium upgrade strategy. Use Helm for easy upgrades and rollbacks. Always test upgrades in a staging environment first.
• Integration with Service Mesh: Cilium integrates well with service meshes like Istio. For example, Cilium can handle network policies and data plane acceleration while Istio manages traffic management and mTLS. Learn more about service mesh capabilities in our Istio Ambient Mesh Production Guide.
• WireGuard Encryption: For encrypting pod-to-pod traffic, especially across untrusted networks or public clouds, consider enabling Cilium’s WireGuard encryption.

Troubleshooting

Here are some common issues you might encounter with Cilium and their solutions.

1. Cilium Pods Not Ready / CrashLoopBackOff

   Issue: Cilium agent pods (`cilium-*`) are not coming up, stuck in `Pending` or `CrashLoopBackOff`.

   Solution:

   • Check Logs: The first step is always to check the logs of the failing pod:

     kubectl logs -n kube-system -l k8s-app=cilium

   • Kernel Version: Ensure your kernel meets the minimum requirements (4.9.17+).

     uname -r

   • `kube-proxy` Disabled: If you installed Cilium with `kubeProxyReplacement=true` (which is common), ensure `kube-proxy` is truly disabled or not running. Conflicts can arise if both are active.
   • Resource Constraints: Check if the nodes have sufficient CPU and memory.
   • CRI Socket: Ensure the `containerd` or Docker socket path is correctly configured if you are using a custom setup. Cilium needs access to the container runtime.
2. Pods Cannot Communicate / Network Issues

   Issue: Pods cannot reach other pods, services, or external endpoints after Cilium installation.

   Solution:

   • Cilium Status: Verify Cilium is healthy on all nodes:

     cilium status

   • Network Policies: Check if any `CiliumNetworkPolicy` or `NetworkPolicy` is inadvertently blocking traffic. Use `cilium policy get` or `hubble observe`.

     cilium policy get

   • Datapath Mode: Ensure your datapath mode (e.g., `tunnel`, `direct-routing`) is correctly configured for your environment. Direct routing requires underlying network support.
   • Firewall Rules: Are there any host-level firewall rules (e.g., `firewalld`, `ufw`) interfering with Cilium’s eBPF programs or traffic?
   • IPAM Configuration: Verify that IP address management is working correctly and pods are getting IPs from the expected range.
3. Hubble UI / CLI Not Showing Flows

   Issue: Hubble UI is empty or `cilium hubble observe` shows no traffic.

   Solution:

   • Hubble Enabled: Ensure Hubble is enabled in your Cilium configuration.

     cilium status | grep Hubble

     If disabled, run `cilium hubble enable`.

   • Hubble Relay & UI Pods: Check if the `hubble-relay` and `hubble-ui` pods are running correctly in the `kube-system` namespace.

     kubectl get pods -n kube-system -l k8s-app=hubble

   • Port Forwarding: Confirm that port forwarding for the Hubble UI is active and not blocked by local firewalls.
   • Traffic Generation: Make sure there’s actual network traffic in your cluster for Hubble to observe.
4. External Service Access Issues

   Issue: Pods cannot reach external services (e.g., `google.com`).

   Solution:

   • Egress Policies: Check if any `CiliumNetworkPolicy` is blocking egress traffic to external IPs. Cilium policies are “deny by default” for egress if an egress rule exists.
   • Masquerading: Ensure Cilium’s masquerading is correctly configured. For `tunnel` mode, this is usually handled automatically. For `direct-routing`, you might need to ensure the worker nodes have proper NAT rules or that the pod CIDRs are routable. Check `cilium config view | grep enable-ipv4-masquerade`.
   • DNS Resolution: Can pods resolve external hostnames? Test with `nslookup google.com` from inside a pod. If not, check `kube-dns` or `coredns` and related network policies.
5. Cilium Upgrade Problems

   Issue: Encountering errors or instability during a Cilium upgrade.

   Solution:

   • Read Release Notes: Always review the Cilium release notes for breaking changes or specific upgrade instructions.
   • Use Helm: If you initially installed with `cilium install`, consider migrating to Helm for future upgrades. The Cilium CLI can generate Helm values.
   • Backup: Before major upgrades, back up your cluster state.
   • Incremental Upgrades: Avoid skipping too many minor versions. Upgrade one minor version at a time if possible.
6. High CPU/Memory Usage by Cilium Agents

   Issue: Cilium agent pods consuming excessive resources.

   Solution:

   • Number of Policies: A very large number of complex network policies can increase resource usage. Optimize your policies.
   • Traffic Volume: High network traffic volume, especially with deep packet inspection features enabled, can increase CPU.
   • eBPF Map Sizes: For very large clusters or many endpoints, eBPF map sizes might need tuning. Refer to Cilium documentation on performance tuning.
   • Debug Logs: Temporarily increase logging verbosity for the Cilium agent to gather more insights, but remember to revert it in production.

     kubectl patch daemonset cilium -n kube-system --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--debug"}]'

FAQ Section

1. What is eBPF and why does Cilium use it?

   eBPF (extended Berkeley Packet Filter) is a revolutionary technology that allows programs to run in the Linux kernel without changing kernel source code or loading kernel modules. Cilium leverages eBPF for high-performance networking, security, and observability. It provides a highly efficient, programmable data plane that can apply policies, route traffic, and collect telemetry directly in the kernel, bypassing traditional `iptables` rules and improving performance and scalability. For an in-depth understanding, refer to the eBPF.io website.

2. Can Cilium replace `kube-proxy`?

   Yes, Cilium can fully replace `kube-proxy` using its `kubeProxyReplacement` feature. When enabled, Cilium handles all service load balancing in eBPF, offering significant performance improvements and simplifying the network stack. This is a highly recommended configuration for production environments. You can enable it during installation with `cilium install –set kubeProxyReplacement=strict` or via Helm values.

3. How does Cilium handle network policies compared to standard Kubernetes Network Policies?

   Cilium supports the standard Kubernetes `NetworkPolicy` API but extends it significantly with its own `CiliumNetworkPolicy` (CNP) Custom Resource Definitions (CRDs). CNPs offer more granular control, identity-based policies (using Kubernetes labels instead of IP addresses), advanced L7 policy enforcement (e.g., HTTP, Kafka), and integration with external services. While standard policies are IP-based and typically use `iptables`, Cilium processes all policies efficiently using eBPF.

4. What are the benefits of using Hubble?

   Hubble provides deep observability into your Kubernetes network. Key benefits include:

   • Network Flow Visibility: See exactly which pods are communicating, to where, and what protocols they are using.
   • Policy Enforcement Monitoring: Understand which policies are allowing or denying traffic.
   • Service Dependency Mapping: Visualize how your services interact.
   • Troubleshooting: Quickly diagnose connectivity issues and policy misconfigurations.
   • Security Auditing: Get a clear audit trail of network activity for compliance.

   It’s an invaluable tool for both operations and security teams. You can learn more about its advanced capabilities in our eBPF Observability with Hubble guide.

5. Is Cilium suitable for multi-cluster environments?

   Yes, Cilium is designed to support multi-cluster environments through its Cluster Mesh feature. Cluster Mesh allows seamless connectivity and policy enforcement across multiple Kubernetes clusters, even if they are in different cloud providers or on-premises. It enables global network policies and shared service discovery, making it a powerful solution for complex distributed architectures. Check the Cilium Cluster Mesh documentation for setup instructions.

Cleanup Commands

To completely remove Cilium and revert your cluster to its pre-Cilium state (or prepare for a fresh install), use the `cilium uninstall` command. This will remove all Cilium components, CRDs, and network configurations.

# Uninstall Cilium from the cluster
cilium uninstall

Expected Output:

❓ Do you want to uninstall Cilium? [y/N] y
🔑 Deleting Cilium release 'cilium'
🗑️ Deleting Cilium CRDs
[...]
🔥 Restarting Kube-proxy (if it was disabled by Cilium)
✅ Cilium has been uninstalled.

If `kube-proxy` was disabled by Cilium, this command will attempt to restart it. You may need to manually restart your CNI (if any was present before Cilium) or reset your cluster’s network configuration if you encounter issues after uninstalling. For Kind clusters, you might need to restart the cluster or recreate it.

Next Steps / Further Reading

Congratulations! You’ve successfully installed and configured Cilium, enabled Hubble, and implemented a basic network policy. This is just the beginning of what Cilium can do. To further enhance your Kubernetes networking and security posture, consider exploring:

• Advanced Network Policies: Dive deeper into Cilium’s policy language, including L7 policies for HTTP, Kafka, DNS, and more.
• WireGuard Encryption: Secure pod-to-pod traffic with built-in WireGuard encryption.
• External Workload Integration: Connect non-Kubernetes workloads (VMs, bare metal) into your Cilium network using Host Firewall or External Workload support.
• Load Balancing with Cilium: Explore Cilium’s advanced load balancing features, including DSR (Direct Server Return) and XDP for high-performance services.
• Cluster Mesh: Expand your Cilium deployment across multiple Kubernetes clusters for unified networking and security.
• Integration with Kubernetes Gateway API: For advanced ingress and traffic management, see how Cilium integrates with the Kubernetes Gateway API.
• Service Mesh Integration: If you’re using a service mesh like Istio, understand how Cilium can complement it for a robust data plane, as discussed in our Istio Ambient Mesh Production Guide.
• Cost Optimization: While not directly related to Cilium, optimizing your cluster’s underlying infrastructure with tools like Karpenter can further enhance the efficiency of your Cilium-powered network.

Conclusion

Cilium stands out as a powerful, high-performance CNI solution for Kubernetes, fundamentally transforming how network, security, and observability are managed in cloud-native environments. By harnessing the power of eBPF, Cilium offers unparalleled visibility, granular policy enforcement, and superior performance compared to traditional `iptables`-based approaches. This guide has walked you through the essential steps of installing Cilium, enabling its critical observability component Hubble, and implementing your first network policy.

Embracing Cilium means investing in a future-proof networking stack that provides the agility and control necessary for modern microservices architectures. As your Kubernetes deployments scale and demand more sophisticated networking and security capabilities, Cilium’s eBPF-driven architecture will prove to be an invaluable asset, ensuring your applications are not only connected but also secure and observable. The journey into advanced Kubernetes networking with Cilium is exciting, and the benefits for your infrastructure are profound.