Just-in

Kagent Tutorial: Build Cloud-Native AI Agents on Kubernetes

Running Ollama LLM on Kubernetes: A Complete Guide

From Docker to Kubernetes: The Orchestration Journey You Didn’t Know You Needed

What is Kubeflow and how can it be used?

Getting Started with Kubeflow: A Complete Beginner’s Guide

Orchestration

Deploy Envoy Gateway with Gateway API: Production

Introduction

In the ever-evolving landscape of Kubernetes networking, managing external traffic securely and efficiently remains a critical challenge. Historically, Kubernetes Ingress has served as the de facto standard, but its limitations in expressiveness and role separation have become increasingly apparent for complex, multi-tenant environments. Enter the Kubernetes Gateway API, a powerful, extensible, and role-oriented alternative designed to overcome these shortcomings, offering a more robust and flexible approach to traffic management.

This guide dives deep into deploying the Gateway API in a production environment using Envoy Gateway as our specific implementation. Envoy Gateway provides a performant, scalable, and feature-rich data plane based on Envoy Proxy, making it an excellent choice for handling diverse traffic routing needs, including advanced features like traffic splitting, retries, and circuit breaking. By the end of this tutorial, you’ll have a fully functional Gateway API setup, ready to manage your application’s ingress traffic with unparalleled control and observability.

TL;DR: Production Gateway API with Envoy Gateway

Deploy Envoy Gateway to manage external traffic into your Kubernetes cluster using the Gateway API. This guide covers installation, configuring a basic HTTPRoute, and essential production considerations.

Key Commands:

# Install Envoy Gateway
helm install envoy-gateway --create-namespace --namespace envoy-gateway \
  oci://docker.io/envoyproxy/envoy-gateway-charts/envoy-gateway --version v1.0.0

# Deploy a sample application
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/nginx-svc.yaml

# Apply Gateway and HTTPRoute
cat <

Prerequisites

Before we embark on this journey, ensure you have the following tools and knowledge:

  • Kubernetes Cluster: A running Kubernetes cluster (v1.24+ recommended). You can use Minikube, Kind, or a cloud-managed cluster (EKS, GKE, AKS).
  • kubectl: The Kubernetes command-line tool, configured to connect to your cluster. Refer to the official Kubernetes documentation for installation.
  • helm: The Kubernetes package manager. Install it from the Helm website.
  • Basic Kubernetes Knowledge: Familiarity with Kubernetes concepts like Pods, Deployments, Services, and Namespaces.
  • DNS Management: Access to a DNS provider to create A/CNAME records for your services.
  • curl: For testing HTTP routes.

Step-by-Step Guide

1. Install Envoy Gateway

The first step is to install Envoy Gateway into your Kubernetes cluster. Envoy Gateway leverages Helm for easy deployment and management. We'll install it into its own dedicated namespace, envoy-gateway, to maintain good resource isolation. This installation will provision the necessary Custom Resource Definitions (CRDs), the Envoy Gateway controller, and the Envoy Proxy data plane components.

The Envoy Gateway controller watches for Gateway API resources (Gateway, HTTPRoute, etc.) and translates them into Envoy Proxy configurations. It then manages the lifecycle of the Envoy Proxy instances that actually handle the traffic. This separation of concerns between the control plane (Envoy Gateway controller) and the data plane (Envoy Proxy) is a core tenet of the Gateway API design.

# Add the Envoy Gateway Helm repository
helm repo add envoy-gateway https://envoyproxy.github.io/envoy-gateway-charts
helm repo update

# Install Envoy Gateway
helm install envoy-gateway --create-namespace --namespace envoy-gateway \
  oci://docker.io/envoyproxy/envoy-gateway-charts/envoy-gateway --version v1.0.0

Verify: Check if the Envoy Gateway pods are running and the Gateway API CRDs are installed.

kubectl get pods -n envoy-gateway

Expected Output:

NAME                                READY   STATUS    RESTARTS   AGE
envoy-gateway-f5f8d9b6c-abcde       1/1     Running   0          2m
envoy-gateway-envoy-xyz             1/1     Running   0          2m

kubectl get crds | grep gateway.networking.k8s.io

Expected Output:

gateways.gateway.networking.k8s.io                           2023-10-27T10:00:00Z
gatewayclasses.gateway.networking.k8s.io                     2023-10-27T10:00:00Z
httproutes.gateway.networking.k8s.io                         2023-10-27T10:00:00Z
... (other Gateway API CRDs)

2. Deploy a Sample Application

To demonstrate traffic routing, we need a backend application. For simplicity, we'll deploy a basic NGINX deployment and an associated Service. This NGINX instance will serve as our target for the HTTPRoute we'll create later. This setup mimics a common scenario where you have microservices that need to be exposed externally.

This NGINX deployment and service will run in the default namespace, but the Gateway API allows routing to services in any namespace, provided proper RBAC is configured. For more complex inter-service communication and security, consider exploring solutions like Istio Ambient Mesh or Kubernetes Network Policies.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/nginx-svc.yaml

Verify: Confirm the NGINX deployment and service are up and running.

kubectl get deployments,services -n default -l app=nginx

Expected Output:

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx   1/1     1            1           1m

NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/nginx   ClusterIP   10.96.100.100    <none>        80/TCP    1m

3. Create a GatewayClass

A GatewayClass defines a set of Gateways that share a common configuration and behavior. It acts as a template for Gateways, allowing infrastructure providers (like Envoy Gateway) to offer different types of Gateway implementations. When you installed Envoy Gateway, it automatically created a GatewayClass named envoy-gateway-class. This is what we'll reference in our Gateway resource.

This abstraction is crucial for role separation: infrastructure providers manage GatewayClass, while cluster operators deploy Gateway resources referencing these classes. This allows for clear boundaries and prevents developers from accidentally misconfiguring the underlying infrastructure.

kubectl get gatewayclass

Expected Output:

NAME                  CONTROLLER                          ACCEPTED   AGE
envoy-gateway-class   gateway.envoyproxy.io/envoy-gateway   True       5m

4. Deploy a Gateway

The Gateway resource represents a point of entry for traffic into your cluster. It's akin to an Ingress Controller instance but with more granular control over listeners and protocols. We'll create a Gateway that references our envoy-gateway-class and listens on HTTP port 80. Envoy Gateway will provision an external load balancer (or NodePort/ClusterIP depending on your cluster's environment) to expose this Gateway.

The listener section is where you define the protocols, ports, and hostnames the Gateway should handle. For production, you'd typically configure HTTPS listeners here as well, along with TLS certificates. We're starting with HTTP for simplicity, but the process extends seamlessly to TLS.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: envoy-gateway-class
  listener:
  - name: http
    protocol: HTTP
    port: 80

Apply the Gateway:

kubectl apply -f - <

Verify: Wait for the Gateway to become ready and get its external IP address. This might take a few minutes depending on your cloud provider's load balancer provisioning time.

kubectl get gateway my-gateway

Expected Output (look for an IP under ADDRESS):

NAME         CLASS                 ADDRESS         READY   HTTP    HTTPS   AGE
my-gateway   envoy-gateway-class   34.123.45.678   True    80      443     2m

Note down the ADDRESS for the next steps. This is your external entry point.

5. Create an HTTPRoute

The HTTPRoute resource defines how HTTP traffic arriving at a Gateway is routed to backend services. It allows for advanced matching based on hostnames, paths, headers, and query parameters, and supports various actions like traffic splitting, URL rewriting, and header manipulation. This is where the application developer or service owner typically defines their routing rules.

In this example, we'll create an HTTPRoute that matches requests for nginx.example.com on any path (/ prefix) and forwards them to our nginx service on port 80. This demonstrates a basic host-based and path-based routing configuration. The parentRefs links this route directly to our my-gateway.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-route-to-nginx
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "nginx.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: "/"
    backendRefs:
    - name: nginx
      port: 80

Apply the HTTPRoute:

kubectl apply -f - <

Verify: Check the status of your HTTPRoute to ensure it's accepted and bound to the Gateway.

kubectl get httproute http-route-to-nginx

Expected Output:

NAME                HOSTNAMES           AGE
http-route-to-nginx   ["nginx.example.com"]   30s

kubectl describe httproute http-route-to-nginx

Look for conditions like Accepted: True and Programmed: True under Status, indicating the route has been successfully configured on the Envoy Proxy.

6. Configure DNS and Test

For external access, you need to configure your DNS provider to point a hostname to the external IP address of your Gateway. Create an A record for nginx.example.com that points to the ADDRESS obtained in step 4.

Once DNS propagation completes (which can take a few minutes), you can test the routing using curl.

# Replace <GATEWAY_EXTERNAL_IP> with the actual IP from 'kubectl get gateway my-gateway'
curl -H "Host: nginx.example.com" http://<GATEWAY_EXTERNAL_IP>

Expected Output:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

If you see the NGINX welcome page, congratulations! You've successfully deployed and configured the Gateway API with Envoy Gateway.

Production Considerations

Deploying the Gateway API with Envoy Gateway in production involves more than just basic routing. Here are critical aspects to consider:

  1. TLS/SSL Termination: For any public-facing application, TLS is non-negotiable. Configure Gateway listeners for HTTPS and provide TLS certificates. Envoy Gateway supports various methods for certificate management, including integration with cert-manager for automated certificate provisioning from Let's Encrypt.
  2. Observability: Integrate with your existing monitoring and logging stacks. Envoy Proxy is highly observable, emitting detailed metrics (Prometheus format), access logs, and traces. Ensure these are collected and analyzed. For advanced eBPF-based observability, consider tools like Hubble with Cilium.
  3. High Availability and Scalability:
    • Envoy Gateway Controller: Deploy with multiple replicas for high availability.
    • Envoy Proxy Data Plane: Configure Horizontal Pod Autoscaler (HPA) for the Envoy Proxy deployments managed by Envoy Gateway, scaling based on CPU, memory, or custom metrics.
    • Load Balancer: Ensure your cloud provider's load balancer is configured for high availability across availability zones.
  4. Security:
    • Network Policies: Implement Kubernetes Network Policies to restrict traffic to and from the Envoy Gateway pods and your backend services.
    • RBAC: Configure fine-grained RBAC for Gateway API resources. For instance, restrict who can create GatewayClasses and Gateways versus who can create HTTPRoutes.
    • WAF/DDoS Protection: For mission-critical applications, consider placing an external WAF (Web Application Firewall) or DDoS protection service (e.g., Cloudflare, AWS WAF) in front of your Envoy Gateway.
    • Rate Limiting & Authentication: Envoy Gateway supports global and local rate limiting, as well as external authorization (e.g., integrating with an OIDC provider) via HTTPRouteFilter.
  5. Traffic Management Advanced Features: Leverage HTTPRoute capabilities for:
    • Traffic Splitting/Canary Deployments: Gradually shift traffic between different versions of a service.
    • Retries and Timeouts: Configure resilience patterns for unreliable backends.
    • Circuit Breaking: Prevent cascading failures by stopping requests to unhealthy services.
    • Header Manipulation & URL Rewriting: Modify requests/responses on the fly.
  6. GitOps Workflow: Manage all Gateway API resources (Gateway, HTTPRoute, etc.) using a GitOps approach with tools like Argo CD or Flux CD. This ensures all configurations are version-controlled, auditable, and automatically deployed.
  7. Resource Limits and Requests: Set appropriate CPU and memory limits and requests for Envoy Gateway controller and Envoy Proxy pods to prevent resource starvation and ensure stable performance.
  8. Cost Optimization: While not directly related to Envoy Gateway, ensure your underlying Kubernetes nodes are efficiently utilized. Tools like Karpenter can help optimize node provisioning and reduce costs.

Troubleshooting

  1. Issue: Gateway ADDRESS remains <pending>.

    Explanation: This usually means the cloud provider's load balancer controller is unable to provision an external IP. This can happen due to insufficient permissions, quota limits, or a misconfigured cloud provider integration.

    Solution:

    • Check the events of the Gateway resource: kubectl describe gateway my-gateway. Look for errors related to load balancer provisioning.
    • Verify your Kubernetes cluster has the necessary IAM/RBAC permissions to create load balancers in your cloud provider.
    • Ensure your cloud account has available load balancer quota.
    • If running on bare-metal or a local cluster, ensure you have a load balancer implementation (e.g., MetalLB) installed and configured.
  2. Issue: HTTPRoute status shows Accepted: False or Programmed: False.

    Explanation: The Envoy Gateway controller failed to process or apply the HTTPRoute configuration to the Envoy Proxy. This often indicates a misconfiguration in the HTTPRoute spec, such as invalid parentRefs, non-existent backendRefs, or conflicting hostnames.

    Solution:

    • Inspect the HTTPRoute events and conditions: kubectl describe httproute http-route-to-nginx. Look for detailed error messages.
    • Check the logs of the Envoy Gateway controller pod: kubectl logs -n envoy-gateway -l app.kubernetes.io/component=controller.
    • Verify the parentRefs correctly points to an existing and ready Gateway.
    • Ensure backendRefs points to an existing Kubernetes Service in the correct namespace.
  3. Issue: curl returns 404 Not Found or 503 Service Unavailable.

    Explanation: The request reached the Envoy Gateway, but it couldn't find a matching route or the backend service was unreachable.

    Solution:

    • Double-check the hostnames in your HTTPRoute match the Host header you are sending with curl.
    • Verify the path matching rules are correct.
    • Ensure the backend Service (e.g., nginx) is healthy and its pods are running: kubectl get pods -l app=nginx.
    • Check the logs of the Envoy Proxy pods: kubectl logs -n envoy-gateway -l app.kubernetes.io/component=proxy. Look for access logs or errors indicating why the request wasn't routed correctly or why the backend failed.
  4. Issue: Unable to connect to Gateway external IP (connection refused/timeout).

    Explanation: The traffic isn't reaching the Envoy Proxy, possibly due to firewall rules, network ACLs, or the load balancer not being fully provisioned.

    Solution:

    • Confirm the Gateway ADDRESS is indeed an external IP and not pending.
    • Check your cloud provider's firewall rules or security groups associated with the load balancer or worker nodes. Ensure ports 80 (and 443 for HTTPS) are open to incoming traffic.
    • Verify the Envoy Proxy pods are running and healthy.
    • If you are using Cilium WireGuard Encryption or other CNI-level security, ensure it's not inadvertently blocking traffic to the Gateway.
  5. Issue: High CPU/Memory usage by Envoy Proxy pods.

    Explanation: The Envoy Proxy instances are under heavy load, potentially due to high traffic volume, complex routing rules, or resource-intensive filters.

    Solution:

    • Scale up the Envoy Proxy deployments using HPA, or manually increase replica count: kubectl scale deployment -n envoy-gateway envoy-gateway-envoy --replicas=3.
    • Review your HTTPRoute configurations for any overly complex regex matches or excessive filter usage that might be CPU-intensive.
    • Increase resource limits and requests for the Envoy Proxy pods if they are consistently hitting limits and being throttled.
    • Analyze Envoy Proxy metrics (e.g., via Prometheus) to identify bottlenecks.
  6. Issue: TLS certificates are not being applied to the Gateway.

    Explanation: This typically happens when the TLSRoute or Gateway TLS configuration is incorrect, or cert-manager is not correctly configured to provision certificates.

    Solution:

    • Ensure your Gateway has an HTTPS listener configured with tls.mode: Terminate and references a valid Kubernetes Secret containing your TLS certificate and key.
    • If using cert-manager, verify your Issuer or ClusterIssuer and Certificate resources are correctly configured and in a Ready state. Check cert-manager controller logs.
    • Confirm the TLS secret exists and is accessible by the Envoy Gateway controller.

FAQ Section

  1. What is the main difference between Kubernetes Ingress and Gateway API?

    The Gateway API is designed to be more expressive, extensible, and role-oriented than Ingress. It separates concerns between infrastructure providers (GatewayClass), cluster operators (Gateway), and application developers (HTTPRoute, etc.). Ingress is simpler but often falls short for complex traffic management, especially in multi-tenant or advanced use cases. For a deeper dive, check out our Kubernetes Gateway API vs Ingress: The Complete Migration Guide.

  2. Why choose Envoy Gateway over other Gateway API implementations?

    Envoy Gateway leverages Envoy Proxy, a high-performance, cloud-native proxy widely adopted in production environments. It offers advanced traffic management features out-of-the-box, excellent observability, and a strong community. Other implementations might use NGINX, HAProxy, or cloud provider-specific load balancers as their data plane.

  3. Can I use Gateway API for internal service-to-service communication?

    While Gateway API is primarily designed for north-south (external to cluster) traffic, it can technically be used for east-west (internal) traffic if desired. However, for internal service mesh capabilities like mTLS, advanced traffic shifting, and policy enforcement, a dedicated service mesh like Istio Ambient Mesh is generally a more robust and feature-rich solution.

  4. How do I handle multiple hostnames or subdomains with Envoy Gateway?

    You can define multiple hostnames within a single HTTPRoute or create separate HTTPRoute resources for different hostnames, all attaching to the same Gateway. The Gateway API supports wildcards (e.g., *.example.com) for hostnames, allowing you to manage many subdomains efficiently.

  5. Is Envoy Gateway production-ready?

    Yes, Envoy Gateway has reached its v1.0 release, indicating a stable API and a commitment to backward compatibility. It's built on the battle-tested Envoy Proxy, making it a solid choice for production deployments. However, like any critical infrastructure component, thorough testing and adherence to best practices are essential.

Cleanup Commands

To remove all resources created during this tutorial:

# Delete Gateway API resources
kubectl delete httproute http-route-to-nginx
kubectl delete gateway my-gateway

# Delete the sample application
kubectl delete -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/nginx-svc.yaml

# Uninstall Envoy Gateway
helm uninstall envoy-gateway -n envoy-gateway
kubectl delete namespace envoy-gateway

# Remove Envoy Gateway Helm repository (optional)
helm repo remove envoy-gateway

Next Steps / Further Reading

You've successfully set up a basic Gateway API with Envoy Gateway. Here are some directions to explore next:

  • TLS Configuration: Implement HTTPS with cert-manager for automated certificate provisioning. Refer to the Envoy Gateway TLS documentation.
  • Advanced Traffic Management: Experiment with traffic splitting, header manipulation, and URL rewriting using HTTPRoute filters. The Gateway API specification is a great resource.
  • Rate Limiting and Authorization: Integrate with external rate limiters or authorization services using Envoy Gateway's extensibility features.
  • Multi-Tenancy: Explore how to set up multi-tenant environments using ReferenceGrants and separate namespaces for different teams.
  • Policy Attachment: Learn about Policy resources for attaching cross-cutting concerns like timeouts or retries to multiple routes or gateways.
  • Observability: Dive deeper into Envoy Proxy metrics and logging. Integrate with Prometheus and Grafana for dashboards. Consider eBPF Observability with Hubble for network insights.
  • Security: Implement robust Kubernetes Network Policies around your Gateway and backend services. For supply chain security, look into tools like Sigstore and Kyverno.

Conclusion

The Kubernetes Gateway API, powered by implementations like Envoy Gateway, represents the future of ingress and traffic management in Kubernetes. Its layered, role-oriented design provides unparalleled flexibility, expressiveness, and control, addressing many of the limitations of the older Ingress API. By following this guide, you've gained hands-on experience deploying and configuring Envoy Gateway, laying a solid foundation for building highly scalable, resilient, and secure applications on Kubernetes. Embrace the Gateway API for a more robust and manageable traffic architecture.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *