TCP/UDP Gateway API: Route Non-HTTP Traffic

Introduction

Kubernetes has become the de facto standard for deploying and managing containerized applications, but orchestrating traffic to these applications, especially non-HTTP/HTTPS services, can be a complex endeavor. Traditionally, Kubernetes Ingress has been the go-to for HTTP traffic, but it falls short when dealing with protocols like raw TCP or UDP, which are prevalent in many critical applications such as databases, gaming servers, IoT platforms, and custom RPC services. This limitation often forces developers to resort to less elegant solutions like NodePort, LoadBalancer services, or even external load balancers configured manually, adding complexity and reducing the native Kubernetes experience.

Enter the Kubernetes Gateway API, a powerful, extensible, and role-oriented API that aims to supersede and enhance Ingress. While the Gateway API shines with its HTTPRoute resource for web traffic, its real power for non-HTTP applications lies in its TCPRoute and UDPRoute resources. These specialized routing mechanisms finally provide a native, Kubernetes-idiomatic way to manage TCP and UDP traffic, offering advanced features like port-based routing, traffic splitting, and policy attachment for these protocols. This guide will walk you through leveraging TCPRoute and UDPRoute to expose your non-HTTP services reliably and efficiently within your Kubernetes clusters.

# Install a Gateway API controller (e.g., Istio)
kubectl apply -f https://istio.io/latest/samples/addons/gateway/istio-ingressgateway.yaml
kubectl apply -f https://istio.io/latest/samples/addons/gateway/gateway-api.yaml

# Create a Gateway
kubectl apply -f tcp-gateway.yaml

# Deploy TCPRoute (example)
kubectl apply -f tcp-route.yaml

# Deploy UDPRoute (example)
kubectl apply -f udp-route.yaml

# Verify Gateway status
kubectl get gateway tcp-gateway -o yaml

# Test TCP connectivity (replace with your service IP/port)
nc -zv <GATEWAY_IP> <TCP_PORT>

# Test UDP connectivity (replace with your service IP/port)
echo "Hello UDP" | nc -u -w1 <GATEWAY_IP> <UDP_PORT>

# Clean up
kubectl delete -f tcp-route.yaml
kubectl delete -f udp-route.yaml
kubectl delete -f tcp-gateway.yaml
# ... and any deployed applications/services

Prerequisites

To follow this guide, you’ll need:

• A running Kubernetes cluster (v1.20+ recommended for Gateway API). You can use Minikube or Kind for local development, or a managed service like AWS EKS, GCP GKE, or Azure AKS.
• kubectl installed and configured to connect to your cluster.
• A Gateway API implementation/controller installed in your cluster. Popular choices include Istio, Contour, Nginx Gateway Fabric, or Kong Gateway. This guide will use Istio for demonstration due to its robust Gateway API support.
• Basic understanding of Kubernetes concepts: Pods, Deployments, Services, and Namespaces.
• Familiarity with network tools like netcat (nc) for testing TCP/UDP connectivity.

Step-by-Step Guide: TCPRoute and UDPRoute for Non-HTTP Traffic

Step 1: Install a Gateway API Controller

Before you can use Gateway, TCPRoute, or UDPRoute resources, you need a controller that implements the Gateway API specifications. For this guide, we’ll use Istio, a powerful service mesh that provides excellent Gateway API support. If you already have a Gateway API controller installed, you can skip this step. For more on service meshes, check out our guide on Istio Ambient Mesh Production Guide.

First, install Istio. You can download the Istio command-line tool (istioctl) and then use it to install the base components and a demo profile. Alternatively, you can directly apply the necessary YAML files for a minimal Gateway API setup.

# Download Istio (if you don't have it)
# curl -L https://istio.io/downloadIstio | sh -
# cd istio-<version>
# export PATH=$PWD/bin:$PATH

# Install Istio base components and Gateway API manifests
# This assumes you have Istio's latest version downloaded and are in its directory
# istioctl install --set profile=demo -y

# Alternatively, for a quicker setup focused on Gateway API:
# Install the Istio Ingress Gateway (if not already present)
kubectl apply -f https://istio.io/latest/samples/addons/gateway/istio-ingressgateway.yaml

# Install the Gateway API CRDs and controller components for Istio
kubectl apply -f https://istio.io/latest/samples/addons/gateway/gateway-api.yaml

# Verify Istio components are running
kubectl get pods -n istio-system

Verify: Ensure that the Istio pods, including the ingress gateway, are running in the istio-system namespace. This indicates that your Gateway API controller is ready.

kubectl get pods -n istio-system

Expected Output:

NAME                                    READY   STATUS    RESTARTS   AGE
istio-ingressgateway-7c47d7b8d4-abcde   1/1     Running   0          2m
istiod-7b8c78c9d-fghij                  1/1     Running   0          2m
# ... other Istio components

Step 2: Define a Gateway

A Gateway resource represents a load balancer or a proxy that exposes services to the outside world. It defines where traffic enters your cluster and what ports it listens on. For non-HTTP traffic, you’ll specify the TCP or UDP ports directly.

We’ll create a Gateway that listens on a specific TCP port (e.g., 9000) and a UDP port (e.g., 9001). The gatewayClassName must match your installed Gateway API controller (e.g., istio for Istio).

# tcp-udp-gateway.yaml
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: tcp-udp-gateway
  namespace: default
spec:
  gatewayClassName: istio # Or whichever Gateway API controller you are using (e.g., nginx, contour)
  listeners:
  - name: tcp-listener
    protocol: TCP
    port: 9000
    allowedRoutes:
      namespaces:
        from: All # Allow TCPRoutes from any namespace
  - name: udp-listener
    protocol: UDP
    port: 9001
    allowedRoutes:
      namespaces:
        from: All # Allow UDPRoutes from any namespace

kubectl apply -f tcp-udp-gateway.yaml

Verify: Check the status of your Gateway. It should eventually show an address assigned, which will be the IP of your ingress gateway.

kubectl get gateway tcp-udp-gateway -o yaml

Expected Output (snippet):

status:
  addresses:
  - type: IPAddress
    value: 192.168.49.2 # This IP will vary, it's your Gateway's external IP
  conditions:
  - ...
    status: "True"
    type: Programmed
  - ...
    status: "True"
    type: Accepted

Make a note of the value under status.addresses. This is the external IP you’ll use to test connectivity.

Step 3: Deploy Sample Backend Services

To demonstrate routing, we need some backend services. We’ll deploy two simple applications:

• A TCP echo server: Listens on a TCP port and echoes back any received data.
• A UDP echo server: Listens on a UDP port and echoes back any received data.

# backend-services.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tcp-echo-server
  labels:
    app: tcp-echo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tcp-echo
  template:
    metadata:
      labels:
        app: tcp-echo
    spec:
      containers:
      - name: tcp-echo
        image: hashicorp/http-echo:latest # This image can also echo TCP
        args: ["-listen", ":5000", "-text", "Hello from TCP Echo!"]
        ports:
        - containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: tcp-echo-service
spec:
  selector:
    app: tcp-echo
  ports:
  - protocol: TCP
    port: 5000
    targetPort: 5000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: udp-echo-server
  labels:
    app: udp-echo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: udp-echo
  template:
    metadata:
      labels:
        app: udp-echo
    spec:
      containers:
      - name: udp-echo
        image: hashicorp/http-echo:latest # This image can also echo UDP
        args: ["-listen", ":5001", "-text", "Hello from UDP Echo!", "-udp"]
        ports:
        - containerPort: 5001
          protocol: UDP
---
apiVersion: v1
kind: Service
metadata:
  name: udp-echo-service
spec:
  selector:
    app: udp-echo
  ports:
  - protocol: UDP
    port: 5001
    targetPort: 5001

kubectl apply -f backend-services.yaml

Verify: Ensure both deployments and services are up and running.

kubectl get deploy,svc -l app=tcp-echo
kubectl get deploy,svc -l app=udp-echo

Expected Output:

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/tcp-echo-server     1/1     1            1           1m

NAME                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/tcp-echo-service     ClusterIP   10.96.123.45     <none>        5000/TCP   1m

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/udp-echo-server     1/1     1            1           1m

NAME                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/udp-echo-service     ClusterIP   10.96.67.89      <none>        5001/UDP   1m

Step 4: Create a TCPRoute

Now, let’s define a TCPRoute to direct incoming TCP traffic on our Gateway’s 9000 port to our tcp-echo-service. The TCPRoute specifies which Gateway it’s attached to, and the rules for routing. In this basic example, all traffic on the specified listener will be forwarded to our backend service.

# tcp-route.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2 # Use v1alpha2 for TCPRoute/UDPRoute
kind: TCPRoute
metadata:
  name: tcp-echo-route
  namespace: default
spec:
  parentRefs:
  - name: tcp-udp-gateway
    namespace: default
    sectionName: tcp-listener # Reference the specific listener defined in the Gateway
  rules:
  - backendRefs:
    - name: tcp-echo-service
      port: 5000

kubectl apply -f tcp-route.yaml

Verify: Check the status of your TCPRoute. It should show as “Accepted” by the Gateway.

kubectl get tcproute tcp-echo-route -o yaml

Expected Output (snippet):

status:
  parents:
  - conditions:
    - message: Route is accepted by the Gateway.
      reason: Accepted
      status: "True"
      type: Accepted
    controllerName: istio.io/gateway-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: tcp-udp-gateway
      namespace: default
      sectionName: tcp-listener

Now, test the TCP connection using netcat. Replace <GATEWAY_IP> with the IP address you noted from the Gateway status in Step 2.

GATEWAY_IP=$(kubectl get gateway tcp-udp-gateway -o jsonpath='{.status.addresses[0].value}')
echo "Hello TCP" | nc -w 1 $GATEWAY_IP 9000

Expected Output:

Hello from TCP Echo!

If you see “Hello from TCP Echo!”, your TCPRoute is working correctly! For more advanced networking topics, consider exploring Kubernetes Network Policies to secure your backend services.

Step 5: Create a UDPRoute

Similarly, we’ll define a UDPRoute to direct incoming UDP traffic on our Gateway’s 9001 port to our udp-echo-service.

# udp-route.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2 # Use v1alpha2 for TCPRoute/UDPRoute
kind: UDPRoute
metadata:
  name: udp-echo-route
  namespace: default
spec:
  parentRefs:
  - name: tcp-udp-gateway
    namespace: default
    sectionName: udp-listener # Reference the specific listener defined in the Gateway
  rules:
  - backendRefs:
    - name: udp-echo-service
      port: 5001

kubectl apply -f udp-route.yaml

Verify: Check the status of your UDPRoute. It should also show as “Accepted” by the Gateway.

kubectl get udproute udp-echo-route -o yaml

Expected Output (snippet):

status:
  parents:
  - conditions:
    - message: Route is accepted by the Gateway.
      reason: Accepted
      status: "True"
      type: Accepted
    controllerName: istio.io/gateway-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: tcp-udp-gateway
      namespace: default
      sectionName: udp-listener

Now, test the UDP connection using netcat. Remember to replace <GATEWAY_IP> with your Gateway’s external IP.

GATEWAY_IP=$(kubectl get gateway tcp-udp-gateway -o jsonpath='{.status.addresses[0].value}')
echo "Hello UDP" | nc -u -w 1 $GATEWAY_IP 9001

Expected Output:

Hello from UDP Echo!

If you see “Hello from UDP Echo!”, your UDPRoute is working correctly!

Step 6: Advanced Routing (Traffic Splitting – TCPRoute Example)

TCPRoute and UDPRoute also support advanced features like traffic splitting, similar to HTTPRoute. This is particularly useful for A/B testing, canary deployments, or phased rollouts of your non-HTTP services. Let’s demonstrate a TCPRoute splitting traffic between two versions of our TCP echo service.

First, create a second version of our TCP echo service:

# tcp-echo-v2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tcp-echo-server-v2
  labels:
    app: tcp-echo-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tcp-echo-v2
  template:
    metadata:
      labels:
        app: tcp-echo-v2
    spec:
      containers:
      - name: tcp-echo
        image: hashicorp/http-echo:latest
        args: ["-listen", ":5000", "-text", "Hello from TCP Echo V2!"]
        ports:
        - containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: tcp-echo-service-v2
spec:
  selector:
    app: tcp-echo-v2
  ports:
  - protocol: TCP
    port: 5000
    targetPort: 5000

kubectl apply -f tcp-echo-v2.yaml

Now, modify the tcp-route.yaml to split traffic 50/50 between tcp-echo-service and tcp-echo-service-v2:

# tcp-route-split.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: tcp-echo-route
  namespace: default
spec:
  parentRefs:
  - name: tcp-udp-gateway
    namespace: default
    sectionName: tcp-listener
  rules:
  - backendRefs:
    - name: tcp-echo-service
      port: 5000
      weight: 50
    - name: tcp-echo-service-v2
      port: 5000
      weight: 50

kubectl apply -f tcp-route-split.yaml

Verify: Repeatedly test the TCP connection. You should see responses alternating between “Hello from TCP Echo!” and “Hello from TCP Echo V2!”.

GATEWAY_IP=$(kubectl get gateway tcp-udp-gateway -o jsonpath='{.status.addresses[0].value}')
for i in $(seq 1 10); do echo "Hello TCP" | nc -w 1 $GATEWAY_IP 9000; done

Expected Output (mixed):

Hello from TCP Echo V2!
Hello from TCP Echo!
Hello from TCP Echo!
Hello from TCP Echo V2!
Hello from TCP Echo!
Hello from TCP Echo V2!
Hello from TCP Echo V2!
Hello from TCP Echo!
Hello from TCP Echo V2!
Hello from TCP Echo!

This demonstrates the power of TCPRoute for sophisticated traffic management even for raw TCP connections. For advanced networking and observability, consider exploring eBPF Observability with Hubble.

Production Considerations

Deploying TCPRoute and UDPRoute in a production environment requires careful planning and consideration beyond basic functionality:

1. Gateway Controller Choice: Select a robust and production-ready Gateway API implementation. Istio, Nginx Gateway Fabric, and Contour are popular choices, each with its strengths in terms of features, performance, and ecosystem integration. Evaluate their stability, community support, and enterprise features like advanced traffic policies, TLS termination (for TCP), and metrics.
2. Security:
   • Network Policies: Even with Gateway API, internal traffic within your cluster should be secured. Implement Kubernetes Network Policies to restrict pod-to-pod communication to only what is necessary, creating a defense-in-depth strategy.
   • TLS Termination for TCP: While TCPRoute itself doesn’t offer HTTP-level TLS, many Gateway implementations can terminate TLS for raw TCP connections. This is crucial for securing protocols like MQTT, databases (e.g., PostgreSQL, MySQL), or custom secure RPC. Configure your Gateway listener with a TLS block pointing to a Kubernetes Secret containing your certificate and key.
   • Authentication/Authorization: For non-HTTP protocols, implementing authentication and authorization often requires application-level logic or integration with your Gateway controller’s capabilities (e.g., Istio’s authorization policies).
   • Supply Chain Security: Ensure the images used for your Gateway controller and backend services are secure. Tools like Sigstore and Kyverno can help enforce image signing and policy compliance.
3. Observability:
   • Metrics: Ensure your Gateway controller exposes metrics (e.g., Prometheus-compatible) for connection rates, error rates, latency, and bytes transferred for TCP/UDP listeners and routes.
   • Logging: Configure comprehensive logging for your Gateway and backend services to aid in debugging and security auditing.
   • Tracing: While less common for raw TCP/UDP, some advanced Gateway controllers (like Istio) can provide connection-level tracing, which can be invaluable for diagnosing issues in complex non-HTTP application flows.
   • eBPF: Leverage eBPF-based tools like eBPF Observability with Hubble to gain deep insights into network traffic at the kernel level, especially useful for debugging low-level TCP/UDP issues.
4. High Availability & Scalability:
   • Gateway Replicas: Run multiple replicas of your Gateway controller’s ingress pods to ensure high availability and distribute load.
   • Load Balancer Integration: For cloud environments, ensure your Gateway’s external IP is backed by a cloud load balancer (e.g., AWS ELB/NLB, GCP Load Balancer) that can handle the expected traffic volume and provide fault tolerance.
   • Autoscaling: Implement Horizontal Pod Autoscaling (HPA) for your Gateway controller pods based on CPU, memory, or custom metrics related to connection count or bandwidth. Consider Karpenter for node autoscaling to ensure underlying infrastructure can scale with demand.
5. Traffic Management:
   • Timeout & Retries: While not directly part of TCPRoute/UDPRoute, your application or Gateway controller might offer configuration for connection timeouts and retries, which are critical for robust non-HTTP service communication.
   • Rate Limiting: Protect your backend services from overload by implementing rate limiting at the Gateway level if supported by your controller.
6. Port Conflicts: Carefully plan your port allocations. Ensure that the ports listened on by your Gateway (e.g., 9000, 9001) do not conflict with other services or the node’s ephemeral port range.
7. Observing API Status: Regularly monitor the status field of your Gateway, TCPRoute, and UDPRoute resources. This provides crucial information on whether the resources have been accepted, programmed, and are functioning correctly by the Gateway controller.

Troubleshooting

Here are common issues you might encounter when working with Gateway API’s TCPRoute and UDPRoute, along with their solutions:

1. Issue: Gateway status.addresses is pending or empty.

   Explanation: This usually means the underlying load balancer or ingress service for your Gateway controller hasn’t provisioned an external IP yet, or there’s an issue with the Gateway controller itself. This is common in cloud environments where external LoadBalancer services take time to become active, or in local environments without a LoadBalancer provisioner.

   Solution:

   • Wait a few minutes. Cloud load balancers can take time.
   • Check the logs of your Gateway controller (e.g., Istio Ingress Gateway pods in istio-system namespace).
   • Ensure your Kubernetes cluster has a LoadBalancer provisioner if running on-premises or a bare-metal cluster. For local clusters like Minikube, you might need to enable the metallb addon or use minikube tunnel.
   • Verify the gatewayClassName in your Gateway resource matches an installed controller.

   kubectl logs -n istio-system $(kubectl get pod -n istio-system -l app=istio-ingressgateway -o jsonpath='{.items[0].metadata.name}')
   

2. Issue: TCPRoute/UDPRoute status.parents shows “Not Accepted” or “No Ready Listeners”.

   Explanation: The Gateway controller has not accepted your route, usually due to a misconfiguration in the route or the Gateway itself. “No Ready Listeners” indicates the Gateway isn’t listening on the specified port/protocol.

   Solution:

   • Check the sectionName in your route’s parentRefs. It must exactly match a name in the Gateway’s listeners.
   • Ensure the Gateway is in an “Accepted” and “Programmed” state.
   • Verify the protocol and port in the Gateway listener match the traffic type you expect.
   • Check the events for the Gateway and Route resources for more clues: kubectl describe gateway <gateway-name> and kubectl describe tcproute <route-name>.
3. Issue: Connectivity fails (nc hangs or times out) even after Gateway and Route are accepted.

   Explanation: This points to issues deeper in the network path or with the backend service itself. It could be network policies, service selector issues, or the application not listening correctly.

   Solution:

   • Check backend service: Can you connect directly to the backend service from another pod inside the cluster? Use kubectl exec into a debug pod and try nc <service-name> <port>.
   • Network Policies: Ensure no Kubernetes Network Policies are blocking traffic from the Gateway controller to your backend service pods.
   • Service Selector: Verify the selector in your Kubernetes Service matches the labels on your backend Deployment’s pods.
   • Firewall Rules: If running on a cloud provider, ensure the security groups/firewall rules associated with your Gateway’s external IP allow incoming traffic on the listener ports (e.g., 9000, 9001).
4. Issue: Traffic splitting is not working as expected (e.g., always hitting one version).

   Explanation: Traffic splitting relies on the Gateway controller’s implementation. If weights are not respected, there might be a configuration error or a bug/limitation in the controller.

   Solution:

   • Double-check the weight values in your backendRefs. They should sum up to 100 for predictable splitting.
   • Ensure you have multiple replicas for each backend service version so that the load balancer has multiple targets to distribute traffic to.
   • Check the logs of the Gateway controller for any errors related to route configuration.
   • Consult the documentation for your specific Gateway API implementation (e.g., Istio traffic management docs) for any specific requirements or nuances for TCP/UDP traffic splitting.
5. Issue: UDP traffic is unreliable or lost.

   Explanation: UDP is a connectionless protocol, which by nature is less reliable than TCP. Packet loss can occur due to network congestion, firewall rules, or application-level issues. UDP routing also requires careful handling by the Gateway controller.

   Solution:

   • Retest with -w: Ensure you’re using nc -u -w 1 (or similar timeout) for UDP tests to prevent the client from waiting indefinitely.
   • Check application: Verify your UDP backend application is robust to packet loss and is indeed listening and responding on the specified UDP port.
   • Network congestion: Monitor network utilization on the Gateway and backend nodes.
   • Gateway Controller Support: Confirm your chosen Gateway API implementation fully supports UDP routing and any specific limitations it might have. Some controllers have more mature TCP support than UDP.
   • eBPF Observability: Use tools like eBPF Observability with Hubble to analyze packet flow and identify where UDP packets might be getting dropped or misrouted at a low level.

FAQ Section

1. Q: What is the main difference between Gateway API and Ingress for non-HTTP traffic?

   A: Ingress is primarily designed for HTTP/HTTPS traffic and lacks native support for raw TCP or UDP protocols. While some Ingress controllers offer extensions for TCP/UDP, these are usually non-standard and implementation-specific. The Kubernetes Gateway API, with its TCPRoute and UDPRoute resources, provides a standardized, native, and extensible way to manage non-HTTP traffic, offering features like port-based routing, traffic splitting, and policy attachment directly within the API.

2. Q: Can I use the same Gateway for HTTP, TCP, and UDP traffic?

   A: Yes, absolutely! A single Gateway resource can define multiple listeners, each for a different protocol and port. For example, you can have a listener for HTTP (port 80), HTTPS (port 443), a custom TCP service (port 9000