Introduction
In the dynamic landscape of modern cloud-native applications, managing traffic effectively at the edge of your Kubernetes cluster is paramount. Beyond basic routing, advanced traffic management capabilities like rate limiting and authentication are crucial for ensuring application stability, security, and fair resource usage. Traditionally, achieving these features often involved complex configurations within Ingress controllers or bespoke sidecar proxies. However, the Kubernetes Gateway API is revolutionizing how we define and manage application traffic, offering a more expressive, extensible, and role-oriented approach.
The Gateway API introduces a powerful concept: Policy Attachment. This mechanism allows you to attach various policies—like rate limiting, authentication, authorization, and even custom logic—to different levels of your Gateway API resources, such as Gateways, HTTPRoutes, or even specific hostnames and paths. This granular control decouples policy definition from core routing logic, making your traffic management configurations cleaner, more modular, and easier to manage across various teams and environments. In this guide, we’ll dive deep into using Policy Attachment for two critical use cases: rate limiting to protect your services from overload and authentication to secure access to your applications.
This tutorial will walk you through setting up a Kubernetes cluster, deploying a Gateway API implementation (like Contour or Istio), and then demonstrating how to apply rate limiting and external authentication policies using the Gateway API’s attachment model. By the end, you’ll have a solid understanding of how to leverage these powerful features to build robust and secure applications on Kubernetes. For those still using Ingress, our Kubernetes Gateway API vs Ingress: The Complete Migration Guide offers a great starting point for understanding the benefits of migrating.
TL;DR: Gateway API Policy Attachment for Rate Limiting and Auth
The Gateway API allows granular control over traffic management policies like rate limiting and authentication through Policy Attachment. This guide demonstrates how to apply these policies to HTTPRoutes using a Gateway API implementation.
Key Steps:
- Install Gateway API CRDs & Controller: Deploy the Gateway API CRDs and a compatible controller (e.g., Contour).
- Deploy Sample Application: A simple NGINX server.
- Create Gateway & HTTPRoute: Define your entry point and routing rules.
- Apply Rate Limiting Policy: Create a
HTTPRateLimitFilterand attach it to your HTTPRoute. - Apply External Auth Policy: Create an
HTTPAuthFilterfor external authentication and attach it. - Test Policies: Verify rate limiting and authentication are enforced.
Key Commands:
# Install Gateway API CRDs (if not already present)
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
# Install Contour (as an example Gateway API implementation)
kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
# Deploy sample application
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/nginx-deployment.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/nginx-service.yaml
# Create Gateway & HTTPRoute
kubectl apply -f gateway.yaml
kubectl apply -f httproute.yaml
# Apply Rate Limit Policy (example using Contour's HTTPProxy extension)
kubectl apply -f rate-limit-policy.yaml
# Apply External Auth Policy (example using Contour's HTTPProxy extension)
kubectl apply -f auth-policy.yaml
# Test Rate Limiting
for i in $(seq 1 10); do curl -s -o /dev/null -w "%{http_code}\n" http://<GATEWAY_IP>/rate-limited; done
# Test Authentication
curl -s -o /dev/null -w "%{http_code}\n" http://<GATEWAY_IP>/auth-protected
curl -s -o /dev/null -w "%{http_code}\n" -H "Authorization: Bearer valid-token" http://<GATEWAY_IP>/auth-protected
Prerequisites
Before we begin, ensure you have the following:
- Kubernetes Cluster: A running Kubernetes cluster (v1.24+ recommended, though Gateway API CRDs can be installed on older versions). You can use Minikube, Kind, or a cloud-managed cluster (EKS, GKE, AKS).
kubectl: The Kubernetes command-line tool, configured to connect to your cluster. Refer to the official Kubernetes documentation for installation instructions.- Basic understanding of Kubernetes: Familiarity with Deployments, Services, and basic networking concepts.
- Basic understanding of Gateway API: Knowledge of Gateway, HTTPRoute, and GatewayClass resources. If you’re new to the Gateway API, our Gateway API Migration Guide is an excellent resource.
- A Gateway API Implementation: We will use Contour as our Gateway API controller for this tutorial due to its robust support for policy attachment through its
HTTPProxyextension, which aligns well with the Gateway API’s extensibility model. Other implementations like Istio (see our Istio Ambient Mesh Production Guide), Kong Gateway, or Traefik also support the Gateway API and policy attachment, though the exact CRDs for policies might differ.
Step-by-Step Guide
Step 1: Install Gateway API CRDs and a Gateway Controller
First, we need to ensure the Gateway API Custom Resource Definitions (CRDs) are installed in your cluster. These CRDs define the Gateway, HTTPRoute, and other related resources. Following that, we’ll install a Gateway API controller that will implement these resources. For this guide, we’ll use Contour, which provides a robust and feature-rich implementation of the Gateway API. Contour will manage the underlying Envoy proxies to handle traffic according to our Gateway API configurations.
The Gateway API CRDs are fundamental for any Gateway API deployment. They provide the necessary schema for Kubernetes to understand and validate the Gateway API resources you create. Once the CRDs are in place, a Gateway controller like Contour watches these resources and translates them into configurations for its data plane (Envoy, in Contour’s case). This separation of concerns is a core tenet of the Gateway API, allowing different implementations to adhere to a common API.
# Install Gateway API CRDs (if not already present on your cluster)
# This command installs the standard Gateway API CRDs.
echo "--- Installing Gateway API CRDs ---"
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
# Verify CRDs are installed
echo "--- Verifying Gateway API CRDs ---"
kubectl get crd gateways.gateway.networking.k8s.io httproutes.gateway.networking.k8s.io gatewayclasses.gateway.networking.k8s.io
# Install Contour as the Gateway API controller
# This quickstart manifest includes Contour and Envoy deployments, services, and necessary RBAC.
echo "--- Installing Contour Gateway API controller ---"
kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
# Verify Contour deployment
echo "--- Verifying Contour deployment ---"
kubectl get pods -n projectcontour
# Wait for Contour and Envoy to be ready
echo "--- Waiting for Contour and Envoy to be ready ---"
kubectl rollout status deployment/contour -n projectcontour
kubectl rollout status deployment/envoy -n projectcontour
Verify:
--- Installing Gateway API CRDs ---
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
--- Verifying Gateway API CRDs ---
NAME CREATED AT
gatewayclasses.gateway.networking.k8s.io 2023-10-27T10:00:00Z
httproutes.gateway.networking.k8s.io 2023-10-27T10:00:00Z
gatewayclasses.gateway.networking.k8s.io 2023-10-27T10:00:00Z
--- Installing Contour Gateway API controller ---
namespace/projectcontour created
serviceaccount/contour created
...
--- Verifying Contour deployment ---
NAME READY STATUS RESTARTS AGE
contour-6f89b94c5-abcde 1/1 Running 0 2m
envoy-7bc7f7f7f-fghij 1/1 Running 0 2m
--- Waiting for Contour and Envoy to be ready ---
Waiting for deployment "contour" rollout to finish: 1 of 1 updated replicas are available...
deployment "contour" successfully rolled out
Waiting for deployment "envoy" rollout to finish: 1 of 1 updated replicas are available...
deployment "envoy" successfully rolled out
Step 2: Deploy a Sample Application
To demonstrate traffic management, we need a backend service. We’ll deploy a simple NGINX application with a Deployment and a Service. This NGINX server will serve as the target for our HTTPRoute and the policies we attach to it.
This standard NGINX deployment and service setup is a common pattern in Kubernetes. The Deployment ensures that a specified number of NGINX pods are running, providing high availability. The Service then provides a stable IP address and DNS name for these pods, abstracting away the individual pod IPs and enabling load balancing across them. Our Gateway API configuration will route external traffic to this Service.
# nginx-deployment.yamlapiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginx replicas: 2 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:latest ports: - containerPort: 80# nginx-service.yaml
apiVersion: v1 kind: Service metadata: name: nginx-service spec: selector: app: nginx ports: - protocol: TCP port: 80 targetPort: 80# Apply the application manifests
echo "--- Deploying NGINX sample application ---"
kubectl apply -f nginx-deployment.yaml
kubectl apply -f nginx-service.yaml
Verify:
--- Deploying NGINX sample application --- deployment.apps/nginx-deployment created service/nginx-service created kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-deployment-78f5d6976-abcde 1/1 Running 0 1m nginx-deployment-78f5d6976-fghij 1/1 Running 0 1m kubectl get svc nginx-service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-service ClusterIP 10.96.100.101 <none> 80/TCP 1mStep 3: Create a Gateway and HTTPRoute
Next, we’ll define a Gateway and an HTTPRoute. The Gateway resource represents the entry point for external traffic into your cluster, typically backed by a LoadBalancer Service. The HTTPRoute defines how HTTP traffic arriving at the Gateway should be routed to backend Services. We’ll configure an HTTPRoute to route traffic for a specific path to our NGINX service.
The Gateway API is designed with role separation in mind. Cluster operators manage GatewayClasses and Gateways, defining the infrastructure. Application developers create HTTPRoutes, defining how their applications consume that infrastructure. This separation allows for better governance and scalability. For an in-depth look at this, refer to our Kubernetes Gateway API Migration Guide.
# gateway.yamlapiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: my-gateway spec: gatewayClassName: contour listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: All# httproute.yaml
apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: nginx-httproute spec: parentRefs: - name: my-gateway sectionName: http hostnames: - "example.com" # Replace with a real domain or use 'localhost' if testing locally rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: nginx-service port: 80# Apply Gateway and HTTPRoute
echo "--- Creating Gateway and HTTPRoute ---"
kubectl apply -f gateway.yaml
kubectl apply -f httproute.yaml
Verify:
--- Creating Gateway and HTTPRoute --- gateway.gateway.networking.k8s.io/my-gateway created httproute.gateway.networking.k8s.io/nginx-httproute created kubectl get gateway my-gateway NAME CLASS ADDRESS READY HTTP HTTPS AGE my-gateway contour 192.168.1.10 True 80 443 1m # Address will be your LoadBalancer IP kubectl get httproute nginx-httproute NAME HOSTNAMES PARENT REFS AGE nginx-httproute ["example.com"] ["my-gateway"] 1m # Get the external IP of the Gateway export GATEWAY_IP=$(kubectl get svc -n projectcontour envoy -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo "Gateway IP: $GATEWAY_IP" # Test basic routing (if using example.com, you need to configure your hosts file or DNS) # For local testing, you might need to curl with Host header: # curl -H "Host: example.com" http://$GATEWAY_IP/Step 4: Implement Rate Limiting Policy Attachment
Now, let’s attach a rate limiting policy. The Gateway API allows implementations to extend its functionality with custom policies. Contour, for example, uses its own HTTPProxy Rate Limiting features, which can be attached directly to an HTTPRoute. This demonstrates the extensibility of the Gateway API’s policy attachment model. The policy defines rules, like allowing only a certain number of requests per unit of time from a specific client.
Rate limiting is a critical defense mechanism for any public-facing application. It prevents abuse, protects against denial-of-service (DoS) attacks, and ensures fair usage of resources. By attaching this policy directly to an HTTPRoute, we can apply it granularly to specific application paths without modifying the application code itself. This is a powerful form of network-level control that complements other security measures like Kubernetes Network Policies.
# rate-limit-policy.yamlapiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: name: nginx-httproute-ratelimit spec: parentRefs: - name: my-gateway sectionName: http hostnames: - "example.com" rules: - matches: - path: type: PathPrefix value: /rate-limited filters: - type: ExtensionRef extensionRef: group: projectcontour.io kind: HTTPProxy name: rate-limit-extension backendRefs: - name: nginx-service port: 80 --- # This is a Contour specific HTTPProxy that acts as an "ExtensionRef" policy # This demonstrates how Gateway API implementations can expose custom policy CRDs. apiVersion: projectcontour.io/v1 kind: HTTPProxy metadata: name: rate-limit-extension namespace: default # Must be in the same namespace as the HTTPRoute spec: virtualHost: fqdn: example.com rateLimitPolicy: local: requests: 5 # Allow 5 requests unit: Minute # Per minute# Apply the rate limiting policy
echo "--- Applying Rate Limiting Policy ---"
kubectl apply -f rate-limit-policy.yaml
Verify:
--- Applying Rate Limiting Policy --- httproute.gateway.networking.k8s.io/nginx-httproute-ratelimit configured httpproxy.projectcontour.io/rate-limit-extension created # Get the Gateway IP export GATEWAY_IP=$(kubectl get svc -n projectcontour envoy -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo "Gateway IP: $GATEWAY_IP" # Test rate limiting: Send more than 5 requests in a minute to /rate-limited path echo "--- Testing Rate Limiting (expecting 200s then 429s) ---" for i in $(seq 1 10); do curl -s -o /dev/null -w "%{http_code}\n" -H "Host: example.com" http://$GATEWAY_IP/rate-limited sleep 0.5 # Small delay to simulate real traffic doneExpected Output:
200 200 200 200 200 429 # After 5 requests, subsequent requests should return HTTP 429 Too Many Requests 429 429 429 429Step 5: Implement External Authentication Policy Attachment
For authentication, we’ll demonstrate how to integrate an external authentication service. Similar to rate limiting, Gateway API implementations can provide custom filters for this purpose. Contour supports an External Authorization filter. We’ll set up a dummy external auth service that approves requests with a specific header and denies others, then attach this policy to a different path on our HTTPRoute.
External authentication is a powerful pattern for centralizing access control. Instead of embedding authentication logic in every microservice, an external auth service can handle tasks like JWT validation, OAuth, or API key verification. The Gateway API’s policy attachment mechanism makes it seamless to integrate such a service at the edge, securing your applications without tight coupling. This approach is highly recommended for securing your services, especially when combined with other security best practices like those explored in our Securing Container Supply Chains with Sigstore and Kyverno guide.
First, deploy a simple external authentication service:
# auth-service.yamlapiVersion: apps/v1 kind: Deployment metadata: name: external-auth-deployment spec: selector: matchLabels: app: external-auth replicas: 1 template: metadata: labels: app: external-auth spec: containers: - name: external-auth image: nginxdemos/hello:plain-text # Using a simple HTTP server for demonstration ports: - containerPort: 80 command: ["/bin/sh", "-c"] args: - | cat <<EOF >/usr/share/nginx/html/index.html <!DOCTYPE html> <html> <head><title>External Auth Service</title></head> <body> <h1>External Auth Service</h1> <p>This service simulates an external authentication server.</p> </body> </html> EOF nginx -g 'daemon off;' --- apiVersion: v1 kind: Service metadata: name: external-auth-service spec: selector: app: external-auth ports: - protocol: TCP port: 80 targetPort: 80# Apply the auth service manifests
echo "--- Deploying External Auth Service ---"
kubectl apply -f auth-service.yaml
Verify Auth Service:
--- Deploying External Auth Service --- deployment.apps/external-auth-deployment created service/external-auth-service created kubectl get pods -l app=external-auth NAME READY STATUS RESTARTS AGE external-auth-deployment-abcdefgh-ijkl 1/1 Running 0 1m kubectl get svc external-auth-service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE external-auth-service ClusterIP 10.96.10.11 <none> 80/TCP 1mNow, attach the authentication policy to a new path on our HTTPRoute:
# auth-policy.yamlapiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: name: nginx-httproute-auth spec: parentRefs: - name: my-gateway sectionName: http hostnames: - "example.com" rules: - matches: - path: type: PathPrefix value: /auth-protected filters: - type: ExtensionRef extensionRef: group: projectcontour.io kind: HTTPProxy name: auth-extension backendRefs: - name: nginx-service port: 80 --- # This is a Contour specific HTTPProxy that acts as an "ExtensionRef" policy # It configures external authentication using the deployed external-auth-service. apiVersion: projectcontour.io/v1 kind: HTTPProxy metadata: name: auth-extension namespace: default # Must be in the same namespace as the HTTPRoute spec: virtualHost: fqdn: example.com # externalAuth: # authPolicy: # authServer: # extensionService: external-auth-service.default.svc.cluster.local # timeout: 100ms # failOpen: false # tls: # caSecret: # Optional: if external auth service uses TLS # pathPrefix: / # port: 80 # loadBalancerPolicy: # strategy: RoundRobin # response: # allowedRequestHeaders: # - "x-ext-auth-user" # allowedResponseHeaders: # - "x-ext-auth-status" # # If you have a real auth service that returns a 200 for success, 401/403 for failure. # # This example uses a simple service that always returns 200, so we use a header check. # # This is a simplified example, a real auth service would respond with appropriate HTTP codes. # # For demonstration, we'll use a simple header check with a real auth service that you need to implement. # # For the purpose of this demo, we assume the external-auth-service would respond with 200 OK # # and a specific header like 'x-auth-status: authorized' for successful authentication. # # If the header is missing or incorrect, Contour would deny based on failOpen: false. # TEMPORARY NOTE: The current Contour HTTPProxy externalAuth filter expects a gRPC service. # For a simple HTTP demo, we can simulate by using a custom filter or a more advanced external auth server. # A true external auth service would implement the Envoy External Authorization gRPC API. # For a pure HTTP check, you might typically use a WASM filter or similar. # For this tutorial, we will illustrate the concept using a placeholder and note the actual implementation complexity. # A real external auth service would be a dedicated application implementing the gRPC protocol. # For illustration, let's assume a basic external auth setup which is common, even if the service logic is simplified. # We will use projectcontour.io/v1beta1.ExtensionService for a generic external service integration. # This example will assume an external service that responds with a 200 for "authorized" and 401/403 otherwise. # A more realistic external auth setup using Contour's ExtensionService (requires Contour 1.25+) # This configuration is for a real external auth server that conforms to Envoy's External Authorization API. # We'll use a dummy service and manually verify behavior. routes: - conditions: - prefix: /auth-protected services: - name: nginx-service port: 80 # This is where the extensionRef for external auth would typically go for a single Route # However, for Contour externalAuth, it's defined at the virtualHost level or via ExtensionService. # We'll use a simplified model for demonstration purposes as a standalone HTTPProxy. --- # For a working example of external auth with Contour, you would typically use an ExtensionService # and an HTTPProxy referencing it. # Let's define an ExtensionService first. apiVersion: projectcontour.io/v1alpha1 kind: ExtensionService metadata: name: auth-extension-service namespace: default spec: services: - name: external-auth-service port: 80 protocol: h2c # Envoy's External Auth service typically uses gRPC (h2c for plaintext HTTP/2) loadBalancerPolicy: strategy: RoundRobin --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: name: nginx-httproute-auth-ext spec: parentRefs: - name: my-gateway sectionName: http hostnames: - "example.com" rules: - matches: - path: type: PathPrefix value: /auth-protected-ext filters: - type: ExtensionRef extensionRef: group: projectcontour.io kind: HTTPProxy name: auth-proxy-with-extauth backendRefs: - name: nginx-service port: 80 --- apiVersion: projectcontour.io/v1 kind: HTTPProxy metadata: name: auth-proxy-with-extauth namespace: default spec: virtualHost: fqdn: example.com externalAuth: authPolicy: authServer: extensionService: auth-extension-service.default.svc.cluster.local # Reference the ExtensionService timeout: 100ms failOpen: false # Deny by default if auth service is down or responds negatively pathPrefix: / loadBalancerPolicy: strategy: RoundRobin response: allowedRequestHeaders: - "Authorization" # Allow Authorization header to pass to auth service allowedResponseHeaders: - "x-ext-auth-user" - "x-ext-auth-status"# Apply the authentication policy
echo "--- Applying External Authentication Policy ---"
kubectl apply -f auth-policy.yaml
Verify:
--- Applying External Authentication Policy --- deployment.apps/external-auth-deployment unchanged service/external-auth-service unchanged extensionservice.projectcontour.io/auth-extension-service created httproute.gateway.networking.k8s.io/nginx-httproute-auth-ext created httpproxy.projectcontour.io/auth-proxy-with-extauth created # Get the Gateway IP export GATEWAY_IP=$(kubectl get svc -n projectcontour envoy -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo "Gateway IP: $GATEWAY_IP" # Test authentication: echo "--- Testing Authentication (expecting 401/403 then 200) ---" # Without authentication header (should be denied by external auth service) echo "Request without auth header:" curl -v -H "Host: example.com" http://$GATEWAY_IP/auth-protected-ext # With a dummy authentication header (assuming our dummy auth service would pass this) # NOTE: Our dummy external-auth-service is just NGINX, it doesn't *actually* authenticate. # A real external auth service would implement gRPC and return success/failure. # For this demo, if failOpen is false, any non-200/non-gRPC-success response from the auth service will result in denial. # To truly test, you'd need a gRPC external auth server that explicitly returns success. # Assuming a real auth service that accepts "Bearer valid-token" and returns 200. echo "Request with valid auth header (conceptual):" curl -v -H "Host: example.com" -H "Authorization: Bearer valid-token" http://$GATEWAY_IP/auth-protected-extExpected Output (conceptual, depends on actual external auth service logic):
Request without auth header: < HTTP/1.1 401 Unauthorized # Or 403 Forbidden, depending on auth service logic ... Request with valid auth header (conceptual): < HTTP/1.1 200 OK ... (contents of NGINX backend)Note: The dummy external-auth-service used in this example is a simple NGINX server and does not implement the Envoy External Authorization gRPC API. For a functional external authentication flow, you would need a dedicated microservice that implements this API and correctly responds with authorization decisions. The YAML provided for
auth-proxy-with-extauthcorrectly configures Contour to use such a service.Production Considerations
When deploying Gateway API policy attachments in a production environment, several factors need careful consideration to ensure reliability, security, and scalability:
- Gateway API Implementation Choice:
- Maturity and Features: Evaluate different Gateway API implementations (Contour, Istio, Kong, Traefik, etc.) based on their maturity,
