Cilium Tetragon: eBPF-Powered Runtime Security for Kubernetes and Docker

Introduction: The Runtime Security Challenge

In today’s cloud-native landscape, traditional security tools struggle to keep pace with the dynamic nature of containerized workloads. By the time most security solutions detect a threat in user space, analyze it, and decide on an action, the damage is often already done. This gap between detection and response creates what security professionals call the Time-of-Check to Time-of-Use (TOCTOU) vulnerability window.

Enter Cilium Tetragon, an open-source eBPF-based security observability and runtime enforcement platform that fundamentally changes this paradigm. As a CNCF project developed by Isovalent (now part of Cisco), Tetragon brings kernel-level security intelligence to Kubernetes, Docker, and Linux environments—with performance overhead typically under 1%.

What makes Tetragon revolutionary is its ability to monitor, filter, and enforce security policies entirely within the Linux kernel using eBPF (extended Berkeley Packet Filter). This means threats can be detected and blocked synchronously, at the exact moment they occur, without the latency and blind spots of traditional user-space security tools.

Why Tetragon Matters: Real-World Security Challenges

The XZ Utils Backdoor: A Case Study

The XZ Utils backdoor (CVE-2024-3094) discovered in March 2024 exemplified the sophisticated supply chain attacks that modern security tools must detect. This backdoor, embedded in a widely-used compression library, could have compromised countless systems running SSH servers.

Tetragon demonstrated its capability to detect this threat through zero-trust observability—monitoring SSH process behavior at the kernel level without prior knowledge of the specific vulnerability. Security teams using Tetragon could observe the anomalous process execution chains that indicated backdoor exploitation, even before the CVE was publicly disclosed.

According to Isovalent’s analysis, Tetragon users could detect and prevent the XZ Utils exploit by monitoring:

• Unexpected process spawning from SSH daemon
• Unusual file system access patterns
• Suspicious network connections initiated by system services

Supply Chain Security at Scale

Google’s security research revealed that 44% of Linux kernel privilege escalation exploits require unprivileged user namespaces. Tetragon addresses this by providing pre-built tracing policies that monitor:

• Creation of user namespaces by unprivileged processes
• Execution of binaries with file capabilities
• Privilege escalation attempts
• Container escape attempts
• Cryptomining activities

How Tetragon Works: eBPF Architecture Explained

The eBPF Foundation

eBPF (extended Berkeley Packet Filter) revolutionized the Linux kernel by allowing programs to run in a sandboxed environment within the kernel itself—without requiring kernel module loading or source code changes. As detailed in the eBPF Ecosystem Progress report, eBPF enables:

1. Dynamic Instrumentation: Hook into kernel functions, system calls, and tracepoints
2. Safety: Verifier ensures programs won’t crash the kernel
3. Performance: In-kernel execution eliminates user-space context switching
4. Flexibility: Load and unload programs at runtime

Tetragon’s Three-Layer Architecture

According to Tetragon’s official documentation, the platform operates through three key components:

1. Hook Points – Where to inject observability:

• kprobes: Dynamic hooks into kernel functions (listed in /proc/kallsyms)
• tracepoints: Stable, standardized kernel event hooks (found in /sys/kernel/debug/tracing/events)
• uprobes: Hooks into user-space application functions
• BPF LSM: Linux Security Module hooks for enforcement

2. Selectors – What to monitor and filter:

• Process execution with arguments
• File system operations (read, write, delete)
• Network connections (TCP/UDP)
• Capability changes
• Namespace operations

3. Actions – How to respond:

• Log events to JSON/gRPC streams
• Send signals (SIGKILL, SIGTERM)
• Override system call return values
• Block operations at kernel level

Kubernetes Identity Awareness

Unlike traditional security tools, Tetragon is deeply Kubernetes-aware. It automatically enriches events with:

• Pod names and namespaces
• Container IDs and image names
• Kubernetes labels and annotations
• Service account information
• Node metadata

This context-rich observability, as explained in InfoWorld’s analysis, allows security teams to correlate process-level activities with network flows, showing exactly which processes within containers are establishing connections or attempting suspicious activities.

Getting Started with Tetragon

Installation on Docker

For Docker environments, deploying Tetragon is straightforward. Run it as a privileged container with host PID namespace access:

docker run --name tetragon \
  --rm -d \
  --pid=host \
  --cgroupns=host \
  --privileged \
  -v /sys/kernel/btf/vmlinux:/var/lib/tetragon/btf \
  quay.io/cilium/tetragon:v1.6.0

Source: Tetragon Official Documentation

Installation on Kubernetes

For Kubernetes clusters, use Helm to deploy Tetragon as a DaemonSet:

# Add the Cilium Helm repository
helm repo add cilium https://helm.cilium.io
helm repo update

# Install Tetragon
helm install tetragon cilium/tetragon \
  -n kube-system \
  --create-namespace

Verify the installation:

kubectl get pods -n kube-system --selector app.kubernetes.io/instance=tetragon

Source: Kubernetes Goat Tutorial

Using the Tetra CLI

Install the tetra CLI tool for easier event monitoring:

# Download and install tetra
GOOS=$(go env GOOS)
GOARCH=$(go env GOARCH)
curl -L https://github.com/cilium/tetragon/releases/latest/download/tetra-${GOOS}-${GOARCH}.tar.gz | tar -xz
sudo mv tetra /usr/local/bin

Monitor events in real-time with compact formatting:

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon \
  -c export-stdout -f | tetra getevents -o compact

Tracing Policies: The Heart of Tetragon

Understanding Tracing Policies

Tracing Policies are Kubernetes Custom Resources (CRs) that define what events to monitor and how to respond. They consist of:

• Hook specification: Where in the kernel to attach
• Argument extraction: What data to capture
• Filtering rules: Which events to report
• Actions: How to respond to matches

Example 1: Monitoring Network Connections

This policy tracks all TCP connections except localhost traffic:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "monitor-network"
spec:
  kprobes:
  - call: "tcp_connect"
    syscall: false
    args:
    - index: 0
      type: "sock"
    selectors:
    - matchArgs:
      - index: 0
        operator: "NotDAddr"
        values:
        - "127.0.0.1"

Apply the policy:

kubectl apply -f monitor-network.yaml

Source: iximiuz Labs Tutorial

Example 2: Detecting Privilege Escalation

Monitor creation of user namespaces by unprivileged processes—a common technique in 44% of privilege escalation exploits:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "detect-privilege-escalation"
spec:
  kprobes:
  - call: "create_user_ns"
    syscall: false
    return: false
    args:
    - index: 0
      type: "nop"
    selectors:
    - matchCapabilities:
      - type: Effective
        operator: NotIn
        values:
        - "CAP_SYS_ADMIN"

Source: Tetragon Policy Library

Example 3: Preventing Container Escapes

Block attempts to mount file systems from containers—a common container escape technique:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "prevent-mount-escape"
spec:
  kprobes:
  - call: "security_sb_mount"
    syscall: false
    return: false
    args:
    - index: 0
      type: "string"
    - index: 1
      type: "path"
    - index: 2
      type: "string"
    returnArg:
      index: 0
      type: "int"
    returnArgAction: Post
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "tmpfs"
      matchActions:
      - action: Sigkill

This policy uses matchActions: Sigkill to immediately terminate processes attempting to mount file systems—enforcing security at the kernel level before the mount operation completes.

Example 4: Cryptomining Detection

Detect cryptocurrency mining activities by monitoring for specific process patterns and network connections:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "detect-cryptomining"
spec:
  kprobes:
  - call: "tcp_connect"
    syscall: false
    args:
    - index: 0
      type: "sock"
    selectors:
    - matchArgs:
      - index: 0
        operator: "DAddr"
        values:
        # Known mining pool addresses
        - "pool.supportxmr.com"
        - "xmr-eu1.nanopool.org"

Source: Medium: Cilium Tetragon Next-Generation Runtime Security

Enforcement Capabilities: Preventing Threats in Real-Time

Synchronous vs. Asynchronous Enforcement

Traditional security tools operate asynchronously: they observe events, send them to user space, analyze them, and then attempt to respond. This creates a dangerous gap where exploits can succeed before enforcement kicks in.

Tetragon provides synchronous enforcement through eBPF. As documented in Tetragon’s Policy Enforcement guide, actions occur directly in the kernel:

• Before system calls complete: Block operations before they take effect
• No TOCTOU vulnerabilities: No gap between check and use
• Immediate termination: SIGKILL processes at the moment of violation

Network Egress Enforcement

Restrict container network access to only known-good destinations. First, capture your cluster’s network CIDRs:

# For GKE
export PODCIDR=$(gcloud container clusters describe ${NAME} --zone ${ZONE} | awk '/clusterIpv4Cidr/ { print $2; }')
export SERVICECIDR=$(gcloud container clusters describe ${NAME} --zone ${ZONE} | awk '/servicesIpv4CidrBlock/ { print $2; }')

# For EKS
export PODCIDR=$(aws eks describe-cluster --name ${NAME} | jq -r '.cluster.kubernetesNetworkConfig.podIpv4Cidr')
export SERVICECIDR=$(aws eks describe-cluster --name ${NAME} | jq -r '.cluster.kubernetesNetworkConfig.serviceIpv4Cidr')

# For kind
export PODCIDR=$(kubectl get nodes -o jsonpath='{.items[0].spec.podCIDR}')
export SERVICECIDR=$(kubectl describe pod -n kube-system kube-apiserver-kind-control-plane | awk -F= '/--service-cluster-ip-range/ {print $2; }')

Apply enforcement policy:

wget https://raw.githubusercontent.com/cilium/tetragon/main/examples/quickstart/network_egress_cluster_enforce.yaml
envsubst < network_egress_cluster_enforce.yaml | kubectl apply -n default -f -

This policy allows connections only to:

• Other pods in the cluster (PODCIDR)
• Kubernetes services (SERVICECIDR)
• DNS (TCP/UDP port 53)

All other egress attempts result in SIGKILL.

File Access Enforcement

Prevent unauthorized access to sensitive files like /etc/shadow:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "protect-shadow-file"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    return: false
    args:
    - index: 0
      type: "file"
    - index: 1
      type: "int"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/etc/shadow"
      matchActions:
      - action: Sigkill

Test the enforcement:

kubectl exec -it pod/xwing -- bash -c "cat /etc/shadow"

You’ll see:

🚀 process default/xwing /bin/bash -c "cat /etc/shadow"
🚀 process default/xwing /bin/cat /etc/shadow
📚 read   default/xwing /bin/cat /etc/shadow
💥 exit   default/xwing /bin/cat /etc/shadow SIGKILL

Source: Tetragon Policy Enforcement Documentation

Integration with Security Ecosystems

SIEM and Observability Platforms

Tetragon exports events in JSON format over multiple channels, making integration straightforward:

# Export to file
kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon \
  -c export-stdout > /var/log/tetragon/events.json

# Stream to Elasticsearch
kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon \
  -c export-stdout -f | fluentd -c fluent.conf

# Export to Prometheus
# Tetragon exposes metrics at :2112/metrics

Supported integrations include:

• Prometheus/Grafana: Time-series metrics and dashboards
• Elasticsearch/Splunk: SIEM correlation and analysis
• OpenTelemetry: Distributed tracing context
• Fluentd/Fluent Bit: Log aggregation and forwarding

Source: CoreWeave Security Tutorial

Cilium Service Mesh Integration

When deployed alongside Cilium, Tetragon gains enhanced network visibility. The combination provides:

• L3-L7 network policy enforcement via Cilium
• Process-level attribution via Tetragon
• Service mesh observability through Hubble
• API-aware security for HTTP, gRPC, Kafka

This unified approach, as described in Isovalent’s blog, allows security teams to trace network connections back to the specific binaries and parent processes that spawned them.

Pre-Built Policy Library

Tetragon maintains a curated policy library for common security scenarios:

Process Execution Monitoring

• Execution of deleted binaries: Detect malware that deletes itself after loading
• File capabilities execution: Monitor privileged binaries
• Kernel module loading: Track loadable kernel modules (LKM)

Privilege and Capability Monitoring

• Privilege escalation via setuid: Detect unauthorized privilege changes
• Capability changes: Monitor CAP_SYS_ADMIN and other dangerous capabilities
• User namespace creation: Identify potential container escape attempts

File System Monitoring

• Sensitive file access: Monitor /etc/shadow, /etc/passwd, SSH keys
• Credential access: Detect reads of cloud provider credentials
• Configuration changes: Track modifications to critical configs

Network Monitoring

• Outbound connections: Monitor egress to external IPs
• Port scanning: Detect reconnaissance activities
• TLS bypass: Identify unencrypted connections to sensitive services

Performance Characteristics

Overhead Measurements

According to multiple sources, Tetragon’s eBPF-based architecture delivers:

• < 1% CPU overhead for typical workloads
• Minimal memory footprint: ~50-100MB per node
• No kernel module compilation: Works on any kernel with eBPF support (Linux 4.19+)
• In-kernel filtering: Reduces events sent to user space by 90%+

Scalability

Tetragon scales efficiently in large environments:

• Tested to 10,000+ nodes in production Kubernetes clusters
• Millions of events per second per node with in-kernel aggregation
• Selective monitoring: Target specific namespaces or pod labels
• DPU acceleration: Enhanced performance on SmartNIC-enabled clusters

Source: CoreWeave DPU Integration

Advanced Use Cases

Supply Chain Security

Monitor the entire lifecycle of software execution:

1. Image scanning – Detect vulnerabilities in container images
2. Admission control – Validate signatures and policies
3. Runtime monitoring – Track actual process behavior via Tetragon
4. Drift detection – Alert on execution not matching image manifest

Zero Trust Architecture

Implement zero trust principles with Tetragon:

• Default deny – Block all network egress except explicit allows
• Least privilege – Restrict file and capability access
• Verify always – Continuously validate runtime behavior
• Assume breach – Detect lateral movement and privilege escalation

Compliance and Audit

Tetragon provides tamper-resistant audit trails:

• Immutable logs – Kernel-level event capture
• Complete lineage – Parent-child process relationships
• Contextual metadata – Kubernetes identity and labels
• Cryptographic integrity – Optional log signing

These capabilities help satisfy requirements for:

• PCI-DSS: Payment card data protection
• HIPAA: Healthcare data security
• SOC 2: Security controls auditing
• FedRAMP: Federal cloud security

Comparing Tetragon to Alternatives

vs. Falco

Falco pioneered eBPF-based runtime security but focuses primarily on detection. Key differences:

Feature	Tetragon	Falco
Enforcement	Synchronous (in-kernel)	Asynchronous (user-space)
Kubernetes-native	Yes	Via Sidekick
Network correlation	Built-in	Limited
Hook types	kprobe, tracepoint, uprobe, LSM	kprobe, tracepoint
Policy language	YAML CRD	YAML + custom syntax

vs. Sysdig/Aqua Security

Commercial platforms like Sysdig and Aqua provide comprehensive security suites but:

• Closed source vs. Tetragon’s Apache 2.0 license
• Agent overhead typically 2-5% vs. Tetragon’s <1%
• Vendor lock-in vs. open CNCF ecosystem
• Cost $$$$ vs. free

Tetragon excels as a focused, open-source runtime security component that integrates into broader security architectures.

Best Practices and Recommendations

Start in Detection Mode

Begin with observability before enabling enforcement:

# Omit matchActions initially
selectors:
- matchArgs:
  - index: 0
    operator: "Equal"
    values:
    - "/sensitive/path"
  # No matchActions = observe only

Analyze baseline behavior for 1-2 weeks, then gradually add enforcement.

Layer Your Policies

Structure policies hierarchically:

1. Cluster-wide defaults – Broad protections (prevent cryptomining)
2. Namespace-specific – Application-aware rules
3. Pod-targeted – Granular controls for sensitive workloads

Use TracingPolicyNamespaced for namespace-scoped policies:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: production-enforcement
  namespace: production
spec:
  # Policy applies only to production namespace

Monitor Performance

Track Tetragon’s resource usage:

# Check memory usage
kubectl top pods -n kube-system -l app.kubernetes.io/name=tetragon

# Review metrics
kubectl port-forward -n kube-system svc/tetragon 2112:2112
curl localhost:2112/metrics | grep tetragon

Integrate with CI/CD

Include Tetragon policies in your deployment pipeline:

# .github/workflows/deploy.yml
- name: Validate Tetragon policies
  run: |
    kubectl apply --dry-run=server -f policies/
    
- name: Deploy with policy
  run: |
    kubectl apply -f policies/
    kubectl rollout restart deployment/$APP_NAME

The Future of Runtime Security

Emerging Trends

As noted in the eBPF Ecosystem Progress Report 2024-2025:

• AI/ML threat detection – Training models on Tetragon event streams
• Cross-cluster correlation – Multi-cluster threat hunting
• Hardware acceleration – DPU-based eBPF processing
• Wasm integration – WebAssembly policies for portability

Tetragon Roadmap

The Cilium community continues enhancing Tetragon with:

• Enhanced LSM hooks – More enforcement points
• User-space crypto verification – JIT signature checking
• Cloud provider integrations – AWS GuardDuty, Azure Sentinel
• Simplified policy authoring – Higher-level DSL

Conclusion: Building Secure-by-Default Infrastructure

Cilium Tetragon represents a paradigm shift in runtime security. By leveraging eBPF to observe and enforce security policies directly in the Linux kernel, it eliminates the TOCTOU vulnerabilities that plague traditional security tools while maintaining minimal performance overhead.

For Docker Captains, SREs, and platform engineers, Tetragon offers:

✅ Kernel-level visibility into process execution, file access, and network activity
✅ Real-time enforcement that blocks threats synchronously
✅ Kubernetes-native design with rich identity context
✅ Open source under Apache 2.0 with strong CNCF backing
✅ Production-ready performance at massive scale

Whether you’re defending against supply chain attacks like XZ Utils, preventing container escapes, or building zero-trust architectures, Tetragon provides the deep observability and enforcement capabilities modern cloud-native security demands.

Getting Started Resources

Official Documentation

• Tetragon Homepage
• Getting Started Guide
• Tracing Policy Reference
• Policy Library