Kubernetes Developer Portal Setup: Get Started

Backstage on Kubernetes: Developer Portal Setup

In today’s fast-paced software development landscape, managing a growing number of microservices, libraries, and infrastructure components can quickly become a monumental challenge. Developers often spend valuable time searching for documentation, understanding service ownership, or navigating complex deployment pipelines. This fragmentation leads to reduced productivity, increased onboarding time for new team members, and a general loss of tribal knowledge. What if there was a single pane of glass, a central hub where all development-related information, tools, and services could be easily discovered and managed?

Enter Backstage, an open-source developer portal created by Spotify and now a CNCF graduated project. Backstage isn’t just a dashboard; it’s an extensible platform designed to standardize and streamline your development ecosystem. It provides a unified user experience for services, documentation, software templates, and more, making it easier for developers to build, deploy, and operate software. Deploying Backstage on Kubernetes offers the best of both worlds: the flexibility, scalability, and resilience of container orchestration combined with a powerful, opinionated developer experience platform. This guide will walk you through the process of setting up Backstage on your Kubernetes cluster, transforming your development workflow.

Prerequisites

Before we dive into the deployment, ensure you have the following tools and knowledge:

• Kubernetes Cluster: A running Kubernetes cluster (e.g., Minikube, Kind, GKE, EKS, AKS). For production, a multi-node cluster is recommended.
• kubectl: The Kubernetes command-line tool configured to connect to your cluster.
• helm: The Kubernetes package manager, useful for deploying dependencies like PostgreSQL.
• docker: Docker Desktop or Docker Engine installed for building container images.
• node.js and yarn: Node.js (LTS version) and Yarn are required to create and build the Backstage application.
• Basic Kubernetes Knowledge: Familiarity with Deployments, Services, Persistent Volumes, and Namespaces.
• Container Registry: Access to a container registry (e.g., Docker Hub, Google Container Registry, AWS ECR) if you’re not using a local registry or Minikube’s built-in daemon.

Step-by-Step Guide

1. Initialize Your Backstage Application

The first step is to create a new Backstage application. This generates a boilerplate project with the necessary structure and configuration files. We’ll use the Backstage CLI tool for this, which streamlines the setup process.

This command will prompt you for a project name and then scaffold out a complete Backstage application, including a frontend, a backend, and a database. It sets up a monorepo structure, which is common for Backstage projects, allowing you to manage all components in a single repository. The generated project will include Dockerfiles, Kubernetes manifests (though we’ll customize them), and configuration files.

npx @backstage/create-app@latest

# Follow the prompts:
# Project name: backstage-kube
# Would you like to use a database? (y/N) y
# Which database would you like to use? (PostgreSQL/SQLite) PostgreSQL

Once the command completes, navigate into your new project directory:

cd backstage-kube
yarn install

Verify: You should see a new directory named backstage-kube with a typical monorepo structure containing packages/backend and packages/frontend.

ls -F

app-config.yaml
app-config.production.yaml
backstage.json
catalog-info.yaml
lerna.json
package.json
packages/
plugins/
README.md
yarn.lock

2. Configure Backstage for Kubernetes Deployment

Before building our Docker images, we need to adjust the Backstage configuration to work within a Kubernetes environment, particularly regarding database connectivity and external URLs. We’ll modify app-config.production.yaml for production settings.

The app-config.production.yaml file dictates how Backstage behaves in a production environment. We need to tell the backend how to connect to our PostgreSQL database, which will be deployed as a separate service in Kubernetes. We’ll also set the baseUrl for the frontend to reflect how users will access Backstage, typically through an Ingress or LoadBalancer. For database, we’ll use environment variables for sensitive credentials, a best practice in Kubernetes via Secrets. This configuration ensures that Backstage components can discover each other and external services correctly within the cluster.

Open app-config.production.yaml and modify the backend and database sections. Replace your-backstage-domain.com with your actual domain or IP if you’re exposing it directly.

# app-config.production.yaml
app:
  baseUrl: https://your-backstage-domain.com # Or your LoadBalancer IP/hostname

backend:
  baseUrl: https://your-backstage-domain.com/api # Matches app.baseUrl
  listen:
    port: 7007
  csp:
    # Content-Security-Policy production settings.
    # Set to 'false' for development to avoid issues with hot reloading.
    # For a production setup, you should customize this to your needs.
    # See https://backstage.io/docs/tutorials/content-security-policy
    enable: true
  database:
    client: pg
    connection:
      host: postgres-service # Kubernetes service name for PostgreSQL
      port: 5432
      user: ${POSTGRES_USER}
      password: ${POSTGRES_PASSWORD}
      database: ${POSTGRES_DB}
  # ... other backend configurations

techdocs:
  builder: 'external' # Recommended for production to offload TechDocs builds
  generator:
    runIn: 'local'
  publisher:
    type: 'local' # For now, we'll keep it local, but consider S3/GCS for production

Verify: Ensure the host under backend.database.connection is set to postgres-service (or whatever you name your PostgreSQL service in Kubernetes) and that baseUrl is updated.

3. Create Docker Images for Backstage Backend and Frontend

Backstage components run as separate services. We need to containerize the backend and frontend applications. The create-app command generates default Dockerfiles for the backend. We’ll use these and build our images.

Building Docker images is a standard practice for deploying applications to Kubernetes. The generated Dockerfiles are optimized for Backstage applications, ensuring all dependencies are packaged correctly. For the backend, we’re using a multi-stage build to keep the final image size minimal. The frontend is typically served by a web server like Nginx, but for simplicity in this guide, we’ll focus on the backend image. You can later extend this to build a dedicated frontend image or use a simple web server to serve the static frontend assets generated by yarn build.

First, build the backend image. Replace localhost:5000 with your registry if you’re using one (e.g., myregistry.com/).

# Build the backend image
docker build . -f packages/backend/Dockerfile --tag localhost:5000/backstage-backend:latest

If you’re using a remote registry, push the image:

docker push localhost:5000/backstage-backend:latest

For the frontend, we’ll build it and assume it’s served by the backend or a simple web server. For a full production setup, you’d typically build the frontend separately and serve it with Nginx or similar, or even a CDN. For this guide, the backend will serve the frontend assets.

yarn build:all

Verify: Check that your Docker image is listed:

docker images | grep backstage-backend

localhost:5000/backstage-backend   latest    a1b2c3d4e5f6   2 minutes ago   500MB

4. Deploy PostgreSQL Database

Backstage requires a PostgreSQL database to store its catalog, user data, and other information. We’ll deploy PostgreSQL within our Kubernetes cluster using a Deployment, Service, and Persistent Volume Claim.

Deploying a stateful application like PostgreSQL in Kubernetes requires careful consideration of data persistence. We use a Persistent Volume Claim (PVC) to ensure that our database data survives pod restarts and rescheduling. The Deployment manages the PostgreSQL pod, and the Service provides a stable network endpoint for Backstage to connect to, abstracting away the pod’s IP address. For production, consider using a StatefulSet and a managed database service from your cloud provider (e.g., AWS RDS, GCP Cloud SQL) for better reliability and operational ease, or a PostgreSQL operator.

Create a file named postgres-deployment.yaml:

# postgres-deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: backstage
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pv-claim
  namespace: backstage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi # Adjust storage as needed
---
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secrets
  namespace: backstage
type: Opaque
stringData:
  POSTGRES_USER: backstage
  POSTGRES_PASSWORD: changeme # IMPORTANT: Change this to a strong password in production!
  POSTGRES_DB: backstage
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-deployment
  namespace: backstage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13 # Use a specific version for stability
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_USER
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: POSTGRES_USER
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: POSTGRES_PASSWORD
        - name: POSTGRES_DB
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: POSTGRES_DB
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pv-claim
---
apiVersion: v1
kind: Service
metadata:
  name: postgres-service
  namespace: backstage
spec:
  selector:
    app: postgres
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  type: ClusterIP

Apply these manifests:

kubectl apply -f postgres-deployment.yaml

Verify: Check the status of the PostgreSQL deployment and service.

kubectl get all -n backstage

NAME                                    READY   STATUS    RESTARTS   AGE
pod/postgres-deployment-7b8c9d...       1/1     Running   0          2m

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/postgres-service       ClusterIP   10.96.123.45             5432/TCP   2m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/postgres-deployment   1/1     1            1           2m

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/postgres-deployment-...   1         1         1       2m

NAME                                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/postgres-pv-claim            Bound    pvc-a1b2c3d4-e5f6-7890-1234-567890abcdef   5Gi        RWO            standard       2m

5. Deploy Backstage Backend and Frontend

Now that the database is ready, we can deploy the Backstage application itself. We’ll use a Kubernetes Deployment for the backend and a Service to expose it. For the frontend, as mentioned, the backend will serve the assets, simplifying our initial deployment.

The Backstage backend Deployment will pull the Docker image we built earlier. It will use the Secret we created for PostgreSQL credentials, injecting them as environment variables. The Service will expose the backend internally within the cluster. To access Backstage from outside the cluster, we’ll need an Ingress (or a LoadBalancer service type), which we’ll cover next. For advanced traffic management, consider integrating with an Istio Ambient Mesh or Kubernetes Gateway API setup.

Create a file named backstage-deployment.yaml:

# backstage-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backstage-deployment
  namespace: backstage
  labels:
    app: backstage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backstage
  template:
    metadata:
      labels:
        app: backstage
    spec:
      containers:
      - name: backstage
        image: localhost:5000/backstage-backend:latest # Replace with your registry if applicable
        env:
          - name: APP_CONFIG_app_baseUrl
            value: https://your-backstage-domain.com # Must match app-config.production.yaml
          - name: APP_CONFIG_backend_baseUrl
            value: https://your-backstage-domain.com/api # Must match app-config.production.yaml
          - name: APP_CONFIG_backend_database_connection_host
            value: postgres-service # The name of our PostgreSQL Service
          - name: APP_CONFIG_backend_database_connection_port
            value: "5432"
          - name: APP_CONFIG_backend_database_connection_user
            valueFrom:
              secretKeyRef:
                name: postgres-secrets
                key: POSTGRES_USER
          - name: APP_CONFIG_backend_database_connection_password
            valueFrom:
              secretKeyRef:
                name: postgres-secrets
                key: POSTGRES_PASSWORD
          - name: APP_CONFIG_backend_database_connection_database
            valueFrom:
              secretKeyRef:
                name: postgres-secrets
                key: POSTGRES_DB
          - name: NODE_ENV
            value: production
        ports:
        - containerPort: 7007
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 7007
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 7007
          initialDelaySeconds: 30
          periodSeconds: 10
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
---
apiVersion: v1
kind: Service
metadata:
  name: backstage-service
  namespace: backstage
spec:
  selector:
    app: backstage
  ports:
    - protocol: TCP
      port: 80
      targetPort: 7007
  type: ClusterIP # Use ClusterIP for internal access, or LoadBalancer for direct external access

Apply these manifests:

kubectl apply -f backstage-deployment.yaml

Verify: Check the status of the Backstage deployment and service.

kubectl get all -n backstage

NAME                                    READY   STATUS    RESTARTS   AGE
pod/backstage-deployment-85f6c7...      1/1     Running   0          1m
pod/postgres-deployment-7b8c9d...       1/1     Running   0          10m

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/backstage-service      ClusterIP   10.96.67.89              80/TCP     1m
service/postgres-service       ClusterIP   10.96.123.45             5432/TCP   10m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/backstage-deployment   1/1     1            1           1m
deployment.apps/postgres-deployment   1/1     1            1           10m

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/backstage-deployment-...   1         1         1       1m
replicaset.apps/postgres-deployment-...   1         1         1       10m

6. Expose Backstage Using Ingress (or Port-Forward)

To access your Backstage instance from your browser, you’ll need to expose the backstage-service externally. We’ll use a Kubernetes Ingress for this, which requires an Ingress Controller (e.g., Nginx Ingress Controller, Traefik). If you don’t have one, or for quick testing, you can use kubectl port-forward.

An Ingress resource allows you to manage external access to services in a cluster, typically HTTP/S. It provides URL-based routing, SSL/TLS termination, and name-based virtual hosting. This is the recommended way to expose applications like Backstage in a production environment. For more advanced networking, especially in multi-cluster or multi-tenant environments, consider using Kubernetes Network Policies or a service mesh like Istio.

Option A: Using kubectl port-forward (for local testing)

kubectl port-forward service/backstage-service 7007:80 -n backstage

Then, open your browser to http://localhost:7007.

Option B: Using Ingress (recommended for production)

First, ensure you have an Ingress Controller installed in your cluster. For example, to install Nginx Ingress Controller:

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

Now, create an Ingress resource (backstage-ingress.yaml). Remember to replace your-backstage-domain.com with your actual domain and ensure it points to your Ingress Controller’s external IP.

# backstage-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: backstage-ingress
  namespace: backstage
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP" # Backstage backend uses HTTP
    # If you're using cert-manager for TLS, uncomment these:
    # cert-manager.io/cluster-issuer: "letsencrypt-prod"
    # nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  ingressClassName: nginx # Or your specific Ingress Controller class
  rules:
  - host: your-backstage-domain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: backstage-service
            port:
              number: 80
  # If using TLS, uncomment this section:
  # tls:
  # - hosts:
  #   - your-backstage-domain.com
  #   secretName: backstage-tls-secret # Cert-manager will create this

Apply the Ingress manifest:

kubectl apply -f backstage-ingress.yaml

Verify: Get the Ingress external IP/hostname:

kubectl get ingress -n backstage

NAME                CLASS    HOSTS                      ADDRESS         PORTS     AGE
backstage-ingress   nginx    your-backstage-domain.com  192.168.1.100   80, 443   1m

Update your DNS A record for your-backstage-domain.com to point to the ADDRESS provided by the Ingress. Then, navigate to https://your-backstage-domain.com in your browser.

Production Considerations

Deploying Backstage for production requires more robust configurations than our basic setup:

• High Availability: Run multiple replicas of the Backstage backend deployment. Consider a Karpenter autoscaling setup for efficient node management.
• Database: Use a managed cloud PostgreSQL service (e.g., AWS RDS, GCP Cloud SQL) or a highly available PostgreSQL cluster (e.g., using a PostgreSQL operator) instead of a single in-cluster deployment with a PVC.
• Secrets Management: Store sensitive information like database passwords and API tokens in a dedicated secrets management solution like Kubernetes Secrets (as we did), Secrets Store CSI Driver, HashiCorp Vault, or cloud provider secret managers.
• Ingress/Gateway: Always use an Ingress Controller or Kubernetes Gateway API with TLS termination for secure external access. Integrate with cert-manager for automated certificate provisioning.
• Observability: Implement robust monitoring (Prometheus/Grafana), logging (Fluentd/Elasticsearch/Kibana or Loki), and tracing (Jaeger/OpenTelemetry) for Backstage components. Explore eBPF Observability with Hubble for network insights.
• Security: Apply Kubernetes Network Policies to restrict traffic between Backstage components and other services. Regularly scan container images for vulnerabilities. Consider integrating with tools like Sigstore and Kyverno for supply chain security.
• TechDocs Storage: For production, configure TechDocs to use external storage like S3 or GCS instead of local storage, especially with multiple replicas.
• Authentication: Implement appropriate authentication providers (e.g., GitHub, Google, Okta, LDAP) as per your organization’s requirements.
• Resource Limits: Set appropriate CPU and memory requests/limits for Backstage pods to prevent resource exhaustion and ensure stable performance.

Troubleshooting

1. Backstage Pod Stuck in Pending State:

   Issue: The Backstage pod isn’t scheduling or starting.

   Solution: Check pod events and node resources. This often indicates insufficient resources (CPU/memory) on your nodes or issues with pulling the Docker image.

   kubectl describe pod <backstage-pod-name> -n backstage
   kubectl get nodes -o wide
   

   Ensure your nodes have enough allocatable resources. If it’s an image pull issue, verify the image name and registry accessibility.

2. Backstage Pod in CrashLoopBackOff:

   Issue: The Backstage application frequently crashes and restarts.

   Solution: Examine the pod logs. This usually points to application-level errors, often configuration mistakes or database connectivity issues.

   kubectl logs <backstage-pod-name> -n backstage
   

   Look for errors related to database connection, missing environment variables, or incorrect app-config.production.yaml settings.

3. Cannot Connect to PostgreSQL:

   Issue: Backstage logs show “database connection refused” or similar errors.

   Solution: Verify the PostgreSQL service name, port, and credentials. Ensure the PostgreSQL pod is running and healthy.

   kubectl get pods -n backstage -l app=postgres
   kubectl get service -n backstage postgres-service
   kubectl logs <postgres-pod-name> -n backstage
   

   Double-check the APP_CONFIG_backend_database_connection_host value in your Backstage deployment to match the PostgreSQL service name.

4. Backstage Frontend Not Loading/Blank Page:

   Issue: You can access the Backstage URL, but the page is blank or shows loading errors.

   Solution: Check your browser’s developer console for JavaScript errors or failed network requests. This often indicates an incorrect app.baseUrl or backend.baseUrl in app-config.production.yaml, or issues with static asset serving.

   Ensure baseUrl matches the external URL you’re using to access Backstage. If using an Ingress, verify its configuration and that traffic is correctly routed to the backend service.

5. Ingress Not Working / 404 Errors:

   Issue: Your domain doesn’t resolve to Backstage, or you get 404 errors.

   Solution: Verify your Ingress Controller is running, the Ingress resource is correctly configured, and your DNS points to the Ingress Controller’s external IP.

   kubectl get ingress -n backstage
   kubectl get pods -n ingress-nginx # Or your Ingress Controller's namespace
   

   Check Ingress Controller logs for routing issues. Ensure the host in your Ingress matches the baseUrl in Backstage config.

6. Permissions Issues Accessing Catalog/Plugins:

   Issue: Backstage UI loads, but you can’t see catalog items or specific plugins fail.

   Solution: This could be related to RBAC within Backstage itself, or external API permissions. For initial setup, ensure your default user (if any) has sufficient permissions. If fetching data from external sources (e.g., GitHub), verify the backend’s environment variables for API tokens.

   Review your app-config.production.yaml for any custom auth or catalog configurations. Check backend logs for errors related to external service communication.

FAQ Section

1. What is Backstage and why should I use it?

   Backstage is an open-source platform for building developer portals. It provides a unified view for all your software, documentation, and tools, helping to solve the “microservice sprawl” problem. It improves developer experience, reduces onboarding time, and fosters a culture of inner sourcing by making internal tools and services easily discoverable and manageable.

2. Why deploy Backstage on Kubernetes?

   Deploying Backstage on Kubernetes provides scalability, resilience, and efficient resource utilization. Kubernetes handles the orchestration, self-healing, and scaling of Backstage components, allowing you to focus on developing features rather than managing infrastructure. It also integrates seamlessly with other cloud-native tools and practices.

3. How do I add new plugins to Backstage?

   Adding plugins typically involves installing them as Yarn workspaces in your Backstage project (yarn add --cwd packages/app @backstage/plugin-my-plugin), then integrating them into your frontend (App.tsx) and potentially backend (backend.ts) code. After making changes, you’ll need to rebuild your Docker images and redeploy your Backstage application to Kubernetes.

4. Can I use a different database than PostgreSQL?

   Yes, Backstage supports other databases. While PostgreSQL is a common choice and used in this guide, Backstage can also be configured to work with SQLite (for development) and other SQL databases via Knex.js. Just ensure your chosen database is properly configured and accessible from your Backstage backend.

5. How can I manage user authentication for Backstage?

   Backstage offers a flexible authentication system with various providers. You can configure authentication through identity providers like GitHub, Google, Okta, Auth0, Microsoft Entra ID (formerly Azure AD), and more. These are configured in your app-config.yaml and typically involve setting up OAuth applications with your chosen provider and providing credentials as environment variables or Kubernetes Secrets.

Cleanup Commands

To remove all resources created by this guide, execute the following commands:

kubectl delete -f backstage-ingress.yaml -n backstage

kubectl delete -f backstage-deployment.yaml -n backstage

kubectl delete -f postgres-deployment.yaml -n backstage

kubectl delete namespace backstage