Tiexin Guo's Blog

Building Your Developer Portal with Backstage

Building Your Developer Portal with Backstage

A Comprehensive Tutorial

Tiexin Guo's photo
Tiexin Guo
·

12 min read

Table of contents

0 Background

"DevOps is dead; long live platform engineering."

"Developer portal."

"Backstage."

In 2023, we started seeing these terms more often in DevOps. While I wouldn't go so far as to say that DevOps is dead, Backstage is undoubtedly becoming more popular.

If you haven't heard of Backstage yet, you can give it a quick look here. Here's my brief summary for those TL;DR guys:

  • Backstage isn't a "developer portal" but a tool to build your developer portal. Think of "create-react-app" V.S., the actual react app you are creating with it.

  • Backstage has a React frontend and a Node.js backend.

  • Backstage extends its capabilities by plugins, and installing plugins requires changing some simple TSX code, hence a bit challenging to use.

  • Some plugin documentation isn't detailed enough, which increases the difficulty if you want to build many features around it.

Enough said; today, we will start from scratch and build a developer portal ourselves. After this tutorial, you can scaffold a new repository using templates, check the CI/CD status, see the deployment status, and view the documentation in the same place. And there are more features.

Without further adieu, let's get started.

1 Prerequisites

  • A Unix-based operating system. For example, you can run this on macOS, Linux, or Windows Subsystem for Linux (WSL).

  • curl, git, Docker

  • Node.js, yarn

For a DevOps engineer, you probably have already gotten most of them on your laptop, maybe with the exception being Node.js and yarn. So, let's talk about these two slightly more:

It's recommended to use nvm to install Node.js. First, install nvm by running:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

Then we run the following command to install the latest LTS version of Node.js:

nvm install node

Yarn is a dependency management tool for Node.js, which can be installed by running:

npm install --global yarn

2 Create Portal Code

OK, now that the dependencies are cleared out of our way, let's create the code base of our developer portal by running only one command:

$ npx @backstage/create-app@latest
? Enter a name for the app [required] my-developer-portal

Creating the app...
...

This runs the backstage/create-app package directly, which generates the default developer portal code for us, and the generated directory is named "my-developer-portal".

Again, given not all DevOps engineers are familiar with Node.js or TSX, let's do a quick walkthrough of the generated code base:

  • The frontend source code is at packages/app/src/.

  • The main file is packages/pp/src/App.tsx. When installing a plugin, sometimes we need to change the source code, and this file is highly subjective to change.

  • The main software catalog page is at packages/app/src/components/catalog/EntityPage.tsx. If you want to add more tabs and stuff for the detail page of a component, this is the place to look.

  • The backend source code is at packages/backend/src/.

  • Since most features are implemented by plugins, the plugins' code is at packages/backend/src/plugins/. For example, put the backend source code here to create another plugin.

  • The main config file is at app-config.yaml, and the local environment configuration (not for production) is at app-config.local.yaml.

OK, with the knowledge above, let's move on to other dependencies.

3 Create the Database Used by Our Developer Portal

Some configurations and stuff of our developer portal are stored in a Postgres DB, so let's prepare and configure it.

First, let's install Postgres locally using brew:

brew install postgresql@14
brew services start postgresql@14

Create a user used by our portal:

psql postgres
create user backstage with encrypted password 'backstage';
alter role backstage with superuser;

Note that this is only for local development and not for production (which explains why we are making the role a superuser).

Then, let's add the PostgreSQL client to our portal. From the root directory of the folder "my-developer-portal", run:

yarn add --cwd packages/backend pg

Next, let's configure our portal to use this DB we created. In the file app-config.yaml, change the "backend" section to the following:

backend:
  database:
    client: pg
    connection:
      host: 127.0.0.1
      port: 5432
      user: "backstage"
      password: "backstage"

OK, after everything, once we start our portal by running:

yarn dev

Some databases will be created automatically, and our portal will use the DB as the persistent storage.

4 Single Sign On

Next, we want to use GitHub for single sign-on so that there is no need to register and manage users, which is one less overhead for us to worry about.

Go to https://github.com/settings/applications/new to create your OAuth App. The Homepage URL should point to Backstage's frontend; in our tutorial, it would be http://localhost:3000. The Authorization callback URL is http://localhost:7007/api/auth/github/handler/frame.

Generate a client secret, copy the client id/secret, and configure them accordingly in the config file "app-config.yaml" in the "auth" section:

auth:
  environment: development
  providers:
    github:
      development:
        clientId: <YOUR CLIENT ID HERE>
        clientSecret: <YOUR CLIENT SECRET HERE>

Backstage then will read the configuration, and we need to update our frontend so that we can log in via GitHub SSO:

Open packages/app/src/App.tsx, and below the last import line, add:

import { githubAuthApiRef } from "@backstage/core-plugin-api";
import { SignInPage } from "@backstage/core-components";

Search for const app = createApp({ in this file, and below apis, add:

components: {
  SignInPage: props => (
    <SignInPage
      {...props}
      auto
      provider={{
        id: 'github-auth-provider',
        title: 'GitHub',
        message: 'Sign in using GitHub',
        apiRef: githubAuthApiRef,
      }}
    />
  ),
},

OK, now we can log in to our developer portal using GitHub single sign-on!

5 GitHub Integration

The GitHub integration will allow us to load catalog entities from GitHub and create repositories using Backstage templates.

Create a personal access token here, select repo and workflow for scope (enough for this tutorial), copy the token, and configure it in the app-config.local.yaml file:

integrations:
  github:
    - host: github.com
      token: <YOUR GITHUB PERSONAL ACCESS TOKEN HERE>

After this, let's restart our portal by pressing CTRL+C in the terminal where we ran yarn dev previously and run yarn dev again.

6 Mid-Way Test

Before we run some tests to see if our setup is correct, let's configure our portal to allow importing templates using URLs.

In file app-config.yaml, for the "catalog" section, let's add "Template" as part of the "rules":

catalog:
  import:
    entityFilename: catalog-info.yaml
    pullRequestBranchName: backstage-integration
  rules:
    - allow: [Component, System, API, Resource, Location, Template]
  ...

OK, we are all set, and let's test:

Go to http://localhost:3000, and we should be able to log in to our portal using GitHub single sign-on.

Click "Create" -> "Register New", and put the URL: "https://github.com/IronCore864/backstage-k8s-github-template/blob/main/template.yaml" (which is a template I created for this tutorial), and we should be able to import a template to our software catalog.

You can see the template when you click "Create" again. Use the newly added template to create a repository named "my-kubernetes-component" You will have it created in your GitHub account!

See the created sample here: https://github.com/IronCore864/my-kubernetes-component.

The created repository has GitHub Actions as continuous integration, and it should run successfully. Go to the software catalog in the developer portal, and click on the newly created component under the CI/CD tab. You should see the CI results in our developer portal directly, which means our GitHub integration is working all right.

The created repository also contains simple documentation created by mkdocs, and under the Docs tab, you should see it rendered and displayed correctly.

7 Mid-Way Summary

So far, our developer portal has achieved the following functionalities:

  • software catalog: a centralized place for all your software

  • repository/service scaffolding: creating a new repository using templates

  • CI: show the status of the CI workflows

  • Docs: show the documentation of our service

We want to add more value to our portal. Specifically, we want to add continuous deployment using Argo CD and see the deployed pods in Kubernetes clusters.

So, let's do that.

8 Adding Argo CD

8.1 Prepare Our Argo CD

We will use minikube to create a local Kubernetes cluster and install Argo CD:

Install minikube, make sure docker is up and running, then run:

minikube start --driver=docker

This creates a local Kubernetes cluster inside a Docker container.

Then, install Argo CD by running:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

This creates the "argocd" namespace and installs Argo CD there.

In another window, run:

kubectl port-forward svc/argocd-server -n argocd 8080:443

This forwards the Argo CD server to our local 8080 port so that we can access it.

Let's initialize the password for the admin user:

argocd admin initial-password
argocd login localhost:8080
argocd account update-password

Create an Argo CD project role in the "default" project and an API token:

argocd proj role create default backstage
argocd proj role create default backstage
argocd proj role create-token default backstage

Store the token temporarily somewhere, which will be used in the next section for our portal-ArgoCD integration.

Then enable Argo CD role-based access control (RBAC):

kubectl edit configmap argocd-rbac-cm

Add the "data" part using the following content:

data:
  policy.csv: |
    p, role:org-admin, applications, *, */*, allow
    p, role:org-admin, clusters, get, *, allow
    p, role:org-admin, repositories, get, *, allow
    p, role:org-admin, repositories, create, *, allow
    p, role:org-admin, repositories, update, *, allow
    p, role:org-admin, repositories, delete, *, allow
    p, role:backstage, applications, *, */*, allow
  policy.default: role:readonly

This enables RBAC in our Argo CD and grants permission to the newly created role "backstage".

8.2 Argo CD Integration

Install the plugin into Backstage.

cd packages/app
yarn add @roadiehq/backstage-plugin-argo-cd

In the config app-config.yaml, in the section proxy, add the following:

proxy:
  "/argocd/api":
    target: https://localhost:8080/api/v1/
    changeOrigin: true
    secure: false
    headers:
      Cookie:
        $env: ARGOCD_AUTH_TOKEN

Note: copy and paste the above section entirely without changing the last line. Note that the last line (keep ARGOCD_AUTH_TOKEN as it is without pasting the actual token there).

Then, let's add Argo CD to our portal's UI. In file packages/app/src/components/catalog/EntityPage.tsx, add the import:

import {
  EntityArgoCDOverviewCard,
  EntityArgoCDHistoryCard,
  isArgocdAvailable,
} from "@roadiehq/backstage-plugin-argo-cd";

In the same file, search for const overviewContent and update the content as follows:

const overviewContent = (
  <Grid container spacing={3} alignItems="stretch">
    ...
    <EntitySwitch>
      <EntitySwitch.Case if={(e) => Boolean(isArgocdAvailable(e))}>
        <Grid item sm={4}>
          <EntityArgoCDOverviewCard />
        </Grid>
        <Grid item sm={4}>
          <EntityArgoCDHistoryCard />
        </Grid>
      </EntitySwitch.Case>
    </EntitySwitch>
    ...
  </Grid>
);

If you read some TSX code, you will know that this adds two cards in the overview tab.

Stop our portal, export ARGOCD_AUTH_TOKEN="argocd.token=YOUR_ARGOCD_ROLE_API_TOKEN_HERE, then restart the portal.

8.3 Deploy an App and Test the Result

Now that Argo CD integration is done, let's test it by deploying some pods which belong to the "my-kubernetes-component" we created earlier in section 6.

You can do this via Argo CD UI, but to make it easier to reproduce the exact same result, prepare a file application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-kubernetes-component
  namespace: argocd
spec:
  destination:
    namespace: default
    server: https://kubernetes.default.svc
  project: default
  source:
    helm:
      valueFiles:
        - values.yaml
    path: go-hello-http
    repoURL: https://github.com/IronCore864/gitops-argocd
    targetRevision: HEAD
  syncPolicy:
    automated: {}

And apply it:

kubectl apply -f application.yaml

This deploys a hello-world pod linked to our "my-kubernetes-component" using annotations.

If we go to our component in our portal, under the overview tab, we shall see the latest Argo CD deploy status and Argo CD history.

9 Kubernetes Integration

Now that we can see the continuous deployment's status, how about the actual resources of the deployment? To have that in our portal, we need to integrate Kubernetes.

9.1 Add Kubernetes Plugins

First, let's add the Kubernetes frontend/backend plugins:

# from your my-developer-portal root directory
yarn add --cwd packages/app @backstage/plugin-kubernetes
yarn add --cwd packages/backend @backstage/plugin-kubernetes-backend

In packages/app/src/components/catalog/EntityPage.tsx, add the following code:

import { EntityKubernetesContent } from '@backstage/plugin-kubernetes';

// You can add the tab to any number of pages, the service page is shown as an
// example here
const serviceEntityPage = (
  <EntityLayout>
    {/* other tabs... */}
    <EntityLayout.Route path="/kubernetes" title="Kubernetes">
      <EntityKubernetesContent refreshIntervalMs={30000} />
    </EntityLayout.Route>

Create a file called kubernetes.ts inside packages/backend/src/plugins/ and add the following (code for the Kubernetes plugin):

import { KubernetesBuilder } from '@backstage/plugin-kubernetes-backend';
import { Router } from 'express';
import { PluginEnvironment } from '../types';
import { CatalogClient } from '@backstage/catalog-client';

export default async function createPlugin(
  env: PluginEnvironment,
): Promise<Router> {
  const catalogApi = new CatalogClient({ discoveryApi: env.discovery });
  const { router } = await KubernetesBuilder.createBuilder({
    logger: env.logger,
    config: env.config,
    catalogApi,
  }).build();
  return router;
}

Then in packages/backend/src/index.ts, import the plugin:

import kubernetes from './plugins/kubernetes';
// ...
async function main() {
  // ...
  const kubernetesEnv = useHotMemoize(module, () => createEnv('kubernetes'));
  // ...
  apiRouter.use('/kubernetes', await kubernetes(kubernetesEnv));

That's it! The Kubernetes frontend and backend have now been added to your Backstage app.

9.2 Configure Kubernetes

We will use a service account (the "default" service account in the "default" namespace) for our portal to access Kubernetes resources.

First, let's give it cluster admin. Create a file named crb.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-cluster-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default

And apply it:

kubectl apply -f crb.yaml

And we need a token for this service account. We need to do this manually because:

Versions of Kubernetes before v1.22 automatically created long-term credentials for accessing the Kubernetes API. This older mechanism was based on creating token Secrets that could then be mounted into running Pods. In more recent versions, including Kubernetes v1.26, API credentials are obtained directly using the TokenRequest API and are mounted into Pods using a projected volume. The tokens obtained using this method have bounded lifetimes and are automatically invalidated when the Pod they are mounted into is deleted.

You can still manually create a service account token Secret; for example, if you need a token that never expires. However, using the TokenRequest subresource to obtain a token to access the API is recommended instead.

To create the token, run:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: default-secret
  annotations:
    kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token
EOF

Then run the following to get the base64 decoded value:

kubectl -n default get secret default-secret -o=json | jq -r '.data["token"]' | base64 --decode

The Kubernetes API URL can be obtained by running the following command.

kubectl cluster-info

OK, with the API URL and the token, we can configure our portal to access the minikube Kubernetes cluster. In file app-config.yaml, add the below section:

kubernetes:
  serviceLocatorMethod:
    type: multiTenant
  clusterLocatorMethods:
    - type: config
      clusters:
        - url: https://127.0.0.1:49671 # replace your API URL value here
          name: minikube
          authProvider: serviceAccount
          skipTLSVerify: true
          skipMetricsLookup: true
          serviceAccountToken: YOUR_K8S_SERVICE_ACCOUNT_TOKEN_HERE

9.3 Test

Now, let's restart our portal, and in the component "my-kubernetes-component", under the Kubernetes tab, we shall see some pods running.

10 Summary

In this tutorial, we learned how to:

  • generate the base code of your developer portal using Backstage (and a quick walkthrough of the code base)

  • install plugins and change frontend/backend.

  • enable GitHub SSO

  • integrate with GitHub

  • integrate with Argo CD

  • integrate with Kubernetes

Of course, building this portal isn't easy, but at least compared to creating one from the ground up, Backstage saved us a whole lot of time.

11 Where to Go from Here

For interested readers, these are the things you can do:

  • Customize the entity page by playing with the file packages/app/src/components/catalog/EntityPage.tsx to give it a more personal look. For example, you might want to remove the dependency card from the overview tab, or you might want to put those Argo CD cards in a separate tab.

  • Configure the looks of your portal.

  • Check out other integrations and plugins(https://backstage.io/plugins/).

  • Create templates.

  • Heck, you can even decide to create a plugin all by yourself. The only limit is your imagination (and your ability to tweak a bit of TSX code, let's be honest :)

If you like this tutorial, please click like, comment, and subscribe. Your support is my supreme motivation. See you next time.

developer portalPlatform Engineering BackstageDevOps trendsDevops