Deploying
GitHub Actions runners
stackd can run your GitHub Actions jobs on your own servers. Settings → Runners keeps self-hosted runners there, each used for one job in a fresh container and then replaced. Jobs on them cost no GitHub Actions minutes.
Quick start: run a project's CI on stackd
Connect a GitHub account in Settings → Builds, if none is. It registers the runners, so it needs admin on the repositories (or, for an organization, its runners); see Permissions.
Make a runner pool in Settings → Runners → Add runner pool (admins only):
- Serves: your repositories (pick them; up to 20), or an organization (all its repositories).
- Servers to run on, and Runners per server: three servers with one each run three jobs at once. Keep them off the server that runs the panel if you can.
- CPUs each and Memory each: the most a job may use. A build takes every core it gets; leave room for what else runs on the server (on a 4 CPU, 8 GB server, 2 CPUs and 4 GB is a fair start).
- Docker in jobs if the project's workflows use
services:,container:, Docker actions ordocker build. Then click Install Sysbox on each server the pool shows without it (Docker in jobs; it doesn't restart anything).
Point the workflow at it. In each job of
.github/workflows/*.yml, replaceruns-on: ubuntu-latest(orubuntu-24.04) with:runs-on: [self-hosted, stackd]With Docker in jobs, ask for it by label where a job needs it:
runs-on: [self-hosted, stackd, docker].Push. The pool's rows show each runner: Free (waiting for a job to start one, for a pool of several repositories), Starting, Waiting for a job, Running a job. GitHub lists them under the repository's Settings → Actions → Runners while they exist.
Adding another project
- A pool of repositories: Edit the pool, add the repository under Serves, save. Its runners start when its jobs wait; the pool's slots are shared by all its repositories.
- An organization's pool: nothing to do for a repository in it, unless the organization's default runner group leaves it out (GitHub: the organization's Settings → Actions → Runner groups; public repositories are left out by default).
Then change the project's runs-on as in step 3.
Labels
Every runner has self-hosted, Linux, X64 or ARM64 (its server's CPU),
stackd, docker with Docker in jobs, and the pool's extra labels. A job
runs on a runner that has every label its runs-on lists:
runs-on |
Runs on |
|---|---|
[self-hosted, stackd] |
any of the pool's runners |
[self-hosted, stackd, docker] |
runners with Docker in jobs |
[self-hosted, stackd, ARM64] |
runners on ARM servers only |
[self-hosted, stackd, big] |
a pool with the extra label big |
ubuntu-latest |
GitHub's own runners, never these |
Two pools with different sizes or servers can serve the same repository:
give one an extra label (big) and ask for it where a job needs it.
Examples
A Node project:
jobs:
test:
runs-on: [self-hosted, stackd]
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with: { node-version: 22, cache: npm }
- run: npm ci
- run: npm testTests against a database, and an image built and pushed (Docker in jobs):
jobs:
test:
runs-on: [self-hosted, stackd, docker]
services:
postgres:
image: postgres:17
env: { POSTGRES_PASSWORD: test }
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 2s --health-retries 30
steps:
- uses: actions/checkout@v5
- run: ./run-tests.sh # reaches the database at localhost:5432
env: { DATABASE_URL: "postgres://postgres:test@localhost:5432/postgres" }
image:
runs-on: [self-hosted, stackd, docker]
permissions: { contents: read, packages: write }
steps:
- uses: actions/checkout@v5
- uses: docker/login-action@v3
with: { registry: ghcr.io, username: "${{ github.actor }}", password: "${{ secrets.GITHUB_TOKEN }}" }
- uses: docker/build-push-action@v6
with: { push: true, tags: "ghcr.io/${{ github.repository }}:latest" }A Rust project: install the toolchain (nothing is preinstalled but a C toolchain) and keep the compiled dependencies in GitHub's cache, which the runners don't keep between jobs:
jobs:
test:
runs-on: [self-hosted, stackd]
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo test --lockedTo switch a project between these runners and GitHub's without editing
every job, read runs-on from a repository variable (GitHub: Settings →
Secrets and variables → Actions → Variables):
# RUNS_ON = ["self-hosted","stackd"]; unset: GitHub's runners
runs-on: ${{ fromJSON(vars.RUNS_ON || '"ubuntu-24.04"') }}How they differ from GitHub's runners
GitHub's ubuntu-24.04 |
stackd's runners | |
|---|---|---|
| Cost | Actions minutes | none (your servers) |
| CPU | x86_64 (or arm64 runners) | your servers' (X64 or ARM64) |
| Size | 2–4 CPUs, 7–16 GB | the pool's limits |
| Jobs at once | many | the pool's slots; others wait |
| Software | hundreds of tools preinstalled | Ubuntu 24.04 with git, curl, jq, a C toolchain, Python 3 and venv, zstd, xz, Docker's CLI; add the rest with setup-* actions or sudo apt-get install |
| Docker | always | with Docker in jobs |
| Network | the internet | the internet only: not your stackd apps, databases or private network |
| Between jobs | fresh machine | fresh container (caches through actions/cache) |
| Start | seconds | seconds; with several repositories up to ~15 s more |
Things that follow from these:
- Architecture. On ARM servers, a job building or downloading x86_64
binaries needs them for arm64, or cross-compiling. Most
setup-*actions pick the right one by themselves. - No private network. A job can't reach your apps or databases by
their private names or the mesh; deploy or test against public
addresses, or have the job call stackd's API (
stackdCLI with an API token as a secret). - Nothing persists. Each job's disk is new: no build cache, no Docker image cache, no files from the last job.
Troubleshooting
A job waits with "Waiting for a runner to pick up this job".
- Its
runs-onlabels must all be on one runner:ubuntu-latestnever matches,dockerneeds Docker in jobs,X64needs an x86_64 server. - The repository must be in the pool (or the organization's runner group).
- Look at the pool in Settings → Runners: Paused, Server offline, Needs Sysbox or Retrying with its error. All slots busy with other jobs means it waits its turn.
A slot shows Retrying. Its error says why the last runner failed; it's tried again after 30 seconds, doubling up to 10 minutes. The pool's Last error shows problems with GitHub itself: "github answered 403" or "404" means the git account lost admin on the repository (or the organization's runners), or its token expired: connect it again in Settings → Builds.
cc: not found, ensurepip is not available. The server built its
runner image before stackd 0.13.48: update stackd (Settings → System).
A step is killed (exit code 137) or the job ends with "lost
communication with the server". It ran out of the pool's memory: raise
Memory each, or use fewer parallel jobs in the build
(CARGO_BUILD_JOBS, make -j2).
docker: command not found or "Cannot connect to the Docker daemon".
Turn on Docker in jobs, and ask for docker in runs-on.
A connection to a private address times out. By design: jobs reach the internet only (see What a job gets).
What a job gets
- A fresh, single-use runner. Each runner takes one job, then is
replaced by a new container. Nothing a job leaves on disk reaches the
next one, so there's no local cache: use
actions/cacheorSwatinem/rust-cacheas on GitHub's runners. - The image:
stackd/actions-runner:2by default (a pool's Image left empty): GitHub's runner image,ghcr.io/actions/actions-runner:latest(Ubuntu 24.04 with git, curl, jq, sudo and the Docker CLI, as the userrunner), plus a C toolchain, pkg-config, Python's venv, zstd and xz; and for Docker in jobs, dockerd's iptables and Compose. It's rebuilt every week with a current runner. Jobs that need more install it (sudo apt-get install -y …), or the pool names an image builtFROMGitHub's with it installed; such an image must keeprun.shin its working directory. - No Docker daemon, unless the pool has Docker in jobs (below). The server's Docker socket is never given to jobs.
- Limits: CPUs and memory per runner, from the pool. 0 is no limit, and a build uses every core it gets, so limits keep room for whatever else the server runs.
- Network: the internet and DNS only, whatever the host firewall says. A job can't reach the server itself (any of its addresses, published ports included), your stackd apps or databases, private, shared or link-local addresses (a cloud's metadata service), nor another runner; nothing can connect to it. A server without nftables starts no runner.
- Time limits. A runner that doesn't connect to GitHub within 10 minutes, or that exits with an error, is replaced and its slot retried after 30 seconds, doubling up to 10 minutes. A job still running after 12 hours is stopped.
- Changing a pool. Pausing it, removing a server from it or lowering
the runners per server removes the idle runners at once; a runner in a
job finishes it first. Deleting a pool does the same for all of its
runners. Runners are named
stackd-<pool>-<server>-<slot>-<random>on GitHub.
Docker in jobs
With Docker in jobs on, each job gets a Docker daemon of its own, as on
GitHub's runners: container:, services:, Docker container actions,
docker build, docker run and Compose all work. Its runners also have the
label docker, for runs-on: [self-hosted, stackd, docker].
- Sysbox runs those runners: a container runtime whose containers can run Docker unprivileged, in a user namespace. The job's Docker can't see the server's Docker or its containers, and what it starts shares the runner's network, so it reaches the internet only, like the runner.
- Installing it: the pool shows each server's Sysbox, with Install
Sysbox (
POST /api/servers/{id}/sysbox). It works on Ubuntu or Debian, x86_64 or arm64, and needs kernel 5.15 on Ubuntu 22.04, 6.8 on 24.04, 5.5 elsewhere. Running containers aren't stopped: stackd pins Docker's current addresses in/etc/docker/daemon.jsonfirst (keeping the old file asdaemon.json.stackd-before-sysbox), so Docker only reloads. Adockerdstarted with--bipor--default-address-poolisn't touched: move those to daemon.json first. - AppArmor: on recent Ubuntu, stackd allows the FUSE mount Sysbox needs
in
/etc/apparmor.d/local/fusermount3. If a runner never starts with "fusermount3: mount failed: Permission denied", that file was changed or removed. - The image: the default one has dockerd, iptables and Compose. A pool's own image must have dockerd and iptables.
- No cache between jobs: each job's Docker starts empty, so it pulls its images every time, as on GitHub's runners.
- A pool with Docker in jobs runs only on its servers with Sysbox ready; the others' slots show Needs Sysbox.
Who can use it, and what it risks
Only admins see or change runner pools. The git account must be able to manage the runners, which making a pool checks first:
- a repository: admin on it. A classic token or OAuth connection with
repo, or a fine-grained token with Administration: read and write. - an organization: a classic token or OAuth connection with
manage_runners:org(accounts connected through OAuth before v0.13.44 have to be connected again to grant it), or a fine-grained token with the organization's Self-hosted runners: read and write.
Finding waiting jobs uses the git account's GitHub API rate limit: two requests a repository every 15 seconds, plus one a workflow run in progress. Unchanged answers don't count, so quiet repositories cost nothing.
A job runs the repository's code, and its dependencies' build scripts, on the server. Use pools for your own private repositories, where only people you trust push. GitHub advises against self-hosted runners on public repositories, where a fork's pull request could run code on them. Keep them off the server that runs the panel and off servers with data you can't risk.