Example workflows

3 stacks are pending: three crates wait upstream of Penny

Most of a Sluiceway workflow is not about Sluiceway: checking out, installing a language and your dependencies, installing the tool, loading credentials. These are complete workflows for common setups, ready to copy. Each one is the workflow of the README with those steps filled in, and each one is checked in this repo’s tests against the action’s inputs and the wiring Sluiceway needs.

Setup	File	Credentials from
Pulumi programs in TypeScript in one repo, with one `package.json` and lockfile at the root	node-monorepo.yml	GitHub secrets
An env file of secret references, loaded with one call per job	secret-manager.yml and export-env.sh	A secret manager (1Password in the example)
A cloud account and a state bucket in that cloud	cloud-oidc.yml	OIDC, no stored cloud key (AWS in the example)

Copy the file to .github/workflows/deploy-dashboard.yml on your default branch. The name of the file is yours to choose. Keep it different from sluiceway.yaml, which is Sluiceway’s own settings file at the repo root, so the two are never mixed up. Keep all four jobs in the one file: the scan looks for waiting ticks among the runs of its own workflow, and the rescan box and settle start a scan by starting that same workflow again.

What to change

The version of the action. The examples use sluiceway/sluiceway@v0, which follows every release from 0.1.0 until 1.0.0. To review every update yourself, pin a full commit SHA instead, as the README’s Pin a commit says.
The branch. The examples scan after a push to main. Use your default branch.
The secret and variable names. PULUMI_READ_TOKEN, OP_PREVIEW_TOKEN, AWS_PREVIEW_ROLE and the others are names the examples made up. Create them under the repo’s settings, or rename them in the file.
The environments. The apply job of every example names the stack’s environment, so that the credentials that change things can be secrets of a GitHub Environment (security). The environment of a stack is sluiceway unless sluiceway.yaml gives it another. Where your plan has no environments, remove the environment: block and keep those credentials as repository secrets.
timeout-minutes on apply. A deploy has no time limit of Sluiceway’s. Set one that fits your slowest stack.
The runner. Change runs-on of scan and apply for self-hosted runners. They need runner version 2.328.0 or newer. resolve and settle hold no credentials and can stay on hosted runners.

The monorepo

The dependencies are installed once, with one npm ci at the root, before the scan previews every stack. The plugin cache keeps the providers between runs: the first run fills it, which takes a while after a large first scan, and every later run gains from it. apply only restores it. Your programs may need more than npm: a build step, a code generator, another language. Put it before the Sluiceway step, once.

The credentials sit on the Sluiceway step only, so the install scripts of your dependencies never see them.

The secret manager

The scan and the apply job each resolve one env file of references with one op run, and export-env.sh masks the secrets and writes every value to the job environment. credentials.md explains the script and what it does not do. Two service accounts: one that sees only the credentials that read, for the scan, and one for apply, whose token is a secret of the environment.

The cloud

The scan assumes a role that can only read, and apply a role that can change things. The trust policy of the second one names the environment, so no other job can assume it. The state lives in a bucket (PULUMI_BACKEND_URL), and stack secrets use a passphrase from a repository secret. The jobs that run the tool ask for id-token: write in their own permissions: block, which replaces the workflow’s, so it repeats the whole block.

Before you merge one

Run the check in a pull request first. It tells you whether Sluiceway finds your stacks and understands sluiceway.yaml, with no credentials and no tool. What it cannot tell you is whether a preview works: whether the runner can fetch what your programs fetch (credentials), and whether every stack exists in the backend. The first scan shows that, one row per stack.

The files

cloud-oidc.yml

# Cloud credentials through OpenID Connect, so no cloud key is stored anywhere,
# and the Pulumi state in a bucket of the same cloud. The example uses AWS;
# docs/credentials.md says what changes for other clouds. Copy it to
# .github/workflows/deploy-dashboard.yml on the default branch.
#
# `@v0` follows every release until 1.0.0. To review every update yourself,
# pin a full commit SHA instead, as the README's "Pin a commit" says.
name: deploy-dashboard

on:
  push:
    branches: [main]
  schedule:
    - cron: "0 6 * * *"
  workflow_dispatch:
  issues:
    types: [edited]

permissions:
  contents: read
  issues: write
  deployments: write
  actions: write
  pull-requests: read
  checks: write

jobs:
  scan:
    if: github.event_name != 'issues'
    runs-on: ubuntu-latest
    concurrency: sluiceway-scan
    # A job's permissions replace the workflow's, so the whole block is here
    # again, with id-token: write added for this job only.
    permissions:
      contents: read
      issues: write
      deployments: write
      actions: write
      pull-requests: read
      checks: write
      id-token: write
    steps:
      - uses: actions/checkout@v7
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      # A role that can only read, trusted for this repo's default branch.
      - uses: aws-actions/configure-aws-credentials@v6
        with:
          role-to-assume: ${{ vars.AWS_PREVIEW_ROLE }}
          aws-region: ${{ vars.AWS_REGION }}
      - uses: sluiceway/sluiceway@v0
        env:
          PULUMI_BACKEND_URL: ${{ vars.PULUMI_BACKEND_URL }}
          PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
        with:
          mode: scan

  resolve:
    if: github.event_name == 'workflow_dispatch' || (github.event_name == 'issues' && contains(github.event.issue.labels.*.name, 'sluiceway'))
    runs-on: ubuntu-latest
    concurrency: sluiceway-resolve
    outputs:
      matrix: ${{ steps.resolve.outputs.matrix }}
    steps:
      - uses: actions/checkout@v7
      - id: resolve
        uses: sluiceway/sluiceway@v0
        with:
          mode: resolve

  apply:
    needs: resolve
    if: ${{ !cancelled() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]' }}
    strategy:
      fail-fast: false
      matrix:
        include: ${{ fromJson(needs.resolve.outputs.matrix) }}
    runs-on: ubuntu-latest
    timeout-minutes: 60
    concurrency:
      group: sluiceway-apply-${{ matrix.stack }}
      queue: max
    environment:
      name: ${{ matrix.environment }}
      deployment: false
    permissions:
      contents: read
      issues: write
      deployments: write
      actions: write
      pull-requests: read
      checks: write
      id-token: write
    steps:
      - uses: actions/checkout@v7
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      # A role that can change things. Its trust policy names the environment
      # (the `environment:<name>` subject), so no other job can assume it.
      - uses: aws-actions/configure-aws-credentials@v6
        with:
          role-to-assume: ${{ vars.AWS_DEPLOY_ROLE }}
          aws-region: ${{ vars.AWS_REGION }}
      - uses: sluiceway/sluiceway@v0
        env:
          PULUMI_BACKEND_URL: ${{ vars.PULUMI_BACKEND_URL }}
          PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
        with:
          mode: apply
          deployment-id: ${{ matrix.deployment }}

  settle:
    needs: [resolve, apply]
    if: always() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: sluiceway/sluiceway@v0
        with:
          mode: settle

export-env.sh

#!/usr/bin/env bash
# Loads an env file of secret references into the job environment, masked.
# Run it inside your secret manager's `run` command, which resolves every
# reference of the file into this process's environment, for example:
#
#   op run --env-file=ci.env --no-masking -- bash export-env.sh ci.env
#
# It prints nothing but ::add-mask:: commands. Never add `set -x` or an echo.
set -euo pipefail
file="$1"
# A line whose value holds this is a secret. Every other line is a plain value.
reference="${SECRET_REFERENCE:-op://}"

while IFS= read -r raw || [ -n "$raw" ]; do
  raw="${raw%$'\r'}"
  # Take NAME from lines like `NAME=...`, `NAME = ...` or `export NAME=...`.
  [[ "$raw" =~ ^[[:space:]]*(export[[:space:]]+)?([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*= ]] || continue
  name="${BASH_REMATCH[2]}"
  # The secret manager's own token and settings stay on this step, and the
  # runner does not let a step set its own names.
  case "$name" in OP_* | GITHUB_* | RUNNER_*) continue ;; esac
  value="${!name-}"
  [ -n "$value" ] || continue

  # Mask first, every line on its own, because the log is matched line by line.
  if [[ "$raw" == *"$reference"* ]]; then
    while IFS= read -r line || [ -n "$line" ]; do
      line="${line%$'\r'}"
      [ -n "$line" ] || continue
      printf '::add-mask::%s\n' "${line//%/%25}"
    done <<<"$value"
  fi

  # Then write, in the delimiter form so that newlines survive.
  delimiter="ghadelimiter_$(od -An -N16 -tx1 /dev/urandom | tr -d ' \n')"
  if [[ "$value" == *"$delimiter"* ]]; then
    echo "The value of $name holds the delimiter. Run the step again." >&2
    exit 1
  fi
  printf '%s<<%s\n%s\n%s\n' "$name" "$delimiter" "$value" "$delimiter" >>"$GITHUB_ENV"
done <"$file"

node-monorepo.yml

# Pulumi programs in TypeScript, in one repo with a shared package.json, and
# credentials stored as GitHub secrets. Copy it to
# .github/workflows/deploy-dashboard.yml on the default branch, and read
# docs/example-workflows.md for what to change.
#
# `@v0` follows every release until 1.0.0. To review every update yourself,
# pin a full commit SHA instead, as the README's "Pin a commit" says.
name: deploy-dashboard

on:
  push:
    branches: [main]
  schedule:
    - cron: "0 6 * * *"
  workflow_dispatch:
  issues:
    types: [edited]

permissions:
  contents: read
  issues: write
  deployments: write
  actions: write
  pull-requests: read
  checks: write

jobs:
  scan:
    if: github.event_name != 'issues'
    runs-on: ubuntu-latest
    concurrency: sluiceway-scan
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      # Once for every program in the repo, not once per stack.
      - run: npm ci
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      # The providers the programs use. The first run fills the cache.
      - uses: actions/cache@v6
        with:
          path: ~/.pulumi/plugins
          key: pulumi-plugins-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
      # The credentials sit on this step only, so the install scripts of npm ci
      # never see them. A preview needs no more than read access, where your
      # backend and cloud offer credentials that can only read.
      - uses: sluiceway/sluiceway@v0
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_READ_TOKEN }}
        with:
          mode: scan

  resolve:
    if: github.event_name == 'workflow_dispatch' || (github.event_name == 'issues' && contains(github.event.issue.labels.*.name, 'sluiceway'))
    runs-on: ubuntu-latest
    concurrency: sluiceway-resolve
    outputs:
      matrix: ${{ steps.resolve.outputs.matrix }}
    steps:
      - uses: actions/checkout@v7
      - id: resolve
        uses: sluiceway/sluiceway@v0
        with:
          mode: resolve

  apply:
    needs: resolve
    if: ${{ !cancelled() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]' }}
    strategy:
      fail-fast: false
      matrix:
        include: ${{ fromJson(needs.resolve.outputs.matrix) }}
    runs-on: ubuntu-latest
    timeout-minutes: 60
    concurrency:
      group: sluiceway-apply-${{ matrix.stack }}
      queue: max
    environment:
      name: ${{ matrix.environment }}
      deployment: false
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version-file: .nvmrc
          cache: npm
      - run: npm ci
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      - uses: actions/cache/restore@v6
        with:
          path: ~/.pulumi/plugins
          key: pulumi-plugins-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
      # A secret of the environment above: only a job that names the
      # environment, on a branch the environment allows, can read it.
      - uses: sluiceway/sluiceway@v0
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_DEPLOY_TOKEN }}
        with:
          mode: apply
          deployment-id: ${{ matrix.deployment }}

  settle:
    needs: [resolve, apply]
    if: always() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: sluiceway/sluiceway@v0
        with:
          mode: settle

secret-manager.yml

# Credentials and backend settings kept in a secret manager as an env file of
# secret references, loaded once per job with one bulk call. The example uses
# 1Password; docs/credentials.md has the same pattern for other managers.
# Copy it to .github/workflows/deploy-dashboard.yml on the default branch, and
# copy export-env.sh to .github/scripts/export-env.sh.
#
# `@v0` follows every release until 1.0.0. To review every update yourself,
# pin a full commit SHA instead, as the README's "Pin a commit" says.
name: deploy-dashboard

on:
  push:
    branches: [main]
  schedule:
    - cron: "0 6 * * *"
  workflow_dispatch:
  issues:
    types: [edited]

permissions:
  contents: read
  issues: write
  deployments: write
  actions: write
  pull-requests: read
  checks: write

jobs:
  scan:
    if: github.event_name != 'issues'
    runs-on: ubuntu-latest
    concurrency: sluiceway-scan
    steps:
      - uses: actions/checkout@v7
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      # Leave this out when op is part of your runner image.
      - uses: 1password/install-cli-action@v4
      # One `op run` resolves the whole file. The service account of this job
      # can see only the vault with the credentials that read.
      - name: Load the environment
        env:
          OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_PREVIEW_TOKEN }}
        run: op run --env-file=ci/preview.env --no-masking -- bash .github/scripts/export-env.sh ci/preview.env
      - uses: sluiceway/sluiceway@v0
        with:
          mode: scan

  resolve:
    if: github.event_name == 'workflow_dispatch' || (github.event_name == 'issues' && contains(github.event.issue.labels.*.name, 'sluiceway'))
    runs-on: ubuntu-latest
    concurrency: sluiceway-resolve
    outputs:
      matrix: ${{ steps.resolve.outputs.matrix }}
    steps:
      - uses: actions/checkout@v7
      - id: resolve
        uses: sluiceway/sluiceway@v0
        with:
          mode: resolve

  apply:
    needs: resolve
    if: ${{ !cancelled() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]' }}
    strategy:
      fail-fast: false
      matrix:
        include: ${{ fromJson(needs.resolve.outputs.matrix) }}
    runs-on: ubuntu-latest
    timeout-minutes: 60
    concurrency:
      group: sluiceway-apply-${{ matrix.stack }}
      queue: max
    environment:
      name: ${{ matrix.environment }}
      deployment: false
    steps:
      - uses: actions/checkout@v7
      - uses: pulumi/actions@v7
        with:
          pulumi-version: ^3.229.0
      - uses: 1password/install-cli-action@v4
      # The token that reaches the credentials that change things is a secret
      # of the environment above.
      - name: Load the environment
        env:
          OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_DEPLOY_TOKEN }}
        run: op run --env-file=ci/deploy.env --no-masking -- bash .github/scripts/export-env.sh ci/deploy.env
      - uses: sluiceway/sluiceway@v0
        with:
          mode: apply
          deployment-id: ${{ matrix.deployment }}

  settle:
    needs: [resolve, apply]
    if: always() && needs.resolve.outputs.matrix != '' && needs.resolve.outputs.matrix != '[]'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: sluiceway/sluiceway@v0
        with:
          mode: settle