Onboarding log

Every hurdle met while putting Sluiceway into a real repo for the first time, written down when it happens, with what we decided to do about it. The aim is an onboarding that a stranger gets through in one sitting, without reading the decision records.

Rules:

• A hurdle gets a line the moment it is met, before it is fixed or worked around. A workaround that only the first user knows is the thing this file exists to catch.
• Write the hurdle as any new user would meet it. The first real user’s repo is the evidence, never the subject.
• Each line ends in a decision: product (change the action), docs (a recipe or a sentence), tooling (something around the action), accept (leave it, with the reason), or open (not decided yet, with a recommendation).
• A product decision becomes a slice in docs/build-plan.md or a line in docs/later.md. This file only points.

Before the first scan

#	Hurdle	What a new user feels	Decision
1	There was no way to check a setup without running it. The first user’s config and stack discovery were dry-run with Sluiceway’s internal code before the first push. That check caught nothing, but it was the only reason to push with confidence.	“I have to merge a workflow and wait for a run to learn that my config has a typo or that half my stacks were not found.”	product, decided by the owner on 2026-09-21: a check mode with no credentials and no tool (record 0042, slice 2.12).
2	The workflow is about 50 lines before Sluiceway’s own step: checkout, the language runtime, dependencies, the tool, a plugin cache, the secret manager, the secrets.	“Most of this file is not about Sluiceway and I had to work it out myself.”	docs, done in slice 2.10: complete example workflows per setup (Node programs in a monorepo, a secret manager, a cloud with OIDC) in examples/workflows/, explained in docs/example-workflows.md, and held by tests to action.yml and the wiring Sluiceway needs. open: a generator, once two or three real setups exist.
3	Loading secrets from a secret manager’s env file into the job needed a 40 line script (one op run, mask every value, write to GITHUB_ENV). The vendor’s own action reads once per reference and drops plain values.	“The scary part of the setup is the part I had to write by hand.”	docs, done in slice 2.10: the script is examples/workflows/export-env.sh, shown in docs/credentials.md (“An env file of secret references”) and tested with fake values in test/docs/export-env.test.ts. It skips OP_*, GITHUB_* and RUNNER_* names, like the variant that ran for the first user. open: whether to also publish it as a small separate action in the sluiceway org. It runs in the user’s job and holds nothing, so it keeps the credential promises.
4	ignore matches the stack id and never the bare path, so apps/grafana ignores nothing and apps/grafana:* is what works.	“I ignored a stack and it is still on the dashboard, with no message.”	product: the check mode warns about an ignore glob that matches no stack and names the glob that would work (record 0042, slice 2.12). open: whether a scan warns too.
5	Files that no stack claims (docs, CI files, tool configs) force a full scan until they are listed under scan.unrelated. A sensible list had to be guessed up front.	“Every push previews everything and I do not know why.”	product, decided 2026-09-22: slice 2.22 prints the ready-to-paste block in the summary of a scan that fell back. Done in slice 2.22: the summary gets a section “Why this was a full scan” with the files of that push and the block the check prints for them (record 0042).
6	The runner needs version 2.328.0 or newer (node24), and the tool needs a minimum version.	“It failed before it started.”	accept. Both fail with one clear message that names the fix. Listed in the README’s requirements.
7	Before the first release the action has to be pinned to a 40 character commit SHA, and the header image only shows from a real tag or commit.	“The example in the README does not work as written.”	accept, done with release 0.1.0 on 2026-09-21: every example says sluiceway/sluiceway@v0, the header pictures come from the release tag, and the README’s “Pin a commit” keeps the full SHA for whoever wants to review every update.
8	Two files with nearly the same name: .github/workflows/sluiceway.yml (GitHub’s, when and how to run) and sluiceway.yaml (Sluiceway’s, settings about stacks). The first user asked why there are two.	“Which one do I edit, and did I put it in the right place?”	docs, done in slice 2.10: the table “What goes where” opens the README’s setup, and every example names the workflow deploy-dashboard.yml and the check deploy-dashboard-check.yml. A test fails when a user doc names a workflow file sluiceway.yml.

The first scan

First real run, 2026-09-21: 58 stacks, a full scan in 172 s with the default pool of 4 (previews added up to 675 s), median preview 11.9 s, slowest 28.4 s, so the defaults of concurrency and preview-timeout hold on a 2 CPU self-hosted runner. 14 pending, 36 in sync, 8 preview failed, job green. The body was 18,625 of 65,536 characters with every row in full. Loading 180 names from the secret manager cost one read request. The dashboard was created and pinned on the first try.

#	Hurdle	What a new user feels	Decision
9	Seven stacks had a stack config file but no stack in the state backend. Each became a row that says “preview failed: the tool exited with an error (exit code 6)”. The real reason, no stack named 'dev' found, is only in the job log.	“Seven red rows on my first dashboard, and the row does not tell me it is a stack I never created.”	product, decided by the owner on 2026-09-21: a failure reason of its own, “the stack does not exist in the backend”, and the summary names the ignore glob (record 0022 amended, slice 2.13).
10	A program that pulls from a private registry (a Helm chart in a private package) works on a laptop, where the person is logged in, and fails on the runner.	“It works on my machine. Why is this one stack broken in CI?”	docs, done in slice 2.10: docs/credentials.md, “What your programs fetch, the runner has to fetch”, with login recipes for private npm packages and a private OCI chart registry, and a pointer from step 4 of the README’s setup. The failure row already links to the log that names the registry.
11	A secret whose value is an ordinary word (here a username stored in the secret manager) is masked everywhere, so links in the log read github.com/***/repo.	“The log is full of stars and the links are broken.”	docs, done in slice 2.10: the first point under the recipe in docs/credentials.md says it, and a test holds the script to it.
12	The first dashboard of a real repo showed the plain header: no colour, no mascot, no voice. Four pending rows held a replace or a delete, and three of those were routine replacements of a Kubernetes Secret or ConfigMap, which Pulumi replaces whenever their content changes. In a Kubernetes repo that is the normal state, so the header that sets Sluiceway apart would almost never show.	“Where is the thing from the screenshots?”	product, decided by the owner on 2026-09-21, done in the records and the art (0043, which amends 0031 to 0034 and 0038 to 0040): the plain header is gone for good, and there is no state of its own in its place. The header always shows the real state with the mascot, and when a pending or deploying row has a delete or a replace, the same picture carries a sign: a warning triangle painted on the wall, which does not move. One sign for both, on the pending and deploying pictures, the count dots stay, and the alt text says it in words. The row warnings do not change, and dashboard.personality: false still removes the image and the voice. The renderer follows in slice 1.7c of the build plan.
13	The first run spent 2 m 16 s after the scan saving the tool’s plugin cache.	“The scan took three minutes and the job took six.”	accept. First run only. The example workflow keeps the cache step because every later run gains from it.
14	After a change to sluiceway.yaml the log said the scan “fell back to a full scan: no stack claims sluiceway.yaml”. The behaviour is right (0010), the words are not: they read as if a stack should claim the config file.	“Did I set something up wrong?”	product (slice 2.13): the line says that sluiceway.yaml changed, so every stack is previewed.
15	The preview link on a row opens the workflow run’s overview page. The stack’s detail is there, but the person has to scroll to the job summary and find their stack among all the others, or open the job and find its log group. The first user expected the link to land on that stack’s preview.	“I clicked preview on grafana and got a page about the whole run.”	product, slice 2.14: the link lands as close to the stack’s own detail as GitHub allows. The slice first finds out what can be addressed: the job summary of one job (#summary-<job id>), a heading inside a summary, a log group or line in the job log, and what it costs to learn the job id (an API call needs actions: read, which the scan-only workflow does not have). The fallback is the summary of the scan job, with the stacks in the same order as on the dashboard and a short index at its top. Done in record 0044: the run page drops a fragment from its address, so the summary with an index is as close as a link to a diff gets. The job id costs no permission (job.check_run_id), and a preview failure links to the job’s log.

Living with it

First real deploys, 2026-09-21: two stacks ticked by the owner a few minutes apart, each deployed by its own run and nothing else. Each record went queued, in_progress, success, the first kept its success when the second succeeded (auto_inactive: false), both rows went to in sync, and both are listed under recently deployed with who ticked and when. A deploy took about two minutes of tool time. From tick to in sync took five to six minutes, most of it waiting for a runner.

#	Hurdle	What a new user feels	Decision
16	In the read-only trial the dashboard draws checkboxes that do nothing. The first user ticked one. Nothing happened and nothing said why, and the tick sat there until the next full scan.	“I ticked it. Is it deploying? Is it broken?”	product, done in slice 2.17 (record 0045): a scan that runs in a workflow without the issues trigger cannot know that by itself, so dashboard.readOnly: true in sluiceway.yaml draws pending rows without boxes, leaves out the rescan box and says so in one line under the Pending heading. A config key and not an input, because a change of sluiceway.yaml makes the next push a full scan, so no row of the other mode is carried. The README’s read-only trial turns it on.
17	With resolve and settle on the same self-hosted runners as scan, apply and the repo’s own CI, a tick waited three minutes for a runner before anything on the dashboard changed.	“I ticked and nothing happened for minutes.”	docs: the README already says resolve and settle hold no secrets and can stay on hosted runners. Say why it matters, tick to “waiting to start” in seconds, and make the self-hosted example do it. open: whether the row should show something the moment GitHub accepts the tick, which no workflow can do, only an app.
18	The preview link works, and what it shows is thin. A row and the summary say update kubernetes:helm.sh/v3:Release arc-tsarr-release · values, eleven times: the top-level property name and nothing under it, and no value anywhere (0021). The first user could not see what he was about to deploy, which is the point of a gate.	“I am approving ‘values changed’ blind. Show me the diff.”	product, decided by the owner on 2026-09-21, both before 0.1.0. (A) Rows, summary and hash use nested property paths, still names and never values (values.controller.image.tag), slice 2.15, which reopens 0007 and 0008. (B) An opt-in setting prints the tool’s own full diff, values included, for every pending stack into that stack’s group in the job log and nowhere else, slice 2.16, which amends 0021 and 0022. Off by default, with a warning for public repos. (A) done in record 0046: a row now says values.controller.image.tag, shortens a long path and lists ten per change in its fold, and the summary shows every path whole. (B) is done in slice 2.16: scan.logDiff (record 0048). (C) decided 2026-09-22: one click lands on a page of its own, a GitHub check run per pending stack with Sluiceway’s diff of that stack (names, never values), slice 2.18, for 0.2.0. (C) built in slice 2.18 (record 0050): the preview page, which needs checks: write in the workflow. Values on that page need their own decision, because the runner’s masks do not apply there (research).
19	On the real dashboard with 14 pending stacks the header showed six crates, the top water level, and the warning sign squeezed between Penny and the wordmark. The picture said “a lot”, not how much, and did not look like it went on.	“Should we have the packages in the water be exactly as much as the pending stacks?”	product, decided by the owner on 2026-09-21, done in the records, the art and the renderer (0047, which supersedes the pending levels of 0039 and amends 0038 and 0043): one crate per pending stack up to 12, some stacked, and past 12 the row runs on off the left edge. The water rises in five steps. The alt text says the number. The destroy sign stands on a pole right of the wordmark. Deploying stays one picture.
20	The first release failed for everyone who started it from the moving tag. The first user’s first run on sluiceway/sluiceway@v0 ended in Sluiceway’s own error, “The action was started from the ref v0, which can move, and its package.json holds no version”, while @v0.1.0 and a commit SHA worked. The version was looked for through GITHUB_ACTION_PATH, which GitHub sets for composite actions only, and every test and the e2e started the bundle as uses: ./ or from a SHA, where the version is never needed.	“I copied the line from the README and it says its own files are broken.”	product, done in 0.1.1: the action finds its package.json from the address of its own bundle, and the e2e now also starts the bundle the way a runner starts it from a moving tag: a copy in a directory of its own, GITHUB_ACTION_REF=v0, no GITHUB_ACTION_PATH, the workspace as working directory.
21	The recently deployed list showed the same stacks deployed two and three times in one evening: after a successful deploy they were pending again on the next scan. A program that produces a value that differs on every run (a freshly minted token, a timestamp) can never reach in sync, and the dashboard cannot tell that from real work waiting.	“I ticked it, it deployed, and it is pending again. Did it not work?”	product, decided 2026-09-22: slice 2.22 adds the line on the row. Done in slice 2.22: a pending row whose newest deployment record went out with the same diff hash says “pending again right after a deploy of this same change, a value in the program may differ on every run”, and points at the tool’s own diff in the job log when scan.logDiff is on (record 0003).
22	A fold line says update kubernetes:helm.sh/v3:Release odoo-release · version and stops. For a version bump the person wants to see 17.0.3 → 17.0.4 right there, without opening anything.	“Now it says update deployment. Could we show what is being updated, so I know it is just a version bump?”	product, decided by the owner on 2026-09-22: an allowlist, dashboard.showValues, of paths whose values may appear, empty by default, with a copy-in list of safe paths in the docs (slice 2.19). A built-in “looks like a version” guess was rejected, in line with 0022. Built in slice 2.19 (record 0052): a row reads version 17.0.3 → 17.0.4 for a listed path, and the docs’ copy-in list leaves values.githubConfigSecret.github_token alone.
23	The first user compared Sluiceway with the internal dashboard that inspired it, on 2026-09-22. Five things it had that a team misses here: exclusions with a written reason, a kill switch in the config, an empty fresh preview treated as “nothing to deploy” instead of a failure, a comment to the ticker when a deploy is refused, and a rehearsal of the whole tick path.	“We stop deploys by editing the workflow, and an exclusion just disappears.”	product, decided by the owner on 2026-09-22: all five, slice 2.20. The raw tool plan on the preview page was compared too and not adopted: the page shows names, and values come only through the showValues list (slice 2.19), because the runner’s masks do not reach a check run.
21	A workflow with actions/cache@v4 for the plugin cache warns in every run that the step runs on Node 20 and was forced onto Node 24. The warning comes from the cache step, not from Sluiceway, which runs on Node 24.	“Is something in my setup about to break?”	docs, done after 0.1.1: actions/cache v5 and v6 run on Node 24, and the example workflows already used actions/cache@v6 and actions/cache/restore@v6. A test in test/docs/workflows.test.ts now holds every cache step in the docs and examples at v5 or newer, so the examples cannot hand the warning on.
24	The earlier internal dashboard (hurdle 23) once saw a tick in the web UI that started no workflow run, and added a sweep every five minutes to catch such ticks. Its own workflow only ran when the event carried changes.body.from. Sluiceway would lose such a tick too if a tick in the web UI could arrive without an issues.edited event.	“I ticked and nothing ever happened.”	accept, checked in slice 2.21. What was already known (issue 17, 2026-09-20 and 2026-09-21): two boxes ticked by hand in the web UI ten seconds apart gave two issues.edited events and two runs. Each event carried changes.body.from, the full body before the tick, and the only difference from the new body was [ ] to [x]. An edit made with the workflow token starts no run, which is why Sluiceway’s own writes never wake it. Sluiceway is not open to the trap either way: resolve takes nothing from the payload but that the issue is the dashboard (0025), no workflow in the docs filters on what the event says changed (a test holds that), a tick whose event was lost is picked up by the next resolve run, and a scan clears a tick that nothing picked up with a note that asks for a fresh one (slice 2.7). A second lab check on 2026-09-22, on a new issue created by the workflow token, with a workflow that only dumps what changes holds and a job behind the old filter: an edit of the title alone, made through the API, delivered changes.body with from as an empty string, so the old filter skipped the run. That is a filter that can drop an event, and Sluiceway has no such filter. A tick by hand in the web UI on that issue, whose body had never been edited, on 2026-09-22: one event and one run, sent by the person, changes holding only body, and from the full body before the tick (575 characters, no box ticked, against one ticked after). The job behind the old filter started. So a tick in the web UI delivers its event with the previous body, also as the first edit of a body. Three ticks by hand have been seen and none came without its event, which does not prove that none ever will; Sluiceway does not need it to. A filter on changes.body.from drops edits that change the title alone, which is harmless for ticks.