July: Observation-mode SCA, a single-source audit plan, and capture that actually matches the session
July was about keeping scanners in the loop without letting low-confidence findings stop the pipeline, refusing to keep three copies of an audit plan in sync, and finding out that a network trace which does not cover the session is not a network trace.
Observation mode is a first-class CI state
The first week of July was spent grinding a Python Black Duck miss on a shared GitHub Actions reusable workflow. The easy on-policy fix was to mirror the existing npm scan step for Python and cascade it to the calling repo. The harder constraint was the one that actually mattered: we were still in observation mode, and a low-confidence finding must not fail the run.
The resulting contract, written as workflow inputs rather than tribal memory:
mode fail the job? what we keep
-------------- ---------------- --------------------
observation no SARIF + a comment
enforcing yes, on policy same artifacts, plus a red check
off no nothing (explicit, not accidental)Parameterizing the fail/continue switch on the reusable side, then keeping both languages in coverage, turned the edge cases into signal instead of outages. Observation is not “the scanner is broken”; it is a named state with the same artifacts as enforcing, minus the red check.
The takeaway: a security gate that cannot be run quietly will be turned off. Give the workflow an observation mode as a first-class input and the coverage stays on while confidence is still being earned.
One living source for an audit plan
A SOC 2 vulnerability and patch-management page needed a gap-closure plan before the next audit window. The draft that came out of the sessions was feasible on paper (a handful of policy and register rows, plus one credibility gap on scanner coverage that had to be restated as a phased rollout). The delivery mechanism was not.
Three representations of the same seven rows were on the table: Jira epics, a Confluence table, and a Roadmap Planner macro that is not API-writable. The macro would have required a browser-automation agent whose dry-run had never been proven. Maintainability was the weakest axis. The picture would drift the moment a date slipped.
keep drop
--------------------------- --------------------------------
Jira epics = source of truth live Roadmap Planner macro
one Jira-issues macro embed a second table that copies Jira
a frozen screenshot at audit three representations in syncThe plan itself survived. The hybrid delivery did not. A one-time audit artifact can be a screenshot; a living program cannot be a picture you have to babysit.
The generalized lesson: if two audiences need two views of the same work, generate the second view at the moment it is needed. Do not keep it live. Anything that is not API-writable is a snapshot pretending to be a database.
Capture the session you actually ran
The other July thread was a QA capture spike: non-technical testers documenting cases so they can later become Playwright specs, without creating external platform users. A 24/7 screen recorder was the wrong centerpiece (compliance, plus it solves capture rather than structured markdown and upload). The prototype that shipped instead was a local CLI that writes a case template, a video, and a HAR.
Two sessions in a row, the HAR lied. It captured only the initial page load (a few dozen entries in a sub-second window) and then went silent for the rest of a ~55 second recording. Frame extraction from the video showed a failed login in one session and a successful login with no logout in the other. The network trace matched neither. Reproducing it confirmed the gap was in how the capture attached to a Flutter web app’s XHR/fetch traffic, not a one-off.
| Artifact | What it proved | What it missed |
|---|---|---|
case.md | intent, env, expected steps | whether the step actually happened |
network.har | the first page load | every XHR after t≈1s |
evidence.mp4 | login success, no logout, the toast | request payloads |
The video is the evidence; the HAR is a hypothesis until its timestamp range covers the session. A capture pipeline whose network trace can go quiet without failing the session will train you to trust the wrong file.
The transferable rule: when two instruments of the same event disagree, believe the one that still has bytes after the interesting moment, and treat a silent companion as a bug in the test, not as a passing result.
Longer write-ups land in /writing when a thread outgrows the monthly format.