August: Homologating SCA across orgs, a second production region, and grouping findings into work

August was about making one SCA reusable workflow the same everywhere it runs, teaching IaC to tell two production regions apart, and converting scanner noise into a small number of stories a developer can actually close.

One SCA workflow, many orgs, no enterprise layer

June’s DevSecOps loop proved an AI first pass can drop hundreds of flags to a dozen real findings. August’s follow-up was operational: the reusable Black Duck workflow had drifted across sibling GitHub orgs, one of them still on a SARIF-shaped path we had already abandoned, another with no scan at all. An enterprise-level reusable workflow would have solved the fan-out. It also would have been a complexity tax we did not want this month.

The shape that actually shipped:

canonical repo  -->  tagged reusable workflow
                         |
          +--------------+--------------+
          |              |              |
        org A          org B          org C
        (move tag)     (call sites)   (from scratch)

Org-wide exclusion variables live at the org, not in each repo. A tag move picks up the new version where the call already exists; the remaining call sites are a PR loop, not a redesign. Ignore-once-apply-everywhere turned out to be a Black Duck UI action plus a documented comment, not a workflow flag. False positives (a bundled JRE matching as an unlicensed component, a critical that was not in the scan inventory) got written down as wiki, not re-litigated every Monday.

The takeaway: homologation is a tag-and-variable problem until you prove it is an architecture problem. Reach for the enterprise layer only after the same version number is already running in every org.

Two productions, one module graph

The infra thread this month was a second production region, standing next to the existing one, reusing as much of the Terragrunt / Terrateam module graph as possible. The first PR is the dangerous one: “mostly up to date with main” is a hypothesis, not a fact, and the GitHub App that Terrateam needs to apply was still waiting on an org owner.

Progress while that App was blocked was the useful part. Modules, security groups, and the naming that distinguishes the existing production from the sibling region could all move without the apply. The apply is the gate; everything else is code review.

ConcernExisting productionSibling region
Identity in codea dedicated namea distinct name, never a silent alias
Modulesorigin/mainsame modules, injected not copied
ApplyTerrateam + GitHub Appsame, blocked until the App exists

A related hygiene pass on an MFA self-service policy caught a quieter gap: AWS’s current sample includes iam:DeleteVirtualMFADevice and iam:GetMFADevice, and ours did not. Users who needed to rotate a virtual MFA were stuck, which looks like an IAM mystery until you diff against the vendor sample.

The generalized lesson: a blocked apply is not a blocked change. Split the work into “needs the credential” and “does not,” and keep shipping the second pile. Also: vendor sample policies drift. Diff yours against the current one before you debug the user.

Findings become stories, or they become noise

The last August thread was the human side of the same scanners. Code scanning on a primary app repo needed to land as work a developer could close, grouped by similarity, tagged security, with enough context to fix and validate. A pile of ungrouped alerts is not a backlog; it is a reason to dismiss.

The grouping rule that survived contact with the board:

scanner alerts
    │  group by CWE / same root cause

epic (already exists, do not invent a second one)

    ├── story: real defect, with the file and the check
    ├── story: real defect, different root cause
    └── do not file the false positive;
        dismiss in the scanner with a comment,
        or rewrite into a pattern the query recognizes

Dismissal is the cheaper, honest path when the bug is already gone from main or the query cannot see the safe pattern. Filing it as a story just so the count drops trains the next person to ignore the board. A related Datadog DevSecOps dashboard needed a repo-level filter next to the product filter, because a product name that does not bind to the repo you are staring at reads as “no data” when the data is simply elsewhere.

The transferable lesson: a finding that cannot become a story with a file path should not become a ticket. Group, dismiss, or rewrite. Counting is not remediating.

Longer write-ups land in /writing when a thread outgrows the monthly format.