case studies, anonymised and including the failures.
five engagements. four worked. one was reported as a failure.
Every client name here is withheld, and that is not a formality. The denylist of real client names is itself personal data, so it lives nowhere near a repository. What remains is the shape of the work: what was asked, what was measured, and what we would do differently. The failures are in here because the successes are not the evidence — the failures are.
An MSP desk that answers itselfrelay
Technicians were answering the same twenty questions daily from a wiki nobody trusted. The ask was a support product that answers from the ticket history, not from vendor documentation.
Thirty-one runbooks were written from resolved tickets, then scoped per role: technician gets the procedure, client gets the outcome, owner gets the commercial, auditor gets the evidence.
A planted-secret battery scored 13 of 13 leaks caught before reaching the model. The vendor allow-list grew from 86 to 148 multi-word spans, and false positives fell from 4 of 10 to 2 of 10.
We tuned the masker against precision first and spent a week un-masking real vendor names. Test both directions from the start — a redactor that hides the product name is not protecting anything, it is degrading the answer.
A LAN that was wider than the internetnetwork
A routine audit found control-plane services — no-auth dashboards, an inference engine, a local proxy — answering on the LAN. Nobody had deliberately exposed them; they were simply never firewalled.
An inet table locked twenty-two ports to loopback, tailnet, docker and IPv6 only, with drop-and-log for everything else. The rollback script was written and tested before the rules were applied.
The proof was a network namespace pretending to be a LAN host. Rules on: 000, dropped. Rules off: 200 — they were genuinely exposed, not theoretically. Rules re-applied: 000.
We would have hardened before the first no-auth service was installed, not after. Expose nothing by default is cheaper than audit everything later.
Nine of fifteenegress
An egress guard had been built and wired into the main model path. It worked. The question was whether it was the only path — and it was not.
Every outbound call site was inventoried. Two bypasses were found and closed: a legacy module posting page context straight to a provider, and a webhook sender pushing Slack and Discord payloads unscrubbed.
The guard was correct and protected nine of fifteen paths. Six had never been counted. A coverage test now fails on any outbound call that is not in the manifest.
The guard was not the problem — the inventory was. Build the inventory before you build the guard, or you will ship a correct guard over an unknown surface and believe you are covered.
The forty conversationsredaction
Redact personal data before it reaches a model, and reverse it on the way back. The first version did both — and destroyed forty real conversations doing it.
The detector matched inside an escaped quote, produced invalid JSON, and wrote it back. The fix was structural: mask the decoded value only, never the raw wire, and refuse anything that cannot be classified.
Four non-negotiable rules replaced the heuristic. The failure mode that mattered was not a missed mask, it was a corrupted write — which is why the rules now cover the write path, not just the read path.
We would have property-tested the round trip on adversarial input before trusting it with real conversations. Masking that corrupts the payload is worse than no masking.
A trading model that failed, and was reported as one· honest failure
Whether a strategy's in-sample edge survived contact with out-of-sample data. Not whether it looked good on a chart — whether it survived a walk-forward test.
908 out-of-sample windows were run across 63 tickers, with entry and exit simulated separately rather than netted, and fill assumptions stated rather than assumed away.
24% of windows were positive. 2 of 63 tickers passed both criteria. Small caps averaged negative net. The one-way fill divergence was +186 basis points — the strategy was paying for its own optimism.
Nothing — and that is the point. The test was built to be able to fail, and it did. A validation harness that cannot return "no" is not validation.
Two things are deliberately absent from this page. Real client names, because the denylist that protects them is itself personal data and does not belong in a public repository. And the full root-cause write-ups, which name systems and would tell a stranger where to look. What is here is enough to judge the work without being enough to attack it.
documented 2026-10-16 · all client identifiers withheld by policy