PII scrubbing on foro.sh: what gets caught, and what does not

What the tool layer actually sees

A previous post admits PII scrubbing has a limit: results are "scrubbed for structured PII on paid plans," which helps, but isn't the same as "no payload data ever leaves." This post covers how that scrubbing works. Every tools/call a deployed server answers passes through the in-container gate before it reaches the client: the CRM record, the support ticket body, the row from a database query. That's a project's real data in plaintext, and the scrubber described below exists to sit in front of it.

What follows is what the scrubber actually does, read straight off the code: two tiers with different reach, a checksum behind every numeric detector, an ordering rule that keeps one type from masquerading as another, and a design that fails closed rather than degrading quietly. It also lists what does not get caught, since a scrubber that hides its own limits is worse than one that states them.

Two tiers, one boundary

The scrubber runs in two tiers, split along a networking constraint more than a feature gate. Tier 1 is a single file with zero imports: five detectors for email, IBAN, credit card, Dutch BSN, and phone, each a regex plus, where one exists, a checksum. Zero imports is what lets it run unmodified inside a deployed server's own container, vendored in next to the in-container gate itself, the same process that authenticates every call and now scrubs the result before it leaves. Synchronous, in-process, language-independent: it has to be, because the gate is on the hot path of every response a deployed server sends.

Tier 2 is named entity recognition over free text, for a person's name or a street address in prose, neither of which a regex can reliably find. It runs EN and NL through a Presidio sidecar reachable only from inside the platform, never from a project's own network, so it lives in a separate API-side module instead, one that imports the Tier 1 core back out and re-exports it unchanged so every existing caller sees no API difference. The gate cannot make that network hop: each deployed project gets its own Docker network, isolated from every other project's and from the platform's internal services, and the Presidio sidecar sits behind that same isolation. So Tier 1 alone travels into the container. Tier 2 stays on the API side, where a dashboard chat session or the Playground's Inspector path can reach it, batching every masked string leaf from a payload into one request and waiting up to five seconds before failing closed.

Why a regex alone is not a detector

A regex that matches "looks like a card number" catches every 16-digit run in a payload: order numbers, database ids, a truncated UUID. Three of the five Tier 1 detectors carry a checksum behind the pattern specifically so that a match has to be a plausible value of its type, not just a string of the right shape.

IBAN uses the ISO 7064 mod-97 check: strip whitespace, uppercase, move the first four characters to the end, map letters to two-digit values, and require the result to be congruent to 1 modulo 97. Credit cards use Luhn, ISO/IEC 7812's check digit, over any 13 to 19 digit run, compact or space and dash grouped in fours. Dutch BSNs use the "11-proef": pad to nine digits, weight each one 9 down to 2 then a final -1, and require the weighted sum to be divisible by 11. None of these can be spoofed by accident the way a length-only pattern can.

Two more details in the same file are there for reasons that only show up under load. The email pattern bounds every quantifier explicitly (64 characters for the local part, RFC 5321's own limit; 63 per domain label, RFC 1035's; 24 for the TLD) instead of leaving the local part an unbounded +. An unbounded quantifier there is a quadratic blowup: a long run of characters that are valid in an email's local part but never followed by @, plausible in a base64 blob or a hex dump a tool legitimately returns, makes the engine retry a greedy match from every one of that run's starting positions. At the roughly 1MB a single tool result is allowed to reach before the API truncates it, that is minutes of backtracking on one string. And the space-grouped IBAN pattern restricts its groups to uppercase letters and digits only, not the case-insensitive match the compact pattern uses, because a grouped pattern that failed its checksum would otherwise keep consuming trailing lowercase prose into the match before the checksum ever got to reject it.

Detector order is a correctness property

The five detectors do not run in an arbitrary order.DETECTORS lists email, IBAN, credit card, BSN, then phone, and that sequence matters because the earlier, checksum-backed detectors mask their matches into bracketed, digit-free placeholders ([CREDIT_CARD], [BSN]) before the loosest detector, phone, ever sees the text. Phone numbers have no checksum to gate on, so the pattern is kept to a handful of concrete shapes, E.164-style international numbers with a plus or double-zero prefix and 8 to 15 digits, Dutch national numbers, North American 3-3-4, rather than one pattern loose enough to catch everything phone-shaped. Run that pattern before the digit checksums instead of after, and a validated credit card number would already have been replaced by ten to nineteen unmasked digits still sitting in the text for the phone pattern to half-match, or worse, would have matched as a phone number first and never reached the Luhn check at all. The ordering is what keeps a card from being re-masked, or mis-masked, as a phone number.

Fail closed, not fail silent

Every detector in Tier 1 redacts rather than blocks: a match gets replaced in place and the call carries on, with a report of how many values of each type were masked. But the module around those detectors, the payload walker that recurses through a tool result's objects and arrays looking for string leaves to scrub, takes the opposite stance the moment anything goes wrong. It throws PiiScrubError rather than returning the unscrubbed input, on a payload that nests past a 200-level depth cap, on a non-plain object (a Date, a Map, anything Object.entries would silently flatten and lose fields out of), on a Tier 2 sidecar that times out, returns a non-200, or answers with a span count that does not match what was sent.

The reasoning is in the file's own header comment: a scrubber that silently returns the original text when it hits a bug is worse than one that errors, because that failure mode is a silent PII leak, to an LLM on the dashboard side, or to a live MCP client on the deployed-server side. In the gate, a scrubbing-entitled response that cannot be fully scrubbed does not go out unscrubbed and does not get dropped either. It gets replaced: every call the gate can attribute the response to receives a synthetic isError result carrying a withheld-content message instead of its real one, and is flushed as a failed call. When the gate cannot attribute the response to any pending call at all, an out-of-band stream, or a body that overflowed before an id was ever read, it sends one transport-level JSON-RPC error object instead, since there is no call left to answer. The log-collector's redaction pass takes the equivalent stance on its own side: if it cannot fetch a project's current secret values or plan entitlement to redact against, it throws, and the flusher drops that batch of log lines rather than persist them unprotected. A dropped batch is a gap in the logs. Unscrubbed PII sitting in durable storage is not something a later fix can undo.

What gets counted, and what null means

Every call the gate runs through the scrubber writes a pii_redactions field onto its metric line, and the value distinguishes two different kinds of nothing. Null means this call was never run through the scrubber at all, the project is not on a plan with Tier 1 entitlement, or the project's own "Data protection" toggle is off. Zero means the scrubber ran and found nothing to mask. A dashboard that only ever saw zero would have no way to tell "this project is clean" from "this project isn't being checked," which is exactly the distinction a customer evaluating the feature would want answered.

That per-call count rolls up into the daily tier the same way token counts do: a plain sum() that skips nulls, paired with a pii_redactions_calls counter that says how many of the day's calls actually carried a count, so a day is never read as fully scrubbed when only part of it was. Raw per-call rows live for the plan's retention window, Free through Enterprise; the daily rollup outlives them on its own longer ladder, but it is not kept forever either; nothing here is meant to be read as a permanent record of what was redacted, only as a metric of how much scrubbing activity a project is generating over the window that plan buys.

A separate pass over a server's own logs

Everything above is the metering path: what a client receives back from a tools/call. A deployed server also writes its own stdout and stderr, whatever it chooses to log for its own purposes, and that is a different stream with a different collector. The log-collector's redaction pass runs two passes over every line before it is persisted to object storage: secret redaction, always on, replacing any occurrence of a project's own stored secret values so a credential injected into the container's environment can never end up readable in a log; and PII scrubbing, gated the same way the metering path is, a no-op on Free.

Both lookups, the current secret values and the current plan entitlement, are fetched fresh on every flush rather than cached. That is deliberate, not an oversight: caching either one would reopen a staleness window where a secret added, or a plan just upgraded, sits unprotected in already-buffered lines until a cache entry expires. Fetching on every flush means the values used to redact a batch of lines are always at least as current as anything that was live while those lines were produced, which is the whole property the design is after.

What this deliberately does not catch

Every scrubber has limits. The ones below are checked against the code as it exists today, not an idealized version of it, because stating them plainly beats pretending they are not there.

The BSN detector is a heuristic, and it over-redacts by design: roughly one in eleven random eight or nine digit numbers pass the 11-proof checksum by chance, so an order number or a reference id can get masked as [BSN]. That is the accepted tradeoff, stated plainly in the code's own comment: over-redacting a stray number is the safe side of a PII scrubber, missing an actual citizen number is not.

Tier 2 does not exist inside a deployed container. A person's name or a street address written in prose, inside a live tools/call result on a deployed server, is not caught by the gate, only Tier 1's structured types are. Reaching NER there would mean a network hop out of a project's isolated network, which the architecture does not allow.

Only strings are scrubbed. A card or a BSN arriving as a JSON number rather than a string is left untouched, because masking it would change the value's type, and would turn roughly one in eleven ordinary numeric fields into a false BSN hit across every result a project returns. Object keys are never touched either, only values; a field literally named ssn is not itself PII.

A card number with non-standard spacing, American Express's 4-6-5 grouping rather than groups of four, is caught in its compact, no-space form but not in a spaced one; that shape is rare enough in tool output that it is left to the Tier 2 pass rather than adding a sixth pattern to Tier 1.

Entitlement is plan-gated, and the floor sits precisely here: Tier 1 from Starter up, Tier 2 from Team up, and a project-level "Data protection" toggle that defaults on but can be switched off, in which case the plan entitlement is moot. None of it runs on Free.

The residual is worth being exact about: with scrubbing on, a tools/call result's error text, the string an isError result carries, is derived from the already-scrubbed copy, because the gate masks result.content and result.structuredContent in place before it ever reads the outcome. But a transport-level JSON-RPC error, the error.message field a server sends instead of a result, is never a scrubbable result at all by the shape the gate checks for, and reaches the metric record's 500 character error field exactly as the server wrote it. That is the accurate version of the caveat, narrower than the sovereignty post's blanket statement that an error string "can carry a fragment of whatever the error contained," and it is best read this way: true of a transport error, no longer true of a tool result on a scrubbing project.

None of these limits are secret. They are the same ones documented on the data protection page; this post is the mechanism behind that page's claims, not a new disclosure.