Add Jev sensitivity classifier to Brain's capture pipeline - #5632
Conversation
builder-io-integration
Bot
changed the title
![builder-io-integration[bot]](https://avatars.githubusercontent.com/in/353721?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- Resolve the Jev credential with the capture source identity instead of the ambient request context, so an editor of a shared source cannot classify the owner's content with their own credential. - Redact contact details, credentials, and links before the payload leaves the process for Typesafe. - Fail closed when the capture was too long for Jev to see in full. - Ask Jev the workspace's own sensitivityCustomInstructions as an extra question so workspace rules keep tightening under the Jev path. - Reject out-of-range probabilities instead of clamping them into a confident "not sensitive". - Treat a Jev credential-lookup failure as a classifier outage rather than an unconfigured workspace. - Make the allow bar exclusive so a score of exactly 0.2 is uncertain.
Visual recap — not generatedThe recap was suppressed because the diff matched a secret/credential pattern. No plan was published. Reason: |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
… contract - Mount PrivacySensitivitySettings in the general tab. It was only attached as a `content` property on a SettingsSearchEntry, which has no such field, so the card (and the new classifier selector) never rendered. Deep-link by hash instead. - Correct the BRAIN_SENSITIVITY_POLICY_VERSION comment: a bump stamps new decisions and separates quarantine rows, but does not retroactively re-screen, because indexSnapshotMatches compares a capture against its own artifact rather than the current constant. Document resanitize-captures as the migration path in the brain skill.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
…tion - resolveBuilderGatewayAuth() reads the ambient request user/org, so wrap it in runWithRequestContext bound to the same identity used for the stored-key lookup. Otherwise a shared-source editor or a queue worker spends their own Builder OAuth grant on the capture owner's content. - Redact unlabelled provider credentials (Slack xox*, AWS access key ids, Google API keys, JWTs) and run all credential patterns before the phone number rule. The phone rule was matching the digit runs inside a Slack token and leaving the token tail in the output. - Pass the request organization to probeJevCredential so org-scoped JEV_API_KEY values are not reported as missing in health and onboarding.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
sensitivityCustomInstructions is admin-authored free text on the same egress path as the capture, but was the only part of the payload not passing through sanitizeSensitiveText.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- Add Slack, AWS, Google, SendGrid, and JWT formats to the secret-credential hard-category screen, not just to the egress redactor. Redaction alone let a credential-bearing capture be stored as 'allowed', while the existing policy for a labelled secret is to suppress the whole capture. - Run the entire Jev credential ladder under the capture owner's identity, so workspace-connection discovery is scoped the same way the vault lookup is. - Preserve source orgId through resanitize-captures so an org-scoped JEV_API_KEY is still discoverable when re-screening.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
The same provider formats were duplicated between the secret-credential hard-category screen and sanitizeSensitiveText, so each new format had to be added twice and could silently drift. They now come from one UNLABELLED_CREDENTIAL_SOURCES list, and Google OAuth client secrets are included.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
… action - PEM private key headers and Authorization Bearer/Basic values are credential-bearing but carry no 'private key:' style label, so add them to the shared unlabelled-credential list. - The quarantine retention input clamped to 8760 hours while set-settings rejects anything over 720, so a user could enter a value that failed to save. This was unreachable until the privacy card started rendering.
There was a problem hiding this comment.
Builder reviewed your changes and found 2 potential issues 🔴
Review Details
Incremental Code Review Summary
The latest commit fixes both previously open issues: the egress redactor now covers PEM/bearer secrets, and the retention-hours UI range is consistent with the action limit. I verified and resolved those stale review threads. Focused Brain classifier, sanitization, readiness, and resanitization tests passed according to the reviewers.
This remains high risk because workspace instructions and classifier output cross the external Jev boundary. Two new high-severity redaction gaps remain: the unlabelled credential regex is case-sensitive, and the sanitizer does not cover all credential labels recognized by the hard-category detector. Both can expose secrets embedded in workspace rules or classifier-produced text sent through the Jev path.
🧪 Browser testing: Will run after this review (PR includes Brain settings UI), subject to Chrome automation availability.
The detector was case-insensitive and recognised access-token/private-key/ api-key, while the redactor was case-sensitive on the unlabelled formats and only knew password/passcode/secret/token/api key. Either direction of that drift is a leak: a label in only the detector suppresses without redacting, and one in only the redactor stores a credential-bearing capture as allowed. Both now compile from one CREDENTIAL_LABELS alternation, case-insensitive.
There was a problem hiding this comment.
Builder reviewed your changes — no new findings
Review Details
Incremental Review Completion
This review cycle is now finalized. The latest code-review results contained no additional findings beyond the credential-redaction issues already posted in the preceding review; those existing comments were not duplicated. The previously checked Brain settings and classifier changes remain high-risk because they process sensitive capture content and credentials across an external Jev boundary.
Focused tests reported passing classifier, sanitization, readiness, and resanitization coverage. Browser verification remains unavailable because Chrome automation tools are not exposed in the environment; route/source-level checks were previously healthy.
🧪 Browser testing: Skipped for this completion pass — prior browser attempts were blocked environment-wide by unavailable Chrome automation tooling.
This branch was successfully deployed
3 participants
Summary
Adds Jev as the default sensitivity classifier for the Brain template's capture pipeline, giving each capture a calibrated per-category probability score that decides whether it's allowed, quarantined, or blocked, while letting workspaces still choose an approved model or deterministic-only screening instead.
Problem
Brain captures needed a way to detect and strip sensitive information (e.g. employee performance, compensation, health data) before storage, similar to how Glean filters sensitive content. Previously, sensitivity classification relied only on deterministic regex screening or a manually configured approved model/engine pair, with no calibrated confidence signal and no integration with Jev.
Solution
Introduced a new
jev-classifiermodule that sends deterministically pre-screened content to Jev, which scores each sensitivity category as a probability. Scores above a block threshold quarantine and name the category; scores in an uncertain middle band quarantine without naming a category (fail-closed); clearly low scores allow the capture. Jev only decides the verdict — safe content is always derived from the deterministic line screen or the configured sanitizer. The classification pipeline degrades gracefully: Jev → approved model → deterministic screening, with uncertain captures staying quarantined if a configured classifier fails.Key Changes
server/lib/jev-classifier.ts: resolves Jev credentials (workspace-stored key first, falling back to Builder gateway), requests per-category probability scores in a single batched call, caches results, and maps scores to aBrainSensitivityDecisionwith block/allow/uncertain thresholds.resolveJevAuthprefers a storedJEV_API_KEY, then falls back to Builder gateway auth;probeJevCredentialreports readiness state (stored-key,builder-gateway,none,unavailable) with short-lived caching for health/onboarding surfaces.privacy-readiness.tsrewritten: now takes the resolved Jev credential status and the workspace'sprivacyClassifierpreference (jev|model|deterministic) to compute readiness and produce targeted warnings (e.g. credential lookup failed vs. not configured vs. deterministic-only chosen deliberately).privacyClassifiersetting: added toBrainSettings,set-settingsaction, and settings UI (Selectcontrol) to let workspaces pick Jev, a custom model, or deterministic-only screening. Defaults tojev.capture-sanitization.ts): now runsclassifyWithJevbefore falling back to the approved-model classifier, threading through failure reasons and auth source for observability.decisionScoresJsoncolumn tobrain_sensitivity_events(migration v27) to persist per-category probability scores;list-proposalsaction now annotates quarantine reasons with the classifier's probability (e.g. "performance 92%").search-index-contracts.ts: extractedBRAIN_SENSITIVITY_CATEGORIESas a shared const array, bumpedBRAIN_SENSITIVITY_POLICY_VERSIONto2, and added"jev"as a classifier type pluscategoryScoreson the decision contract.settings.tsxandbrain-health.tsnow surface classifier choice, Jev credential status, and readiness warnings; onboarding step description updated to reflect the new classifier options.jev-classifier.test.tscovering preference resolution, probability-to-disposition mapping, transport/credential ladder behavior, end-to-end classification, and graceful degradation; updatedprivacy-readiness.test.tsfor the new readiness logic.To clone this PR locally use the Github CLI with command
gh pr checkout 5632You can tag me at @BuilderIO for anything you want me to fix or change