fix(brand-profile): bind Wikipedia/Wikidata lookups to a site-validated entity (LLMO-6580) - #339
Open
cwjwisse wants to merge 1 commit into
Open
fix(brand-profile): bind Wikipedia/Wikidata lookups to a site-validated entity (LLMO-6580)#339cwjwisse wants to merge 1 commit into
cwjwisse wants to merge 1 commit into
Conversation
…ed entity (LLMO-6580)
The brand-profile products extractor could attach a foreign (often harmful)
entity's product catalogue to a real customer. Root cause: when Wikidata SPARQL
returned fewer than the threshold, the code ran a decoupled opensearch for
`${brandName} company` and blindly took titles[0] with no check that the article
belonged to the resolved entity. A bare 2-3 letter acronym (or a dev/www
subdomain label) produced by the old domain-based brand-name fallback then
fuzzy-matched famous same-initials articles (d*->"D-Company", e*->"E Company").
Fix:
- New services/brand-resolver.js resolves a brand name with a confidence signal
and the site's registrable domain (hand-rolled multi-part-TLD table, no new
runtime dependency). It never emits a bare acronym or a stop-label subdomain as
a high-confidence name, and keeps a best-effort, short-timeout homepage title
fetch that fails safe to null.
- wikipedia.js gains getWikidataEntity, validateEntityAgainstSite,
findValidatedWikidataEntity, fetchWikipediaExtractByTitle and
fetchValidatedSummary. Every fetch is bound to an entity validated against the
site (strong P856 official-website host match, or a weak label match only for
non-low-confidence names). Low-confidence acronyms require P856. The decoupled
by-name opensearch is gone from the product and competitor paths; the old
findWikidataId/fetchWikipediaFullText are kept but marked @deprecated.
- product-extractor.js extractProducts takes an options object bound to the site
identity, produces no products when nothing validates, and applies a
content-safety backstop: harmful categories are hard-dropped from unvalidated
(label-only) sources but kept-and-flagged (sensitive_category) for
P856-validated entities and the customer's own sitemap.
- index.js rewires the brand-name, competitor-summary and product call sites,
adds the BRAND_PROFILE_ENABLE_WIKI_PRODUCTS kill-switch (default off), and adds
a persist() guard that never overwrites a manual-curated product catalogue.
Adds unit tests (brand-resolver, entity validation, d*/e* regression fixtures
asserting no products and no by-name opensearch, manual-curated persist guard);
lint clean, full suite green, branch coverage 96.5% (>= 95% gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Mysticat review failed: Claude CLI crashed (exit 1): stderr= stdout={"type":"result","subtype":"error_max_budget_usd","duration_ms":5,"duration_api_ms":1456609,"is_error":true,"num_turns":1,"stop_reason":null,"session_id":"591d73c7-53df-4796-bf41-6e7cc38b91bf","total_cost_usd":11.888550450000004,"usage":{"input_tokens":0,"cache_creation_input_tokens":0,"cache_read_i |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Abstract
Binds the brand-profile agent's Wikipedia/Wikidata product lookups to a site-validated entity, replacing the decoupled by-name search that allowed one organisation's article to be extracted as another organisation's product catalogue.
2. Reasoning
LLMO-6580: the brand-profile agent fabricated defamatory product catalogues for real customers and government agencies — e.g. DHL, Dai Nippon Printing, and Dun & Bradstreet tagged as the "D-Company" (Dawood Ibrahim) organised-crime syndicate, and the Singapore Economic Development Board and Export Development Canada tagged with military/weapons content. Root cause: when a site had no brand name, the agent derived one from the domain and collapsed it to a 2–3 letter acronym; both the Wikidata entity search and a separate Wikipedia opensearch then fuzzy-matched a famous same-initials article, whose contents the LLM extracted as the customer's products. The failure was reproduced deterministically against the public APIs.
3. High-level overview of the changes
dev/www/store/supportsubdomain label, as the brand. A new resolver prefers the base profile's brand name, the site's own page title, and the registrable apex domain; a low-confidence result does not proceed to fuzzy lookups.opensearch "<name> company"calls are removed from both the product-extraction and competitor-inference paths; article text is fetched via the validated entity's Wikipedia sitelink.products_metadata.sourceismanual-curated, protecting the hand-verified catalogues already written to production under this ticket.4. Required information
6. Additional information outside the code
d*-acronym brand ("Dnp/Dhl/Dnb company") returns the "D-Company" (Dawood Ibrahim) article, and for ane*-acronym ("Edb/Edc/Eon company") returns "E Company, 506th Infantry Regiment" — the two fabrication clusters observed in production.products_metadata.source = manual-curated; the persist guard added in this PR exists to keep the pipeline from overwriting those rows.7. Test plan
BRAND_PROFILE_ENABLE_WIKI_PRODUCTS=truein Vault (dx_mysticat/{env}/task-processor) and run the agent against a short-acronym site (e.g. ad*domain): confirm it no longer fabricates an unrelated organisation's catalogue, and that a correctly-resolved brand yields its real products.manual-curatedrows untouched.8. Deployment & merge order
wikipedia_llm/hybridsites (separate follow-up) — must run only after this ships and the flag is enabled.BRAND_PROFILE_ENABLE_WIKI_PRODUCTSin Vault → run the P2 backfill.🤖 Generated with Claude Code