feat: kNN auto-router, compare tabs, per-assistant routing controls, PWA#54
Merged
Conversation
Pulls only the skip_first_rephrase=True change from the rephrase fix (feature/assistant-mention): when there's no conversation history, the chat flow searches the user's full question verbatim instead of running it through history_based_query_rephrase. Matches the Slack/one-shot flow, which already leaves the first query untouched, and avoids an unnecessary rephrase round-trip when there's no prior turn to fold in. (The softened HISTORY_QUERY_REPHRASE prompt from that same fix is intentionally NOT included here.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a Google-like Search experience: the user types a question and the system picks the right assistant automatically, instead of selecting one manually. Built behind a staged rollout gate (admin-only by default) since chat is now GA. Backend: - Settings.auto_search_rollout (off | admin_only | everyone, default admin_only). A new Settings field is absent from the persisted dict, so existing deployments land on admin_only automatically; flip in Admin -> Settings, no redeploy. - Persona.routing_instructions: router-only free-text metadata (migration c0d1e2f3a4b5, nullable). NEVER shown to users; the router falls back to `description` when blank. Wired through CreatePersonaRequest / PersonaSnapshot / upsert_persona + an admin-editor textarea. - secondary_llm_flows/assistant_router.py: build_router_catalog (from an ACL-filtered persona list) + route_question (one fast-LLM call -> best persona id + confidence). Fail-OPEN: ambiguity / parse failure / LLM error -> None, and the caller uses the all-source default persona. parse_route_response also accepts the gateway model's single-quoted Python-dict output (json.loads -> ast.literal_eval fallback). Unit tests included. - POST /query/auto-search: enforces the rollout gate on the trusted side (403), routes, re-checks the chosen persona's ACL with get_persona_by_id(..., user=), then answers via the existing one-shot get_search_answer. Honors the routed assistant's prompt + doc-set scope. Passes the authenticated user so the Q + A persist per-user (one_shot=True); returns answered_by + chat_message_id. Frontend: - New /auto-search page: single search box, no pickers; renders the answer + "Answered by X" + sources + 👍/👎 + text feedback (reuses the existing /chat/create-chat-message-feedback endpoint with the returned chat_message_id). Visibility gated by auto_search_rollout + user role (backend enforces for real). - Admin -> Settings: rollout selector. routing_instructions textarea in the assistant editor (labelled "not shown to users"). Validated end-to-end against prod personas + the live gateway LLM: routes Orchestrator/SRE/DU/AMERBenefits/help-ownership correctly and falls back for off-topic questions. The current Chat flow is untouched (purely additive). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rod→local clone tool Second wave on top of the auto-routed Search tab: Router quality + perf: - Route assistant selection with a configurable model (ASSISTANT_ROUTER_LLM_VENDOR + _MODEL); prod set to Claude Sonnet 4.5 (awsbedrock / anthropic.claude-sonnet-4-5-20250929-v1:0). Empty => default fast LLM. - parse_route_response also accepts the gateway's single-quoted Python-dict output (json.loads -> ast.literal_eval). build_router_catalog now takes a RoutableAssistant Protocol (Persona or PersonaSnapshot). - Router catalog reads the Redis persona cache (get_personas_for_user_cached), ACL-filtered + write-through busted on every assistant mutation, cross-worker. @mention direct invocation (Search tab): - Type @ to pick an assistant inline; sends persona_id so the router LLM step is SKIPPED and that assistant is invoked directly (still ACL-re-checked server side). lib/assistants/mentions.ts + tests. UX: - In-page Chat | Search tabs (ChatWithSearchTabs) — ChatPage untouched; renders it verbatim when the rollout doesn't grant the user Search. - Refreshed Search page: centered hero (empty) / top-anchored answer (active), brand mark + user menu, recent-search chips, rotating loading words. Removed the standalone /auto-search route (now a tab inside /chat). Tests/tooling: - test_auto_search_gate.py: the admin-only rollout gate (incl. default=admin_only). - scripts/clone_prod_to_local.py + docs/clone-prod-to-local.md: copy the latest N docs/source from prod Vespa (embeddings/ACLs/doc-set membership) + personas / prompts / document sets from the prod DB into a local env, for realistic local testing. Embedding-model parity enforced. AGENTS.md workflow pointer added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… vha-205) This branch had only pulled skip_first_rephrase, leaving the old keyword- compression HISTORY_QUERY_REPHRASE. Prod (vha-205) runs the softened natural-language version (the Pepsi/TAM retrieval fix); bring it here so a deploy from this branch doesn't regress it. Also commits the clone script's streaming+gzip export (avoids OOM on large sources). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… live) Deployed the auto-routed Search tab + Sonnet router + routing_instructions migration. Rollout gate defaults to admin_only, so it's dark to non-admins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Additive override in front of the LLM router: a new admin-editable Persona.routing_keywords (comma-separated phrases, migration d1e2f3a4b5c6). Before the Sonnet router runs, keyword_route() checks the question for any configured keyword (case-insensitive substring); on a match it routes straight to that assistant and skips the LLM (faster + free). No match => falls through to the existing LLM router unchanged, so existing behavior is untouched. Longest-match wins on multi-match. Wired through CreatePersonaRequest / PersonaSnapshot / upsert_persona + an admin-editor field. Unit tests included. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deterministic keyword pre-route for the auto-router + admin-editable routing_keywords field. Migration d1e2f3a4b5c6 applied on backend boot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r, recommended-assistant chips Router pipeline is now keyword -> (optional) semantic intent -> LLM instruction router. - keyword_route: unified fuzzy matcher — a keyword's words must all appear (any order/position, prefix>=4 for stems); wrap in "quotes" for an exact contiguous phrase (for abbreviations that are also common words, e.g. "as environment"). - Semantic intent tier: an LLM matches the question against assistants' routing_intents phrases in one call; gated behind Settings.auto_search_intent_enabled (default OFF). - Persona.routing_intents column (+ migration e2f3a4b5c6d7), threaded through create/upsert/snapshot; discrete-row editors for routing_keywords + routing_intents under a collapsed "[Recommended] Search Configuration" section, ordered by firing. - Auto-search: single-pick answer + "Recommended assistants" chips (ranks 2..N). - enable_assistant_creation Settings flag (default OFF) hides Create-assistant UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ckend=vha-208 web=vha-106 web-server had no probes + 1 replica, so a rollout could route to a booting pod. Added readiness + startup probes on :3000 (host-less httpGet -> pod IP; app returns 307, in the 200-399 success range) so the Service only serves the new pod once it's up and maxUnavailable rounds to 0 (old pod kept until new is Ready). Verified live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…anking
keyword_route now ranks matches by (priority, contiguous, exact-word-count,
num_words, char_len, -persona_id):
- "!" prefix on a keyword = always-wins; it outranks any non-priority match.
- Automatic quality tiebreak: a keyword whose words appear ADJACENT and EXACT
beats one matched scattered / via prefix — fixes cases like an AutomationSuite
question ("...Automation Suite... integrations... services") being grabbed by
IntegrationService's "integration service" on the incidental plurals.
Editor: per-keyword "Always wins" checkbox (round-trips the "!" prefix).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ty ranking) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Auto-routed Search tab gains a tabbed compare view: the top-1 answer plus a lazily-fetched second answer over the UNION of the router's top-N assistants' document sets (answered by a configurable model, default Sonnet). New endpoint /query/auto-search/union serves the union answer so neither answer blocks the other. Gated by Settings.auto_search_compare_enabled (default ON) with an admin toggle. Answer model + union persona/model are env-parameterised. Empty search box now teaches the "@Assistant" convention via a typewriter placeholder cycling real, accessible assistants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rail - Chat URL supports ?assistant=<name> (ACL-safe, resolved server-side against the user's accessible personas) so per-channel Slack "Ask Darwin" links can preselect an assistant; utm_source etc. pass through. - chat_referral table + POST /chat/referral + a fire-and-forget client beacon so inbound traffic (source/channel/assistant) is measurable with plain SQL. - process_message: build each persona tool defensively — a tool whose prerequisites aren't met is skipped with a WARNING log instead of 500-ing the whole message. - Remove ImageGenerationTool from BUILT_IN_TOOLS (unusable on this gateway-only deployment); harden load_builtin_tools to detach persona links before deleting a removed built-in tool so startup can never FK-crash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tant param + tool cleanup) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… polish - globals.css: add a :root light palette (cool #f1f3f7 canvas, white raised surfaces, near-black #1e2430 text, firmer borders) instead of the washed Tailwind gray fallbacks, and declare color-scheme:light so browsers with auto-dark stop force-inverting the light theme. Applies app-wide (chat + search + admin); dark mode untouched. - danswerbot /personas: match on display_name OR name and show friendly labels (aligned with the web UI); scope the persona list to the Slack user's access (all users still see every global/public assistant), fail-open on lookup error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…polish) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ting light mode Plain `color-scheme: light` isn't a strong enough opt-out for Chrome's "Auto Dark Mode for Web Contents"; `only light` explicitly forbids the browser from applying its own scheme, so the app's chosen light theme renders as designed even when the browser defaults to dark. `.dark` still overrides to dark mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…redirect-safe Unauthenticated hits on /chat?assistant=… now redirect to /auth/login?next=<path>; middleware stashes the validated next in a short-lived HttpOnly/Secure/SameSite=Lax cookie that survives the IdP round-trip; the OAuth/OIDC/SAML callbacks restore it (and clear it) so per-channel Slack links still preselect the assistant after login. next is validated with getSafeNextPath at BOTH boundaries (set + consume): a real URL parser accepts only same-origin root-relative paths under an allowlist (/chat), rejecting absolute/protocol-relative/backslash/encoded/userinfo/scheme/CRLF inputs. Verified locally end-to-end on the real OIDC flow + 35 validator unit tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…k user group) Opt-in per channel: bot answers get a red "Yet to be verified by an SME" button; a member of the channel's configured Slack user group (by name/@handle, resolved live via usergroups) can click it to turn it green "✅ Verified by an SME" with the verifier + date, so readers can trust the answer. Non-members are rejected; live membership means leavers lose access automatically. - ChannelConfig: enable_sme_validation + sme_group_name (UI-driven via the admin Slack bot config form; wired through SlackBotConfigCreationRequest + assembly). - Persistence: extend chat_feedback with sme_verified_by / sme_verified_at (migration a7b8c9d0e1f2) + mark_message_sme_verified; recorded on verify. - Authorization enforced server-side against the live Slack group membership. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ist + tests buildRequestBodyFromCreationRequest whitelists fields into the JSON body, so enable_sme_validation / sme_group_name were silently dropped on save (form + backend were correct; saves no-opped). Add both to the builder + interface, and a vitest asserting new config fields reach the serialized body on create (POST) and update (PATCH) so this class of bug is caught in future. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add web app manifest (app/manifest.ts, auto-served + auto-linked by App Router), icon set (192/512/maskable + apple-touch), and dark theme-color. Enables the desktop/mobile install prompt in Chrome/Edge and Add-to-Dock in Safari 17+. No service worker — modern Chromium no longer requires one for installability, so we keep the surface minimal. No backend or auth changes (middleware matcher is path-scoped, so manifest/icons serve publicly). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… tab Replace the LLM routing-instructions logic with a self-maintaining kNN router: keyword_route first, then nearest-neighbor over the existing Vespa slack chunk_id=0 corpus (labeled channel->persona via slack_bot_config), a similarity-weighted vote, a confidence gate, and an LLM tiebreak on the low-confidence band. build_router_catalog now includes every ACL persona. Add a third compare tab (shown on the tiebreak/ambiguous case): a source-scoped answer over HighSpot + the docs sites, lazy-fetched via /auto-search/sources. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the SME button from 'Yet to be verified by an SME' to 'Awaiting SME Review'. Support a comma-separated list of SME Slack user groups; a member of ANY listed group may verify (union of live memberships). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pin each mermaid diagram to a light base theme via an init directive so the diagrams (esp. the sequence diagram) stay readable in dark-mode viewers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… editor The kNN router made routing_instructions + routing_intents inert (only routing_keywords is still used). Remove both field UIs from the assistant editor and fix the section caption to describe the real flow (keywords, then the Slack-similarity fallback). DB columns + form payload left untouched so the change is reversible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… code Remove the now-dead fields from the persona request/response models, the create/update write path, and the frontend (editor initialValues, API body, interfaces). Only routing_keywords remains. The two Postgres columns are kept as inert nullable storage (no migration, no data loss) and marked deprecated in the ORM model — safe to drop later with a standalone migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add persona.is_router_candidate (bool, default true) so admins can exclude an assistant from the auto-routed Search tab — the router catalog now filters on it, removing the assistant from BOTH the keyword route and the kNN fallback while it stays manually selectable. Editable via a checkbox in the assistant editor's Search Configuration section. Safe additive migration (NOT NULL, server_default true) chained off the current head; existing assistants keep participating (no behavior change on upgrade). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…SME label, router flag, PWA) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an 'Auto-Route' column to /admin/assistants with a per-row On/Off toggle
(PATCH /admin/persona/{id}/router-candidate) so admins can enable/disable an
assistant's participation in auto-routing without opening each editor. Mirrors
the existing 'Is Visible' toggle; busts the persona cache so the router catalog
updates immediately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop eagerly firing the union ('All top matches') and source ('HighSpot & Docs')
answers after the primary answer. Each is now generated only when the user opens
its tab (once) — so if they're satisfied with the top match, the two extra LLM
calls are never made.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…le, on-click compare) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Only surface an assistant as a 'recommended assistant' (and in the compare union scope) when >= N neighbors matched it — dropping single, incidental kNN matches (the RTO-query noise where 1-vote channels surfaced irrelevant recs). The answering pick is unaffected. Threshold is SLACK_KNN_ROUTER_MIN_RECOMMENDATION_VOTES (env/configmap, default 2), added to prod env.properties. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ninja-shreyash
approved these changes
Jul 16, 2026
Sarath1018
approved these changes
Jul 16, 2026
swati354
approved these changes
Jul 16, 2026
Run the repo's pre-commit hooks (black, reorder-python-imports, autoflake, ruff, prettier) across the changed files to satisfy the PR quality checks. Formatting/import-order only — no logic changes. Also drops an unused import flagged by ruff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Auto-routed Search tab overhaul + assistant/Slack-bot improvements. All changes are live on prod (
backend=vha-216,web=vha-116).Routing (auto-routed Search tab)
keyword_route→ nearest-neighbor over the existing Vespa Slack corpus (chunk_id=0, labeledchannel → personaviaslack_bot_config) → similarity-weighted vote → confidence gate → LLM tiebreak on the low-confidence band. Self-maintaining (no per-assistant routing instructions to curate). Prototype: ~65% top-1 vote / ~70% + tiebreak / ~85% top-3.routing_instructions/routing_intentsfrom code (request/response models, write path, editor UI). Columns kept inert (no migration).is_router_candidateflag (default true) to exclude a catch-all assistant from auto-routing — with an inline Auto-Route toggle in/admin/assistants(no editor visit needed).SLACK_KNN_ROUTER_MIN_RECOMMENDATION_VOTES, default 2) to drop single, incidental matches from "recommended assistants".SLACK_KNN_ROUTER_TOP_K, prod=25).Search-tab compare UI
highspot+ docs sites).Slack bot
Web
Docs / config
SLACK_KNN_ROUTER_TOP_K,SLACK_KNN_ROUTER_MIN_RECOMMENDATION_VOTES.Migrations
31f30b318163—persona.is_router_candidate(additive,NOT NULL server_default true; safe on existing rows). Already applied on prod.Testing
tsc+next lint+vitestgreen.🤖 Generated with Claude Code