Chat: Improve deterministic intent routing with token-set matching while preserving semantic fallback

[goodtocode]

Title

Improve deterministic intent routing with token-set matching while preserving semantic fallback

Summary

The current intent classification pipeline is reliable for exact and near-exact phrase matches and capture-based commands, but it is brittle for natural-language variants that reorder words or add filler terms. This issue proposes adding deterministic token-set and co-occurrence matching such as find + actor + name as a middle layer between existing phrase and capture rules and embedding-based semantic fallback.

This keeps deterministic behavior predictable for critical routes while still using semantic classification for long-tail paraphrases.

Current implementation as-is

Tiered routing flow

Current routing behavior is:

1. Tier 1a: Rule-based deterministic classifier
   • Capture rules via PhraseCapture for parameterized patterns
   • Phrase examples via substring Contains matching
2. Tier 1b: Semantic classifier, only if Tier 1a misses and semantic is enabled
   • Embedding generation plus vector search plus threshold and topK
3. Tier 2: Agent fallback
   • Forced tool selection, then open agent turn if needed

Relevant files in this codebase

• src/Infrastructure.AgentFramework/Intents/RuleIntentClassifier.cs
• src/Infrastructure.AgentFramework/Intents/PhraseCapture.cs
• src/Infrastructure.AgentFramework/Intents/HybridIntentClassifier.cs
• src/Infrastructure.AgentFramework/Intents/SemanticIntentClassifier.cs
• src/Infrastructure.AgentFramework/Intents/DefaultIntentCatalogFactory.cs
• src/Infrastructure.AgentFramework/ChatMessageIntentRouter.cs
• src/Infrastructure.AgentFramework/Options/IntentClassificationOptions.cs
• src/Presentation.Api/appsettings.json
• src/Presentation.Api/appsettings.Development.json

Observed gap

Example-based deterministic matching uses literal substring containment. This misses equivalent variants such as:

• find this actor by name
• find my friend who is an actor with the name of

when only a phrase like find an actor by name is registered.

Problem statement

We currently need to add many phrase variants to cover common user wording permutations. This scales poorly and still leaves routing gaps, especially on critical intents such as actor, chat, and pipeline lookups, causing unnecessary fallback to semantic and agent behavior.

Proposal intent

Add a deterministic token-set intent rule mechanism to improve recall for known intents without sacrificing precision.

Goals:

• Better deterministic coverage of natural phrasing variants
• Fewer false fallthroughs to semantic and agent
• Preserve current architecture and fallback chain

Non-goals:

• Replacing semantic classification
• Replacing capture rules for structured parameters such as Guid values

Proposed implementation approach

1) Add token-set rule model to intents

Extend intent definitions with optional deterministic token criteria, for example:

• AllOf tokens required
• AnyOf token groups where at least one token from each group must match
• optional NoneOf tokens as blockers

Example concept for actor-by-name:

• AllOf includes actor and name or named
• AnyOf includes find or lookup or search

2) Add text normalization for token matching

Before token matching:

• lowercase
• punctuation removal
• whitespace normalization
• simple alias mapping such as named to name and conversations to conversation

Keep this lightweight and deterministic with no external dependencies.

3) Update RuleIntentClassifier evaluation order

Recommended order:

1. Capture rules in PhraseCapture for highest precision
2. Token-set rules for robust deterministic matching
3. Existing example substring rules for backward compatibility

This preserves structured capture behavior and compatibility with current catalogs.

4) Keep semantic fallback unchanged

HybridIntentClassifier behavior remains:

• rule match returns deterministic route
• no rule match plus semantic enabled runs embeddings search
• semantic miss or error falls through to agent path

5) Add ambiguity safeguards

If multiple token-set intents match:

• select highest specificity such as most required tokens satisfied
• optionally reject low-confidence ties and fall through to semantic

6) Initial target intents

Start with high-value intents with known phrase variance issues:

• actor lookup by name
• chat sessions and messages listing
• other intents currently requiring repeated phrase additions

Why this approach

• Deterministic logic remains testable and predictable
• Embeddings remain a fallback for true long-tail phrasing
• Reduces catalog bloat from near-duplicate example phrases
• Portable pattern for sibling codebases with minimal architecture drift

Acceptance criteria

• New token-set rule mechanism exists in intent model and classifier
• Existing capture and example behavior remains backward compatible
• Known variant prompts route deterministically without semantic dependency
• No increase in false-positive routing for unrelated intents
• Semantic fallback still executes on deterministic misses
• Build passes and relevant routing tests pass

Test plan

Unit tests

• token normalization
• token-set matching for AllOf, AnyOf, and NoneOf
• tie-break behavior
• classifier order capture then token-set then example

Integration tests

• HybridIntentClassifier fallback behavior with semantic on and off
• intent-to-route mapping unchanged for existing deterministic cases

End-to-end chat routing tests

• add prompts with reordered words and filler text
• validate deterministic route output shape such as tool-result table or expected deterministic response
• validate miss path still reaches semantic and agent fallback

Rollout notes for multiple codebases

Because two sibling repositories share this pattern, implement in this order:

1. add token-set primitives and classifier logic
2. add tests
3. migrate one to two intents first as a pilot
4. measure miss and fallthrough rate
5. migrate remaining intents incrementally

Optional: use a feature flag for safe rollout.

Risks and tradeoffs

• over-broad token rules can cause false positives
• too many aliases can blur intent boundaries
• tie-break logic must stay deterministic and documented

Mitigations:

• require domain-anchor tokens for critical intents such as actor and chat
• start narrow and expand using test evidence
• log matched rule type and intent for observability