Project 2: High-Precision Interaction Model Lab
Harden intents, slots, and repair prompts until quality becomes measurable and repeatable.
Quick Reference
| Attribute | Value |
|---|---|
| Difficulty | Level 2 (Intermediate) |
| Time Estimate | 1 week |
| Main Programming Language | TypeScript |
| Alternative Programming Languages | Python, Kotlin |
| Key Topics | Intent overlap, slot normalization, repair ladders |
1. Learning Objectives
- Reduce intent confusion rate with systematic utterance curation.
- Build a slot normalization pipeline with validation checkpoints.
- Measure prompt quality using conversion by repair level.
2. All Theory Needed (Per-Concept Breakdown)
Concept A: Intent Collision Management
- Fundamentals: intents should represent user goals, not sentence patterns.
- How it works: compute confusion matrix on evaluation utterances.
- Failure modes: too many intents, semantic overlap, low data variety.
Concept B: Repair Ladder Design
- L1: paraphrase and clarify one missing field.
- L2: constrained choices.
- L3: graceful escape with next action.
3. Architecture and Build Plan
- Build evaluation corpus of top user phrases.
- Instrument confusion and slot quality metrics.
- Refactor prompts for one critical transaction intent.
4. Validation and Testing
- Confusion rate below threshold.
- No repeated fallback loops.
- Locale parity on key flows.
5. Troubleshooting
- Symptom: high recognition, low completion.
- Fix: shorten prompts and add explicit next-step language.
6. Deliverables
- NLU quality dashboard.
- Prompt variant experiment results.
- Repair ladder state diagram.
7. Stretch Goals
- Add automatic prompt linting for clarity constraints.