Project 8: Product Configuration DSL

Build a business-readable DSL for product constraints and recommendations, compiled safely into runtime rule artifacts.

Quick Reference

Attribute Value
Difficulty Level 4 (Expert)
Time Estimate 3-4 weeks
Main Programming Language Python (Alternatives: TypeScript, Rust, Go)
Coolness Level Level 4
Business Potential 4. Open Core Infrastructure
Prerequisites Projects 1-2, parser basics, semantic validation
Key Topics Grammar design, AST, semantic checks, rule compilation

1. Learning Objectives

  1. Design a DSL syntax business analysts can author.
  2. Parse DSL inputs into AST/IR representations.
  3. Validate semantic correctness and safety before publish.
  4. Compile rules into runtime-ready artifacts.

2. All Theory Needed (Per-Concept Breakdown)

Concept A: DSL Grammar and Semantic Passes

Fundamentals A DSL should be small, predictable, and domain-focused. Grammar alone is not enough; semantic validation must guarantee safe execution.

Deep Dive into the concept Use a multi-stage pipeline: parse, semantic validation, compile. Keep diagnostics precise with line/column and actionable fixes. Maintain grammar versions for backward compatibility.

How this fit on projects Primary here, directly supports P09-pricing-rules-dsl.md and P10-visual-rule-builder.md.

Definitions & key terms

  • Grammar
  • AST
  • Semantic validator
  • IR

Mental model diagram

DSL Text -> Lexer/Parser -> AST -> Semantic Pass -> IR -> Runtime Rules

How it works

  1. Parse input into AST.
  2. Validate references and rule legality.
  3. Compile into normalized IR.
  4. Publish only after simulation checks.

Minimal concrete example

requires "API_ACCESS" when option "SSO" selected

Common misconceptions

  • “If parser accepts it, it is valid.” Semantic checks may still fail.

Check-your-understanding questions

  1. Why separate parse and semantic stages?
  2. Why compile to IR before runtime?

Check-your-understanding answers

  1. Syntax validity does not guarantee business validity.
  2. IR normalizes execution and improves traceability.

Real-world applications Rule authoring for CPQ, feature flags, workflow conditions.

Where you’ll apply it This project and P10-visual-rule-builder.md.

References

  • “Language Implementation Patterns” by Terence Parr.
  • Lark and ANTLR docs.

Key insights Good DSLs reduce deployment bottlenecks without reducing control.

Summary Grammar + semantic validation + simulation creates safe business extensibility.

Homework/Exercises to practice the concept Write three DSL rules and intentionally break references for semantic diagnostics.

Solutions to the homework/exercises Implement validators for unknown option ids and circular dependencies.

3. Project Specification

3.1 What You Will Build

A parser/compiler toolchain for product rule DSL files with validation, error reporting, and runtime export.

3.2 Functional Requirements

  1. Parse rule files into AST.
  2. Validate semantic constraints.
  3. Generate normalized runtime rule objects.
  4. Export diagnostics and publish status.

3.3 Non-Functional Requirements

  • Performance: parse and validate 1,000 rules under 2 seconds.
  • Reliability: deterministic compilation output.
  • Usability: analyst-friendly errors.

3.4 Example Usage / Output

$ cpq dsl validate product_rules.dsl
status=failed errors=2
line 14: unknown option "API_ADDON"

3.5 Data Formats / Schemas / Protocols

  • DSL source file format.
  • AST JSON export format.
  • IR schema for runtime evaluator.

3.6 Edge Cases

  • Duplicate rule ids.
  • Circular dependency definitions.
  • Grammar version mismatch.

3.7 Real World Outcome

3.7.1 How to Run (Copy/Paste)

$ cpq dsl parse product_rules.dsl --out ast.json
$ cpq dsl validate product_rules.dsl
$ cpq dsl compile product_rules.dsl --out runtime_rules.json

3.7.2 Golden Path Demo (Deterministic)

Same DSL file should compile to byte-equivalent normalized IR.

3.7.3 If CLI: exact terminal transcript

$ cpq dsl validate product_rules.dsl
[ok] parsed_rules=128
[ok] semantic_checks=412
[result] status=pass

4. Solution Architecture

4.1 High-Level Design

Source File -> Parser -> AST -> Semantic Validator -> IR Compiler -> Rule Store

4.2 Key Components

| Component | Responsibility | Key Decisions | |———–|—————-|—————| | Parser | Build AST from grammar | deterministic grammar versioning | | Semantic Validator | business correctness | strict reference checks | | IR Compiler | runtime normalization | stable canonical ordering |

4.4 Data Structures (No Full Code)

AstNode { type, children, location }
IrRule { id, family, condition, action, metadata }

4.4 Algorithm Overview

  1. Lex and parse tokens.
  2. Run semantic checks.
  3. Compile to canonical IR.
  4. Persist and publish.

5. Implementation Guide

5.1 Development Environment Setup

$ cpq seed --scenario dsl-product
$ cpq test --suite dsl-product

5.2 Project Structure

dsl-product/
  src/
    grammar/
    parser/
    validator/
    compiler/

5.3 The Core Question You’re Answering

“How can business teams author configuration policy without creating unsafe runtime behavior?”

5.4 Concepts You Must Understand First

  • EBNF grammar basics
  • AST traversal
  • semantic validation

5.5 Questions to Guide Your Design

  • Which syntax constructs are intentionally excluded?
  • How will you communicate semantic errors to non-developers?

5.6 Thinking Exercise

Design three diagnostics that teach a user exactly how to fix an invalid rule.

5.7 The Interview Questions They’ll Ask

  1. How do you design a domain-specific language?
  2. How do you version grammar changes?
  3. How do you prevent invalid rules from reaching production?

5.8 Hints in Layers

  • Hint 1: keep first grammar minimal.
  • Hint 2: separate parse errors and semantic errors.
  • Hint 3: compile to canonical IR for stable outputs.

5.9 Books That Will Help

| Topic | Book | Chapter | |——-|——|———| | Language tooling | “Language Implementation Patterns” | Ch. 2-7 | | Safe architecture boundaries | “Clean Architecture” | Ch. 22 |

5.10 Implementation Phases

  • Phase 1: grammar + parser.
  • Phase 2: semantic checks.
  • Phase 3: compiler + publish pipeline.

5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale | |———-|———|—————-|———–| | Rule source | JSON only, DSL + JSON | DSL + IR | human + runtime needs | | Publish process | direct push, validated gate | validated gate | safety |

6. Testing Strategy

6.1 Test Categories

| Category | Purpose | Examples | |———-|———|———-| | Unit | parser tokens | grammar fixtures | | Semantic | business validation | unknown refs | | Regression | compile stability | canonical IR snapshots |

6.2 Critical Test Cases

  1. Valid DSL compile path.
  2. Semantic failure with actionable diagnostics.
  3. Canonical output stability across runs.

6.3 Test Data

product_rules_valid.dsl, product_rules_invalid.dsl, ir_expected.json.

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

| Pitfall | Symptom | Solution | |———|———|———-| | Overly complex grammar | low analyst adoption | simplify constructs | | No semantic phase | runtime failures | strict compile gate |

7.2 Debugging Strategies

  • Print AST with source spans.
  • Compare IR snapshots after grammar changes.

7.3 Performance Traps

Avoid repeated full-parse for tiny edits; cache token streams where safe.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add comments and metadata tags.
  • Add rule categories.

8.2 Intermediate Extensions

  • Linter with style guidance.
  • Rule dependency graph export.

8.3 Advanced Extensions

  • Incremental parser updates.
  • Schema compatibility checker between grammar versions.

9. Real-World Connections

9.1 Industry Applications

  • Business rule authoring in CPQ and policy systems.

9.3 Interview Relevance

Demonstrates compiler-thinking applied to business systems.

10. Resources

10.1 Essential Reading

  • “Language Implementation Patterns” by Terence Parr.

10.2 Video Resources

  • Parser and DSL architecture talks.

10.3 Tools & Documentation

  • Lark/ANTLR documentation.

11. Self-Assessment Checklist

  • I can explain grammar vs semantic validation.
  • I can produce stable IR outputs.
  • I can provide actionable diagnostics for analysts.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Parser and semantic validator work for core syntax.

Full Completion:

  • Compiler emits canonical IR.
  • Publish gate includes simulation checks.

Excellence (Going Above & Beyond):

  • Build a DSL linter and migration assistant.