Project 20: Matrix Systems and Visualization Studio

A self-contained deep-dive from the canonical Math from Foundations to Machine Learning curriculum.

  • Difficulty: Advanced
  • Time: 16-26 hours
  • Language: Python (alternatives: Julia, MATLAB/Octave, C++, Rust)
  • Prerequisites: Projects 3 and 16
  • Merged from: HS P17; ML-Math P04

What You Will Build

Build a matrix calculator and linear-system studio from first principles. Support shape-checked addition, scalar multiplication, matrix multiplication, transpose, determinant for small matrices, inverse when appropriate, and Gaussian/Gauss-Jordan elimination with recorded row operations. Solve Ax=b and classify unique, infinitely many, or inconsistent systems using rank/pivot structure rather than determinant alone. Visualize two-variable equations as lines and three-variable systems where practical, and show row operations step by step. Report residuals and a conditioning warning when small input perturbations cause large solution changes.

Real World Outcome

The CLI accepts matrices in a stable text/JSON format, prints dimensions, and shows each row-reduction step. A unique system returns its solution and ||Ax-b||; an underdetermined system reports free variables and a parameterized solution; an inconsistent one identifies a contradictory row. A 2D plot shows line intersection, coincidence, or parallelism, while a stress example demonstrates why an almost singular system can be numerically unreliable despite having a formal solution.

Core Question

How do matrix operations encode simultaneous equations, and what do pivots, rank, and conditioning reveal that a computed answer alone cannot?

Concepts You Must Understand First

  1. Matrix shape and operations: multiplication composes row-column dot products and requires compatible dimensions. See Orland, Math for Programmers.
  2. Elementary row operations: swap, scale by nonzero value, and add a multiple preserve the solution set.
  3. Echelon form, pivots, and rank: pivot structure controls constraints and free variables.
  4. Determinant and inverse: determinant detects invertibility for square matrices but is not the universal solving method.
  5. Residual versus conditioning: a small residual can coexist with a solution highly sensitive to input error.

Build Milestones

  1. Implement a validated matrix type and fundamental operations with explicit shape errors.
  2. Add elimination with partial pivoting and a replayable row-operation transcript.
  3. Classify augmented systems by ranks/pivots and return unique or parameterized solutions.
  4. Implement determinant/inverse as derived tools and verify identities/residuals.
  5. Add 2D visualizations, perturbation experiments, and comparisons with NumPy as an oracle.

Hints in Layers

  1. During elimination, choose the largest available absolute pivot in the column to reduce avoidable numerical error.
  2. Determine inconsistency from a row [0 ... 0 | nonzero]; free non-pivot columns imply infinitely many solutions.
  3. Test the implementation against exact integer/rational examples before ill-conditioned floating-point systems.

Common Pitfalls and Debugging

  • Symptom: matrix multiplication gives plausible wrong shapes/values. Cause: elementwise multiplication or swapped indices. Fix: assert dimensions and test each output as a row-column dot product.
  • Symptom: elimination divides by a tiny or zero pivot. Cause: no pivoting/tolerance policy. Fix: search lower rows, swap, and classify near-zero using documented scale-aware tolerance.
  • Symptom: every singular system is called inconsistent. Cause: singularity was equated with no solution. Fix: compare coefficient and augmented rank and expose free variables.

Definition of Done

  • Fundamental matrix operations enforce shape contracts and pass reference tests.
  • Elimination uses pivoting and produces a replayable row-operation trace.
  • Systems are correctly classified as unique, infinite, or inconsistent.
  • Solutions include residual checks; parameterized solutions include free variables.
  • Visual examples match algebraic classifications.
  • A perturbation case demonstrates conditioning/sensitivity and is documented.

Previous: Project 19 · Complete learning path · Next: Project 21