Project 24: Eigenvalue and Eigenvector Explorer

A self-contained deep-dive from the canonical Math from Foundations to Machine Learning curriculum.

  • Difficulty: Advanced
  • Time: 2-3 weeks
  • Language: Python with NumPy and Matplotlib
  • Prerequisites: Project 20
  • Source: ML-Math P06

What You Will Build

Build an interactive explorer that applies a 2x2 matrix repeatedly to points, vectors, and a grid while highlighting invariant directions. Users can enter a matrix, drag candidate vectors, and watch whether direction is preserved while length and orientation change. Implement the characteristic polynomial for 2x2 matrices and power iteration for a dominant eigenpair, then compare results with a trusted numerical library. Show real, repeated, complex, and defective cases without pretending every matrix has a full real eigenbasis. Residuals and iteration history must be visible so eigenpairs are validated rather than accepted because a plotted arrow “looks right.”

Real World Outcome

For a selected matrix, the tool displays eigenvalues, normalized eigenvectors, residual norms ||Av - λv||, and an animation of repeated transformation. A convergence panel graphs power-iteration estimates, while matrices with rotations or repeated eigenvalues produce an honest explanation of why real directions or convergence may be absent.

Core Question

Which directions survive a linear transformation without turning, and how can repeated multiplication reveal them?

Concepts You Must Understand First

  1. Eigenpair equation: Av = λv, with nonzero v; λ scales or reverses an invariant direction. See Axler, Linear Algebra Done Right.
  2. Characteristic polynomial: det(A - λI) = 0 identifies eigenvalue candidates for small matrices.
  3. Geometric multiplicity: repeated eigenvalues do not guarantee enough independent eigenvectors.
  4. Power iteration: repeated normalized multiplication tends toward a uniquely dominant eigenvector when its assumptions hold.
  5. Residual verification: a small ||Av - λv|| is stronger evidence than matching rounded output.

Build Milestones

  1. Transform a grid and arbitrary vectors while preserving original and transformed views.
  2. Solve 2x2 characteristic equations and compute corresponding real eigenvectors.
  3. Animate eigenvectors and ordinary vectors through repeated applications of A.
  4. Implement power iteration with normalization, sign-insensitive convergence, and iteration diagnostics.
  5. Catalog symmetric, diagonal, shear, rotation, repeated-eigenvalue, and defective examples.

Hints in Layers

  1. Begin with diagonal matrices, where eigenvectors are visible coordinate axes.
  2. Compare directions using absolute dot product because v and -v represent the same eigendirection.
  3. Separate “no real eigenvector,” “nonunique dominant magnitude,” and “defective matrix” diagnostics; they are different phenomena.

Common Pitfalls and Debugging

  • Symptom: power iteration alternates signs forever. Cause: eigenvector sign ambiguity or a negative dominant eigenvalue. Fix: compare |dot(v_new, v_old)| or residuals.
  • Symptom: normalization yields NaN. Cause: the transformed vector became zero. Fix: detect near-zero norm and restart or report the null-space case.
  • Symptom: rotation matrix produces bogus real arrows. Cause: complex eigenvalues were forced into real arithmetic. Fix: detect negative discriminant and explain the complex pair.

Definition of Done

  • Known diagonal and symmetric matrices produce correct eigenpairs.
  • Every reported pair includes a residual norm and normalized vector.
  • The animation distinguishes invariant from turning directions.
  • Power iteration exposes estimates, convergence tolerance, and failure reason.
  • Complex, repeated, and defective examples are handled explicitly.
  • Results are cross-checked against NumPy within documented tolerance.

Previous: Project 23 · Complete learning path · Next: Project 25