Project 21: Matrix Transformations and 3D Rendering Engine

A self-contained deep-dive from the canonical Math from Foundations to Machine Learning curriculum.

  • Difficulty: Advanced
  • Time: 3-5 weeks
  • Language: Python with NumPy and Pygame or Matplotlib
  • Prerequisites: Projects 16 and 20
  • Merged from: HS P10; Prog P02; ML-Math P05; ML-Found P02

What You Will Build

Build a staged graphics engine that makes every matrix operation visible. Begin with a 2D image or polygon editor supporting translation, scale, rotation, reflection, shear, and a four-point perspective warp computed as a homography. Add homogeneous coordinates so affine and projective transformations compose into matrices, then graduate to a 3D wireframe pipeline: model-space vertices, world and camera transforms, perspective projection, clipping, and screen coordinates. The interface must display the current matrix and transformation order beside the rendered result. Include both nearest-neighbor and bilinear image resampling, plus a rotatable cube or loaded mesh. This combines the sources’ image-transformation lab, matrix visualizer, and renderer into one coherent artifact rather than several nearly identical demos.

Real World Outcome

A learner can load an image, drag its four corners onto a photographed sign or screen, and see the source rectified through a computed 3x3 homography. The same interface can load a cube, issue rotate, scale, translate, and camera commands, and show before/after views with the composed 3x3 or 4x4 matrix. A saved diagnostic frame shows correspondence points, axes, transformed vertices, clipped edges, projection parameters, and the exact pipeline stage where each coordinate lives.

Core Question

How can one matrix pipeline turn mathematical coordinates into a correctly transformed image on a flat screen, and why does changing multiplication order change the picture?

Concepts You Must Understand First

  1. Linear maps and basis vectors: a matrix’s columns show where basis directions move. See Paul Orland, Math for Programmers, chs. 4-6.
  2. Homogeneous coordinates: translation becomes matrix multiplication by appending a coordinate. See Gabriel Gambetta, Computer Graphics from Scratch, transformation chapters.
  3. Composition order: column-vector pipelines apply the rightmost transform first; matrix multiplication is not commutative.
  4. Perspective projection: divide camera-space x and y by depth, while guarding the near plane. See Dunn and Parberry, 3D Math Primer for Graphics and Game Development.
  5. Image resampling: inverse-map destination pixels to source pixels; bilinear interpolation reduces holes and jaggedness.
  6. Projective homographies: four non-collinear point correspondences determine a planar perspective warp up to scale; degenerate point layouts must be rejected.

Build Milestones

  1. Draw axes and transform 2D points with tested rotation, scale, reflection, and shear matrices.
  2. Add 3x3 homogeneous translation, composition, inverse mapping, and image interpolation.
  3. Solve a four-point homography, inverse-map every destination pixel through it, and rectify a perspective-distorted planar image.
  4. Represent a 3D mesh as vertices and edges; implement 4x4 model, view, and projection matrices.
  5. Clip unsafe geometry, perform the perspective divide, map normalized coordinates to the viewport, animate a cube, and expose every pipeline stage in an inspector.

Hints in Layers

  1. Start with a triangle whose coordinates you can calculate by hand; draw its local and transformed axes.
  2. Keep model, view, and projection separate until display time, and label vector conventions explicitly.
  3. For affine and homography image warps, iterate destination pixels and apply the inverse transform; reject nearly collinear four-point inputs. For 3D, clip vertices at or behind the near plane before dividing by depth.

Common Pitfalls and Debugging

  • Symptom: rotate-then-translate behaves like translate-then-rotate. Cause: matrices were composed in the wrong order. Fix: test two deliberately noncommuting transforms on one point and document the convention.
  • Symptom: transformed images have gaps. Cause: forward pixel mapping. Fix: inverse-map every destination pixel and interpolate.
  • Symptom: geometry explodes or flips near the camera. Cause: perspective division by zero or negative depth. Fix: inspect camera-space z and clip against a positive near plane.

Definition of Done

  • Unit tests verify identity, inverse, composition order, and known 90-degree rotations.
  • The 2D tool transforms and saves both polygons and raster images.
  • Bilinear interpolation visibly improves a rotated checkerboard over nearest-neighbor sampling.
  • A four-point homography rectifies a perspective-distorted test image, rejects degenerate correspondences, and matches its corner targets within tolerance.
  • A cube can be translated, rotated, camera-transformed, clipped, and perspective-projected.
  • The UI exposes matrices and intermediate coordinates, not only the final picture.
  • A short report explains coordinate conventions and three diagnosed failure cases.

Previous: Project 20 · Complete learning path · Next: Project 22