LEARN VIRTUAL REALITY FROM SCRATCH

Learn Virtual Reality From Scratch

Goal: Deeply understand virtual reality technology—from the mathematics of 3D orientation to building complete VR systems. Master stereoscopic rendering, head tracking, spatial audio, and the techniques that make VR feel real.

Why Virtual Reality Matters

VR represents one of the most challenging intersections of technology: real-time graphics, sensor fusion, human perception, and low-latency systems. Understanding VR deeply means understanding:

3D Graphics at a level most developers never reach
Sensor Fusion combining accelerometers, gyroscopes, and cameras
Human Perception how our vestibular and visual systems work together
Low-Latency Systems where every millisecond matters
Spatial Computing the foundation of future interfaces

After completing these projects, you will:

Understand every component of a VR headset
Build head tracking systems using IMUs
Render stereoscopic 3D with lens distortion correction
Implement spatial audio that sounds like real 3D space
Create VR applications using OpenXR
Know why VR causes motion sickness and how to prevent it

Core Concept Analysis

The VR System Pipeline

┌─────────────────────────────────────────────────────────────┐
│                     SENSING LAYER                           │
│  ┌─────────┐  ┌─────────────┐  ┌──────────────┐            │
│  │   IMU   │  │   Cameras   │  │ Controllers  │            │
│  │(Rotation)│  │ (Position)  │  │   (Input)    │            │
│  └────┬────┘  └──────┬──────┘  └──────┬───────┘            │
└───────┼──────────────┼────────────────┼────────────────────┘
        │              │                │
        ▼              ▼                ▼
┌─────────────────────────────────────────────────────────────┐
│                   PROCESSING LAYER                          │
│  ┌─────────────┐  ┌────────────┐  ┌────────────────┐       │
│  │Sensor Fusion│  │   SLAM     │  │ Input Mapping  │       │
│  │ (Madgwick)  │  │(Positional)│  │   (Actions)    │       │
│  └──────┬──────┘  └─────┬──────┘  └───────┬────────┘       │
└─────────┼───────────────┼─────────────────┼────────────────┘
          │               │                 │
          ▼               ▼                 ▼
┌─────────────────────────────────────────────────────────────┐
│                   SIMULATION LAYER                          │
│  ┌──────────┐  ┌──────────────┐  ┌───────────────┐         │
│  │ Physics  │  │ Game Logic   │  │ Audio Engine  │         │
│  └────┬─────┘  └──────┬───────┘  └───────┬───────┘         │
└───────┼───────────────┼──────────────────┼─────────────────┘
        │               │                  │
        ▼               ▼                  ▼
┌─────────────────────────────────────────────────────────────┐
│                   RENDERING LAYER                           │
│  ┌─────────────────┐  ┌───────────────┐  ┌──────────────┐  │
│  │ Stereo Rendering│  │Lens Distortion│  │  Timewarp    │  │
│  │  (Two Views)    │  │  Correction   │  │(Reprojection)│  │
│  └────────┬────────┘  └───────┬───────┘  └──────┬───────┘  │
└───────────┼───────────────────┼─────────────────┼──────────┘
            │                   │                 │
            ▼                   ▼                 ▼
┌─────────────────────────────────────────────────────────────┐
│                    DISPLAY LAYER                            │
│  ┌───────────────────────────────────────────────────────┐ │
│  │              VR Headset Display (90+ Hz)              │ │
│  │           ┌─────────────┬─────────────┐               │ │
│  │           │  Left Eye   │  Right Eye  │               │ │
│  │           └─────────────┴─────────────┘               │ │
│  └───────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Fundamental Concepts

Degrees of Freedom (DOF):
- 3DOF: Rotation only (pitch, yaw, roll) - like turning your head in place
- 6DOF: Rotation + Translation (x, y, z) - full movement in space
- Most modern VR requires 6DOF for presence
Stereoscopic Vision:
- Each eye sees a slightly different view (interpupillary distance ~64mm)
- The brain fuses these into depth perception
- Render the scene twice from two offset camera positions
Motion-to-Photon Latency:
- Time from physical movement to pixels updating
- Must be < 20ms to prevent nausea
- < 10ms for comfortable experience
- Components: sensor reading + processing + render + display
Quaternions:
- 4-component rotation representation: q = (w, x, y, z)
- No gimbal lock (unlike Euler angles)
- Smooth interpolation (SLERP)
- Efficient composition
Sensor Fusion:
- Gyroscope: fast but drifts
- Accelerometer: stable but noisy
- Magnetometer: absolute heading but susceptible to interference
- Combine all three for robust orientation
Lens Distortion:
- VR lenses create pincushion distortion
- Pre-distort the image with barrel distortion
- Also correct chromatic aberration (color fringing)
Spatial Audio (HRTF):
- Head-Related Transfer Functions
- Each ear receives sound differently based on direction
- Time delay (ITD) and level difference (ILD)
- Creates convincing 3D sound localization
Presence vs Immersion:
- Immersion: Technical quality (resolution, tracking, FOV)
- Presence: Psychological feeling of “being there”
- Good VR maximizes both

Project List

Projects progress from mathematical foundations to complete VR systems.

Project 1: 3D Vector and Matrix Library

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: Rust, C++, Go
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 1: Beginner
Knowledge Area: Linear Algebra / 3D Mathematics
Software or Tool: Pure C, no libraries
Main Book: 3D Math Primer for Graphics and Game Development by Fletcher Dunn & Ian Parberry

What you’ll build: A math library with 3D vectors (add, subtract, dot, cross, normalize), 4x4 matrices (multiply, inverse, transpose), and transformation matrices (translate, rotate, scale, perspective, lookAt).

Why it teaches VR: Every VR calculation uses these primitives. Head position, camera view, object transforms—all rely on vectors and matrices. Building your own ensures deep understanding.

Core challenges you’ll face:

Matrix multiplication order → maps to transformation composition
Inverse matrix calculation → maps to world-to-view transforms
Floating-point precision → maps to numerical stability
Handedness conventions → maps to left vs right-handed coordinate systems

Key Concepts:

Vector Operations: 3D Math Primer Chapter 2 - Dunn & Parberry
Matrix Transforms: 3D Math Primer Chapter 5 - Dunn & Parberry
Perspective Projection: Computer Graphics from Scratch Chapter 9 - Gabriel Gambetta
Coordinate Systems: Real-Time Rendering Chapter 4 - Akenine-Möller et al.

Difficulty: Beginner Time estimate: 1 week Prerequisites: Basic algebra, understanding of 3D space

Real world outcome:

$ ./math_test
Vector3 tests:
  v1 = (1, 2, 3)
  v2 = (4, 5, 6)
  v1 + v2 = (5, 7, 9)
  v1 · v2 = 32 (dot product)
  v1 × v2 = (-3, 6, -3) (cross product)
  |v1| = 3.742 (magnitude)
  normalize(v1) = (0.267, 0.535, 0.802)

Matrix4 tests:
  Identity matrix: I × M = M ✓
  Rotation 90° around Y:
    (1, 0, 0) → (0, 0, -1) ✓
  Perspective projection test: ✓
  Matrix inverse: M × M⁻¹ = I ✓

All 24 tests passed!

Implementation Hints:

Vector structure:

typedef struct {
    float x, y, z;
} Vec3;

typedef struct {
    float x, y, z, w;
} Vec4;

Matrix as column-major (OpenGL convention):

typedef struct {
    float m[16];  // Column-major: m[col*4 + row]
} Mat4;

// Access: m[column][row] = mat.m[column * 4 + row]

Essential functions:

Vec3 vec3_add(Vec3 a, Vec3 b);
Vec3 vec3_sub(Vec3 a, Vec3 b);
Vec3 vec3_scale(Vec3 v, float s);
float vec3_dot(Vec3 a, Vec3 b);
Vec3 vec3_cross(Vec3 a, Vec3 b);
float vec3_length(Vec3 v);
Vec3 vec3_normalize(Vec3 v);

Mat4 mat4_identity(void);
Mat4 mat4_multiply(Mat4 a, Mat4 b);
Mat4 mat4_translate(float x, float y, float z);
Mat4 mat4_rotate_x(float radians);
Mat4 mat4_rotate_y(float radians);
Mat4 mat4_rotate_z(float radians);
Mat4 mat4_scale(float x, float y, float z);
Mat4 mat4_perspective(float fov, float aspect, float near, float far);
Mat4 mat4_look_at(Vec3 eye, Vec3 target, Vec3 up);
Vec4 mat4_transform(Mat4 m, Vec4 v);

Questions to explore:

Why does transformation order matter (rotate then translate ≠ translate then rotate)?
What’s the difference between row-major and column-major matrices?
Why use homogeneous coordinates (w component)?
How does the perspective matrix create the “divide by z” effect?

Learning milestones:

Vector operations work correctly → Basic math works
Rotation matrices produce correct results → Trigonometry correct
Matrix inverse validates with M × M⁻¹ = I → Inverse works
LookAt matrix points camera correctly → View transform works

Project 2: Quaternion Library and Rotations

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: Rust, C++, Go
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: 3D Mathematics / Rotations
Software or Tool: Your vector library from Project 1
Main Book: 3D Math Primer for Graphics and Game Development by Fletcher Dunn & Ian Parberry

What you’ll build: A quaternion library for 3D rotations—multiplication, normalization, SLERP interpolation, conversion to/from Euler angles and rotation matrices.

Why it teaches VR: VR headsets report orientation as quaternions. All head tracking, controller orientation, and smooth rotation blending uses quaternions. Understanding them deeply is essential.

Core challenges you’ll face:

Quaternion multiplication → maps to combining rotations
SLERP interpolation → maps to smooth animation and filtering
Euler angle conversion → maps to human-readable angles
Gimbal lock understanding → maps to why quaternions are necessary

Key Concepts:

Quaternion Fundamentals: 3D Math Primer Chapter 8 - Dunn & Parberry
SLERP: Game Engine Architecture Chapter 4 - Jason Gregory
Euler Angles and Gimbal Lock: 3Blue1Brown - Quaternions
Rotation Representations: Real-Time Rendering Chapter 4 - Akenine-Möller et al.

Difficulty: Intermediate Time estimate: 1 week Prerequisites: Project 1 (Vector/Matrix Library), trigonometry

Real world outcome:

$ ./quaternion_test
Quaternion tests:
  q1 = 90° rotation around Y axis
  q2 = 90° rotation around X axis
  q1 × q2 = Combined rotation

  Rotating vector (1, 0, 0):
    by q1: (0, 0, -1) ✓
    by q2: (1, 0, 0) → (0, 1, 0) ✓

  SLERP interpolation (t=0.5):
    q1 → q2: smooth intermediate rotation ✓

  Conversion tests:
    Quaternion → Matrix → Quaternion: identical ✓
    Quaternion → Euler → Quaternion: identical ✓

  Gimbal lock demonstration:
    Euler (90°, 90°, 0°) loses a degree of freedom
    Quaternion: no gimbal lock ✓

All tests passed!

Implementation Hints:

Quaternion structure:

typedef struct {
    float w;  // Scalar part (real)
    float x, y, z;  // Vector part (imaginary)
} Quat;

Key operations:

// Identity (no rotation)
Quat quat_identity(void) {
    return (Quat){1, 0, 0, 0};
}

// Create from axis-angle
Quat quat_from_axis_angle(Vec3 axis, float radians) {
    float half = radians * 0.5f;
    float s = sinf(half);
    Vec3 n = vec3_normalize(axis);
    return (Quat){
        cosf(half),
        n.x * s, n.y * s, n.z * s
    };
}

// Quaternion multiplication (rotation composition)
Quat quat_multiply(Quat a, Quat b) {
    return (Quat){
        a.w*b.w - a.x*b.x - a.y*b.y - a.z*b.z,
        a.w*b.x + a.x*b.w + a.y*b.z - a.z*b.y,
        a.w*b.y - a.x*b.z + a.y*b.w + a.z*b.x,
        a.w*b.z + a.x*b.y - a.y*b.x + a.z*b.w
    };
}

// Rotate a vector
Vec3 quat_rotate(Quat q, Vec3 v);

// Spherical linear interpolation
Quat quat_slerp(Quat a, Quat b, float t);

// Convert to 4x4 rotation matrix
Mat4 quat_to_matrix(Quat q);

SLERP formula:

slerp(q0, q1, t) = (sin((1-t)θ) / sin(θ)) * q0 + (sin(tθ) / sin(θ)) * q1
where θ = arccos(q0 · q1)

Questions to explore:

Why does quaternion multiplication represent rotation composition?
What happens if you interpolate quaternions linearly (LERP) instead of SLERP?
Why must quaternions be normalized?
How do you find the shortest rotation path between two orientations?

Learning milestones:

Axis-angle creation works → Basic construction
Multiplication composes rotations → Core operation
SLERP is smooth → Interpolation works
Matrix conversion round-trips → Representation conversions

Project 3: IMU Sensor Reader (Arduino/ESP32)

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C/Arduino
Alternative Programming Languages: MicroPython, Rust (embedded)
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 2: Intermediate
Knowledge Area: Embedded Systems / Sensors
Software or Tool: Arduino/ESP32, MPU-6050 or BNO055 IMU
Main Book: Make: Sensors by Tero Karvinen

What you’ll build: Read raw accelerometer and gyroscope data from an IMU sensor, transmit it to a PC over USB serial, and visualize the raw values in real-time.

Why it teaches VR: VR headsets contain IMUs for rotation tracking. Understanding raw sensor data—noise, drift, bias—prepares you for sensor fusion. This is how Quest and other headsets track head rotation.

Core challenges you’ll face:

I2C communication → maps to sensor bus protocols
Sensor calibration → maps to bias and scale correction
Sample rate management → maps to timing and consistency
Understanding sensor noise → maps to why fusion is needed

Key Concepts:

IMU Basics: Make: Sensors Chapter 11 - Karvinen
I2C Protocol: Arduino I2C documentation
MPU-6050 Registers: InvenSense MPU-6050 datasheet
Sensor Characteristics: Adafruit IMU Guide

Difficulty: Intermediate Time estimate: 1 week Prerequisites: Basic electronics, Arduino familiarity

Real world outcome:

Arduino Serial Monitor:
Sample Rate: 500 Hz
Calibration: Done (bias: ax=-0.02, ay=0.01, az=-0.03)

Accelerometer (g): x=-0.012  y=0.008  z=1.002
Gyroscope (°/s):   x=0.32   y=-0.45  z=0.12

[Tilt the sensor]
Accelerometer (g): x=0.707  y=0.000  z=0.707
Gyroscope (°/s):   x=45.2   y=0.8    z=-1.2

[Rotate the sensor]
Gyroscope (°/s):   x=2.1    y=180.5  z=-3.2

$ python visualizer.py
[3D cube on screen rotates as you rotate the physical IMU]

Implementation Hints:

Hardware setup:

MPU-6050 → Arduino/ESP32
  VCC → 3.3V
  GND → GND
  SDA → A4 (or GPIO21 on ESP32)
  SCL → A5 (or GPIO22 on ESP32)

Basic reading code:

#include <Wire.h>

#define MPU_ADDR 0x68

void setup() {
    Wire.begin();
    Serial.begin(115200);

    // Wake up MPU-6050
    Wire.beginTransmission(MPU_ADDR);
    Wire.write(0x6B);  // PWR_MGMT_1 register
    Wire.write(0);     // Wake up
    Wire.endTransmission();
}

void loop() {
    int16_t ax, ay, az, gx, gy, gz;

    Wire.beginTransmission(MPU_ADDR);
    Wire.write(0x3B);  // Starting register
    Wire.endTransmission(false);
    Wire.requestFrom(MPU_ADDR, 14);

    ax = Wire.read() << 8 | Wire.read();
    ay = Wire.read() << 8 | Wire.read();
    az = Wire.read() << 8 | Wire.read();
    Wire.read(); Wire.read();  // Temperature
    gx = Wire.read() << 8 | Wire.read();
    gy = Wire.read() << 8 | Wire.read();
    gz = Wire.read() << 8 | Wire.read();

    // Convert to physical units
    float accel_scale = 16384.0;  // ±2g range
    float gyro_scale = 131.0;     // ±250°/s range

    Serial.print(ax / accel_scale); Serial.print(",");
    Serial.print(ay / accel_scale); Serial.print(",");
    Serial.print(az / accel_scale); Serial.print(",");
    Serial.print(gx / gyro_scale); Serial.print(",");
    Serial.print(gy / gyro_scale); Serial.print(",");
    Serial.println(gz / gyro_scale);

    delay(2);  // ~500 Hz
}

Questions to explore:

Why does the accelerometer read ~1g on Z axis when flat?
What happens to gyroscope readings when stationary (hint: drift)?
How do you calibrate the sensor bias?
What’s the difference between MPU-6050 (6DOF) and MPU-9250 (9DOF)?

Learning milestones:

Sensor communicates over I2C → Hardware works
Readings change when you move sensor → Data is valid
You understand gyro drift → See why fusion is needed
Accelerometer shows gravity direction → Understand reference frame

Project 4: Sensor Fusion with Madgwick Filter

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust, Python
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 3: Advanced
Knowledge Area: Signal Processing / Sensor Fusion
Software or Tool: IMU from Project 3, your quaternion library
Main Book: Sebastian Madgwick’s original paper

What you’ll build: Implement the Madgwick sensor fusion algorithm to combine accelerometer and gyroscope data into a stable quaternion orientation that doesn’t drift.

Why it teaches VR: This is exactly how VR headsets compute head rotation. The Madgwick filter is used in many production systems. Understanding it means understanding VR tracking.

Core challenges you’ll face:

Gyroscope integration drift → maps to why accelerometer correction is needed
Gradient descent optimization → maps to how Madgwick corrects orientation
Filter tuning (beta parameter) → maps to responsiveness vs stability trade-off
Quaternion derivative → maps to how rotation rate updates orientation

Key Concepts:

Madgwick Filter: Original Paper
Sensor Fusion Overview: QSense Motion
Quaternion Calculus: 3D Math Primer Chapter 8 - Dunn & Parberry
AHRS Systems: AHRS Python Documentation

Resources for key challenges:

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Project 2 (Quaternions), Project 3 (IMU Reader)

Real world outcome:

$ ./sensor_fusion_demo
Connected to IMU on /dev/ttyUSB0
Madgwick filter initialized (beta=0.1, sample_rate=500Hz)

Orientation (quaternion): w=1.000, x=0.000, y=0.000, z=0.000
Orientation (Euler): pitch=0.0°, roll=0.0°, yaw=0.0°

[Rotate sensor 90° right]
Orientation (quaternion): w=0.707, x=0.000, y=0.707, z=0.000
Orientation (Euler): pitch=0.0°, roll=0.0°, yaw=90.0°

[Leave sensor still for 60 seconds]
Drift observed: < 0.1° (vs 15°+ without fusion!)

[3D visualization shows a cube that perfectly follows your hand movements]

Implementation Hints:

Madgwick filter core algorithm (simplified):

typedef struct {
    Quat q;          // Current orientation
    float beta;      // Filter gain (0.01 - 0.5)
    float sample_period;
} MadgwickFilter;

void madgwick_update(MadgwickFilter *f,
                     float gx, float gy, float gz,  // Gyro (rad/s)
                     float ax, float ay, float az)  // Accel (normalized)
{
    Quat q = f->q;

    // Normalize accelerometer
    float norm = sqrtf(ax*ax + ay*ay + az*az);
    ax /= norm; ay /= norm; az /= norm;

    // Gradient descent step to align accelerometer with gravity
    float f1 = 2*(q.x*q.z - q.w*q.y) - ax;
    float f2 = 2*(q.w*q.x + q.y*q.z) - ay;
    float f3 = 2*(0.5f - q.x*q.x - q.y*q.y) - az;

    // Jacobian (partial derivatives)
    float J_11or24 = 2*q.y, J_12or23 = 2*q.z;
    float J_13or22 = 2*q.w, J_14or21 = 2*q.x;
    float J_32 = 2*J_14or21, J_33 = 2*J_11or24;

    // Gradient
    Quat grad = {
        .w = J_14or21*f2 - J_11or24*f1,
        .x = J_12or23*f1 + J_13or22*f2 - J_32*f3,
        .y = J_12or23*f2 - J_33*f3 - J_13or22*f1,
        .z = J_14or21*f1 + J_11or24*f2
    };
    grad = quat_normalize(grad);

    // Rate of change from gyroscope
    Quat q_dot = quat_scale(
        quat_multiply(q, (Quat){0, gx, gy, gz}),
        0.5f
    );

    // Apply gradient descent correction
    q_dot.w -= f->beta * grad.w;
    q_dot.x -= f->beta * grad.x;
    q_dot.y -= f->beta * grad.y;
    q_dot.z -= f->beta * grad.z;

    // Integrate
    q.w += q_dot.w * f->sample_period;
    q.x += q_dot.x * f->sample_period;
    q.y += q_dot.y * f->sample_period;
    q.z += q_dot.z * f->sample_period;

    f->q = quat_normalize(q);
}

Beta parameter tuning:

Higher beta (0.5): More accelerometer trust, stable but sluggish
Lower beta (0.01): More gyro trust, responsive but may drift
Typical VR: 0.033 to 0.1

Questions to explore:

Why use gradient descent instead of a Kalman filter?
How does the magnetometer extend this to 9DOF?
What happens during rapid rotation (accelerometer less reliable)?
How do you detect when the sensor is in free-fall?

Learning milestones:

Filter produces stable orientation → Basic fusion works
No drift over 60+ seconds → Accelerometer correction works
Responsive to fast motion → Beta tuned correctly
Euler angles are intuitive → Conversion works

Project 5: USB HID Head Tracker

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C/Arduino
Alternative Programming Languages: Rust (embedded)
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: USB / HID Protocols
Software or Tool: Arduino Pro Micro (ATmega32U4) or ESP32-S3
Main Book: USB Complete by Jan Axelson

What you’ll build: A head tracker that appears as a USB HID device (like a joystick), sending orientation data that games can read directly without special drivers.

Why it teaches VR: This is how many DIY VR trackers work, including Relativty. Understanding USB HID and how to present sensor data as standard input is key to creating VR peripherals.

Core challenges you’ll face:

USB HID descriptors → maps to defining the device’s capabilities
Report structure → maps to formatting data for the host
Polling rate → maps to latency vs bandwidth trade-off
Axis range and precision → maps to 16-bit vs 8-bit axes

Key Concepts:

USB HID Class: USB HID specification
Arduino USB Libraries: Arduino HID-Project documentation
Device Descriptors: USB Complete Chapter 7 - Axelson
DIY VR Tracking: Relativty Project

Difficulty: Advanced Time estimate: 1-2 weeks Prerequisites: Project 4 (Sensor Fusion), USB basics

Real world outcome:

$ lsusb
Bus 001 Device 015: ID 2341:8037 Arduino SA DIY VR Tracker

$ cat /dev/input/js0  # Linux joystick
[Moving the tracker shows axis changes]

Windows Game Controllers:
  "DIY VR Tracker" - 3 axes detected
  X axis (yaw): -180° to +180°
  Y axis (pitch): -90° to +90°
  Z axis (roll): -180° to +180°

[OpenTrack or FreePIE reads the device]
[Games with head tracking support work!]

Implementation Hints:

Arduino Pro Micro HID approach:

#include <HID-Project.h>

// Define a custom HID report for head tracking
static const uint8_t _hidReportDescriptor[] PROGMEM = {
    0x05, 0x01,           // USAGE_PAGE (Generic Desktop)
    0x09, 0x04,           // USAGE (Joystick)
    0xa1, 0x01,           // COLLECTION (Application)
    0x09, 0x01,           //   USAGE (Pointer)
    0xa1, 0x00,           //   COLLECTION (Physical)

    // 3 axes: X, Y, Z (16-bit each)
    0x09, 0x30,           //     USAGE (X)
    0x09, 0x31,           //     USAGE (Y)
    0x09, 0x32,           //     USAGE (Z)
    0x15, 0x00,           //     LOGICAL_MINIMUM (0)
    0x27, 0xff, 0xff, 0x00, 0x00, // LOGICAL_MAXIMUM (65535)
    0x75, 0x10,           //     REPORT_SIZE (16)
    0x95, 0x03,           //     REPORT_COUNT (3)
    0x81, 0x02,           //     INPUT (Data,Var,Abs)

    0xc0,                 //   END_COLLECTION
    0xc0                  // END_COLLECTION
};

typedef struct {
    uint16_t yaw;
    uint16_t pitch;
    uint16_t roll;
} HeadTrackReport;

void loop() {
    // Get orientation from Madgwick filter
    Quat q = madgwick_get_orientation();
    float yaw, pitch, roll;
    quat_to_euler(q, &yaw, &pitch, &roll);

    // Convert to 16-bit range (0-65535)
    // Yaw: -180 to +180 → 0 to 65535
    // Pitch: -90 to +90 → 0 to 65535
    // Roll: -180 to +180 → 0 to 65535
    HeadTrackReport report;
    report.yaw   = (uint16_t)((yaw + 180.0f) / 360.0f * 65535);
    report.pitch = (uint16_t)((pitch + 90.0f) / 180.0f * 65535);
    report.roll  = (uint16_t)((roll + 180.0f) / 360.0f * 65535);

    HID().SendReport(1, &report, sizeof(report));
    delay(2);  // ~500 Hz
}

Questions to explore:

What’s the difference between HID boot protocol and report protocol?
How do you achieve 1000Hz polling rate?
How does OpenTrack translate joystick input to virtual camera movement?
What’s the advantage of USB HID over serial communication?

Learning milestones:

Device appears as joystick → USB enumeration works
Axes respond to movement → Report structure correct
Works in games → Real-world compatibility
Low latency feels responsive → Performance tuned

Project 6: Stereoscopic 3D Renderer

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: 3D Graphics / Stereoscopy
Software or Tool: OpenGL or software renderer
Main Book: Computer Graphics from Scratch by Gabriel Gambetta

What you’ll build: A 3D renderer that produces side-by-side stereoscopic output—two views offset by interpupillary distance (IPD)—that creates the illusion of depth when viewed through VR lenses or a stereoscope.

Why it teaches VR: This is the visual core of VR. Every VR headset renders two slightly offset views. Understanding stereo geometry, camera setup, and view frustums is essential.

Core challenges you’ll face:

Asymmetric view frustums → maps to toe-in vs parallel camera setup
Interpupillary distance (IPD) → maps to correct depth perception
Convergence distance → maps to where 3D objects appear in focus
Rendering twice efficiently → maps to performance optimization

Key Concepts:

Stereoscopic Vision: ARM VR SDK - Intro to Stereo Rendering
Asymmetric Frustums: Real-Time Rendering Chapter 21 - Akenine-Möller et al.
IPD and Convergence: Valve’s VR optics documentation
View Matrix Setup: 3D Math Primer Chapter 7 - Dunn & Parberry

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Projects 1-2 (Math libraries), 3D rendering experience

Real world outcome:

$ ./stereo_renderer
Rendering stereoscopic scene...
IPD: 64mm
Resolution: 1920x1080 per eye (3840x1080 total)

[Window shows side-by-side 3D view]
[Cross your eyes or use a stereoscope to see depth!]

Press +/- to adjust IPD
Press [ ] to adjust convergence
Press WASD to move through scene

[Objects at different distances have correct parallax]
[Near objects shift more between eyes than far objects]

Implementation Hints:

Camera setup for stereo (off-axis projection):

float ipd = 0.064f;  // 64mm in meters
float convergence = 2.0f;  // Focus distance in meters

// Left eye
Vec3 left_eye_pos = vec3_sub(camera_pos, vec3_scale(camera_right, ipd/2));
Mat4 left_view = mat4_look_at(left_eye_pos, look_at_point, camera_up);
Mat4 left_proj = asymmetric_frustum(-near * (half_width + ipd/2) / convergence,
                                      near * (half_width - ipd/2) / convergence,
                                      -near * half_height / convergence,
                                       near * half_height / convergence,
                                      near, far);

// Right eye
Vec3 right_eye_pos = vec3_add(camera_pos, vec3_scale(camera_right, ipd/2));
Mat4 right_view = mat4_look_at(right_eye_pos, look_at_point, camera_up);
Mat4 right_proj = asymmetric_frustum(-near * (half_width - ipd/2) / convergence,
                                       near * (half_width + ipd/2) / convergence,
                                       -near * half_height / convergence,
                                        near * half_height / convergence,
                                       near, far);

Wrong approach (toe-in cameras):

// DON'T DO THIS - causes eye strain!
Mat4 left_view = mat4_look_at(left_eye_pos, center_point, up);
Mat4 right_view = mat4_look_at(right_eye_pos, center_point, up);
// This converges the cameras, causing vertical parallax

Render loop:

void render_stereo_frame() {
    // Left eye (left half of screen)
    glViewport(0, 0, width/2, height);
    render_scene(left_view_matrix, left_proj_matrix);

    // Right eye (right half of screen)
    glViewport(width/2, 0, width/2, height);
    render_scene(right_view_matrix, right_proj_matrix);
}

Questions to explore:

Why do asymmetric frustums prevent eye strain?
How does IPD affect depth perception (too wide = miniaturization)?
What’s the vergence-accommodation conflict in VR?
How do you optimize rendering when both eyes see mostly the same scene?

Learning milestones:

Two views render side-by-side → Basic stereo works
Depth is perceivable → Geometry is correct
No eye strain → Using off-axis projection
IPD adjustment changes depth → Proper stereo math

Project 7: Lens Distortion Correction Shader

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C/GLSL
Alternative Programming Languages: HLSL, Rust with wgpu
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: Shaders / Optics
Software or Tool: OpenGL with shaders
Main Book: Real-Time Rendering by Akenine-Möller, Haines, Hoffman

What you’ll build: A post-processing shader that applies barrel distortion to counteract the pincushion distortion from VR lenses, including chromatic aberration correction.

Why it teaches VR: VR lenses distort the image so your eyes can focus on a screen inches away. The software must pre-distort in the opposite direction. This is essential VR rendering knowledge.

Core challenges you’ll face:

Barrel distortion math → maps to polynomial distortion functions
Chromatic aberration → maps to different distortion per color channel
Performance optimization → maps to avoiding texture bandwidth bottleneck
Lens calibration → maps to measuring real lens parameters

Key Concepts:

Lens Distortion: VR Lens Distortion Approaches by Boris Smus
Barrel Distortion Math: Steven LaValle’s VR Book
Shader Optimization: Imagination Blog
Chromatic Aberration: ustwo VR Distortion

Difficulty: Advanced Time estimate: 1-2 weeks Prerequisites: Project 6 (Stereo Renderer), GLSL basics

Real world outcome:

$ ./distortion_demo
Loading lens calibration...
Distortion coefficients: k1=0.22, k2=0.24
Chromatic aberration: red=1.0, green=1.015, blue=1.03

[Window shows undistorted 3D scene]
[Press D to toggle distortion]
[With distortion: edges are barrel-distorted]
[When viewed through VR lenses: appears correct!]

Press 1/2/3 to adjust distortion strength
Press R/G/B to adjust chromatic aberration

Performance: 4.2ms for distortion pass (244 FPS on RTX 3080)

Implementation Hints:

Fragment shader for barrel distortion:

#version 330 core

uniform sampler2D scene_texture;
uniform vec2 lens_center;       // Center of lens (0.25 for left, 0.75 for right)
uniform vec4 distortion_k;      // Distortion coefficients (k1, k2, k3, k4)
uniform vec4 chroma_ab;         // Chromatic aberration (1.0, scale_g, scale_b, 0)

in vec2 tex_coord;
out vec4 frag_color;

vec2 distort(vec2 coord, float scale) {
    vec2 centered = coord - lens_center;
    float r2 = dot(centered, centered);

    // Polynomial distortion
    float distortion = 1.0
        + distortion_k.x * r2
        + distortion_k.y * r2 * r2
        + distortion_k.z * r2 * r2 * r2
        + distortion_k.w * r2 * r2 * r2 * r2;

    return lens_center + centered * distortion * scale;
}

void main() {
    // Sample each color channel at different distortion levels
    // (chromatic aberration correction)
    vec2 coord_r = distort(tex_coord, chroma_ab.r);
    vec2 coord_g = distort(tex_coord, chroma_ab.g);
    vec2 coord_b = distort(tex_coord, chroma_ab.b);

    float r = texture(scene_texture, coord_r).r;
    float g = texture(scene_texture, coord_g).g;
    float b = texture(scene_texture, coord_b).b;

    // Check if we're sampling outside the rendered area
    if (coord_r.x < 0.0 || coord_r.x > 1.0 ||
        coord_r.y < 0.0 || coord_r.y > 1.0) {
        frag_color = vec4(0.0);  // Black outside
    } else {
        frag_color = vec4(r, g, b, 1.0);
    }
}

Mesh-based approach (faster):

// Pre-compute distorted UV coordinates for a mesh
void create_distortion_mesh(Mesh *mesh, int resolution) {
    for (int y = 0; y <= resolution; y++) {
        for (int x = 0; x <= resolution; x++) {
            float u = (float)x / resolution;
            float v = (float)y / resolution;

            // Position is regular grid
            add_vertex(mesh, u * 2 - 1, v * 2 - 1);

            // UV is distorted
            vec2 distorted = distort(vec2(u, v));
            add_texcoord(mesh, distorted.x, distorted.y);
        }
    }
}
// Now just render this mesh with the scene texture - very fast!

Questions to explore:

Why is the mesh approach faster than per-pixel shader?
How do you measure real lens distortion parameters?
What’s the relationship between lens shape and distortion coefficients?
How does Oculus/Meta calibrate their lenses?

Learning milestones:

Basic distortion visible → Shader works
Looks correct through VR lenses → Parameters tuned
No color fringing → Chromatic aberration corrected
Performance is acceptable → Optimization works

Project 8: Timewarp / Asynchronous Reprojection

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C/C++
Alternative Programming Languages: Rust
Coolness Level: Level 5: Pure Magic
Business Potential: 4. The “Open Core” Infrastructure
Difficulty: Level 4: Expert
Knowledge Area: Real-Time Systems / Graphics
Software or Tool: OpenGL, multi-threading
Main Book: Real-Time Rendering by Akenine-Möller, Haines, Hoffman

What you’ll build: Implement asynchronous timewarp—a technique that reprojects the last rendered frame to account for head rotation that occurred after rendering started, reducing perceived latency.

Why it teaches VR: Timewarp is the secret weapon of VR. It’s why modern VR feels smooth even when frame rate drops. Understanding it means understanding why VR works at all.

Core challenges you’ll face:

Last-moment pose sampling → maps to getting freshest tracking data
Image reprojection → maps to warping rendered image to new orientation
Asynchronous execution → maps to separate render and warp threads
Positional timewarp → maps to handling translation, not just rotation

Key Concepts:

Timewarp Concept: Oculus Developer Blog on ATW
Motion-to-Photon Latency: VR/AR Wiki
Reprojection Math: Oculus Insight
Multi-threaded Rendering: Game Engine Architecture Chapter 8 - Jason Gregory

Difficulty: Expert Time estimate: 3-4 weeks Prerequisites: All previous projects, multi-threading experience

Real world outcome:

$ ./timewarp_demo
Render thread: running at 45 FPS (slow scene)
Timewarp thread: running at 90 FPS

Motion-to-photon latency:
  Without timewarp: 32ms
  With timewarp: 11ms

[Rapid head movement test]
Without timewarp: visible judder and swimming
With timewarp: smooth rotation, world stays stable

[Drop render to 30 FPS intentionally]
Without timewarp: nauseating stuttering
With timewarp: rotation still smooth at 90Hz

Press T to toggle timewarp
Press L to show latency overlay

Implementation Hints:

Basic timewarp algorithm:

// Render thread stores the pose used for rendering
typedef struct {
    Quat orientation;
    Vec3 position;
    uint64_t timestamp;
    GLuint texture;
} RenderedFrame;

// Timewarp thread
void timewarp_thread(void *data) {
    while (running) {
        wait_for_vsync();  // Sync with display

        // Get the latest head pose
        Quat current_orientation = get_current_head_orientation();

        // Get the orientation that was used to render
        RenderedFrame *frame = get_latest_rendered_frame();
        Quat render_orientation = frame->orientation;

        // Calculate the difference
        Quat delta = quat_multiply(
            current_orientation,
            quat_conjugate(render_orientation)
        );

        // Convert to rotation matrix
        Mat4 warp_matrix = quat_to_matrix(delta);

        // Apply the warp and display
        render_warped_frame(frame->texture, warp_matrix);
        swap_buffers();
    }
}

Warp shader:

#version 330 core

uniform sampler2D rendered_frame;
uniform mat4 warp_matrix;  // Rotation difference

in vec2 tex_coord;
out vec4 frag_color;

void main() {
    // Convert texture coordinate to 3D direction
    vec3 dir = normalize(vec3(tex_coord * 2.0 - 1.0, 1.0));

    // Apply the rotation warp
    vec3 warped_dir = (warp_matrix * vec4(dir, 0.0)).xyz;

    // Project back to 2D
    vec2 warped_uv = warped_dir.xy / warped_dir.z * 0.5 + 0.5;

    // Sample the rendered frame
    frag_color = texture(rendered_frame, warped_uv);
}

Thread synchronization:

// Producer (render thread)
void render_loop() {
    while (running) {
        Quat pose = get_predicted_head_pose(time + RENDER_AHEAD);
        render_scene_to_texture(pose, &frame.texture);
        frame.orientation = pose;
        frame.timestamp = get_time();
        submit_frame(&frame);  // Thread-safe handoff
    }
}

// Consumer (timewarp thread)
void timewarp_loop() {
    while (running) {
        wait_for_vsync_start();
        RenderedFrame *frame = get_latest_frame();
        Quat current = get_head_pose_now();
        apply_timewarp(frame, current);
        wait_for_vsync_end();
        present();
    }
}

Questions to explore:

Why can timewarp only correct for rotation, not translation?
What is Asynchronous Spacewarp (ASW) and how does it handle translation?
How does prediction help further reduce latency?
What are the artifacts when timewarp is pushed too far?

Learning milestones:

Separate threads for render and display → Architecture works
Warp visibly corrects for rotation → Basic timewarp works
Smoother than no-warp at same framerate → Latency reduced
Survives 30 FPS rendering → Robust implementation

Project 9: Spatial Audio Engine with HRTF

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 3: Advanced
Knowledge Area: Audio Processing / DSP
Software or Tool: SDL_audio or PortAudio, HRTF datasets
Main Book: DAFX: Digital Audio Effects edited by Udo Zölzer

What you’ll build: A 3D spatial audio engine that uses Head-Related Transfer Functions (HRTFs) to position sounds in 3D space around the listener, with proper distance attenuation and environmental effects.

Why it teaches VR: Sound is half the VR experience. Spatial audio sells the illusion of presence. Understanding HRTFs and 3D audio positioning is essential for immersive VR.

Core challenges you’ll face:

HRTF convolution → maps to applying direction-dependent filters
Interpolating between HRTFs → maps to smooth sound movement
Head tracking integration → maps to sounds stay in world space
Real-time performance → maps to low-latency audio processing

Key Concepts:

HRTF Theory: Wikipedia - HRTF
Spatial Audio Implementation: Meta Spatial Audio
ITD and ILD: Medium - Spatial Audio for VR
Audio DSP: DAFX Chapter 2 - Zölzer

Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Basic audio concepts, FFT understanding

Real world outcome:

$ ./spatial_audio_demo
Loading HRTF database (MIT KEMAR)...
Loaded 710 HRTFs covering full sphere

Audio system: 48kHz, 256 sample buffer (5.3ms latency)

Sound sources:
  [0] Bird chirp: position (2, 1, 0) - to your right and up
  [1] Stream: position (0, 0, 3) - in front of you
  [2] Footsteps: moving around you in a circle

[Put on headphones]
[Sounds are clearly localized in 3D space!]

Press H to toggle head tracking
[Without: sounds move with your head]
[With: sounds stay fixed in world space!]

Press R for reverb (room simulation)

Implementation Hints:

HRTF data structure:

typedef struct {
    float azimuth;      // Horizontal angle (-180 to 180)
    float elevation;    // Vertical angle (-90 to 90)
    float *left_ir;     // Left ear impulse response
    float *right_ir;    // Right ear impulse response
    int ir_length;      // Typically 128-512 samples
} HRTF;

typedef struct {
    HRTF *hrtfs;
    int count;
} HRTFDatabase;

Applying HRTF (time-domain convolution):

void apply_hrtf(float *input, int input_len,
                HRTF *hrtf,
                float *output_left, float *output_right)
{
    // Convolve input with left ear impulse response
    for (int i = 0; i < input_len + hrtf->ir_length - 1; i++) {
        output_left[i] = 0;
        output_right[i] = 0;
        for (int j = 0; j < hrtf->ir_length; j++) {
            if (i - j >= 0 && i - j < input_len) {
                output_left[i] += input[i - j] * hrtf->left_ir[j];
                output_right[i] += input[i - j] * hrtf->right_ir[j];
            }
        }
    }
}

// Faster: use FFT-based convolution for real-time
void apply_hrtf_fft(float *input, int input_len,
                    HRTF *hrtf,
                    float *output_left, float *output_right);

Sound positioning:

typedef struct {
    Vec3 position;        // World position
    float *audio_data;    // Mono audio source
    int sample_offset;    // Current playback position
    float volume;
} SoundSource;

void update_sound_source(SoundSource *src, Vec3 listener_pos, Quat listener_rot) {
    // Calculate direction in listener's local space
    Vec3 world_dir = vec3_normalize(vec3_sub(src->position, listener_pos));
    Vec3 local_dir = quat_rotate(quat_conjugate(listener_rot), world_dir);

    // Convert to spherical coordinates
    float azimuth = atan2f(local_dir.x, local_dir.z) * 180.0f / M_PI;
    float elevation = asinf(local_dir.y) * 180.0f / M_PI;

    // Find nearest HRTF
    HRTF *hrtf = find_nearest_hrtf(azimuth, elevation);

    // Calculate distance attenuation
    float distance = vec3_length(vec3_sub(src->position, listener_pos));
    float attenuation = 1.0f / (1.0f + distance * 0.5f);

    // Apply HRTF and attenuation
    // ...
}

Questions to explore:

Why do generic HRTFs work for most people?
How does HRTF interpolation prevent “jumping” sounds?
What’s the difference between binaural audio and surround sound?
How do you simulate room acoustics (reverb)?

Learning milestones:

Stereo panning works → Basic spatialization
Sounds are clearly localized → HRTF working
Sounds stay fixed as head turns → Tracking integration
No audio glitches → Real-time performance

Project 10: VR Controller Input System

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 2: Intermediate
Knowledge Area: Input Systems / HID
Software or Tool: OpenXR or SteamVR API
Main Book: Game Engine Architecture by Jason Gregory

What you’ll build: An input abstraction layer that reads VR controller data (position, orientation, buttons, triggers, joysticks, haptics) and maps them to application actions.

Why it teaches VR: VR input is fundamentally different from desktop input. 6DOF tracked controllers with pose data require new thinking about input handling.

Core challenges you’ll face:

Pose data handling → maps to position + orientation per controller
Action mapping → maps to abstracting hardware differences
Haptic feedback → maps to vibration patterns for immersion
Hand presence detection → maps to knowing when user is holding controller

Key Concepts:

OpenXR Actions: OpenXR Tutorial
Input Abstraction: Game Engine Architecture Chapter 9 - Jason Gregory
Haptic Design: Oculus Haptics Design Guidelines
Controller Tracking: 6DOF Tracking Methods

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Basic input handling, API familiarity

Real world outcome:

$ ./controller_demo
OpenXR runtime: SteamVR
Left controller: connected, tracked
Right controller: connected, tracked

Controller poses:
  Left:  pos=(−0.3, 1.0, 0.5) rot=(0.98, 0.0, 0.2, 0.0)
  Right: pos=(0.3, 1.0, 0.5) rot=(0.98, 0.0, −0.2, 0.0)

Inputs:
  Left trigger: 0.0 (released)
  Right trigger: 0.85 (mostly pressed)
  Left grip: held
  A button: pressed!

[Trigger haptic pulse on A press]
*buzz*

Action bindings:
  "grab" → grip button → left or right
  "teleport" → joystick click → left only
  "shoot" → trigger → right only

Implementation Hints:

Controller state structure:

typedef struct {
    bool connected;
    bool tracked;

    // 6DOF pose
    Vec3 position;
    Quat orientation;
    Vec3 velocity;
    Vec3 angular_velocity;

    // Buttons (digital)
    bool button_a;
    bool button_b;
    bool button_menu;
    bool button_joystick;
    bool grip_button;

    // Axes (analog)
    float trigger;          // 0.0 - 1.0
    float grip;             // 0.0 - 1.0
    Vec2 joystick;          // (-1,-1) to (1,1)

    // Touch capacitance
    bool trigger_touched;
    bool joystick_touched;
    bool a_touched;
    bool b_touched;
} ControllerState;

typedef struct {
    ControllerState left;
    ControllerState right;
    ControllerState head;   // HMD pose
} VRInputState;

Action mapping system:

typedef enum {
    ACTION_GRAB,
    ACTION_TELEPORT,
    ACTION_PRIMARY_FIRE,
    ACTION_MENU,
    ACTION_COUNT
} ActionType;

typedef struct {
    ActionType action;
    bool pressed;
    bool just_pressed;
    bool just_released;
    float value;        // For analog actions
    Vec3 position;      // For pose-based actions
    Quat orientation;
} ActionState;

// Map physical inputs to actions
void update_actions(VRInputState *input, ActionState *actions) {
    // Grab = grip button on either hand
    actions[ACTION_GRAB].pressed =
        input->left.grip_button || input->right.grip_button;

    // Teleport = left joystick up
    actions[ACTION_TELEPORT].value =
        fmaxf(0, input->left.joystick.y);

    // etc.
}

Haptic feedback:

void trigger_haptic(Controller hand, float amplitude, float duration_sec) {
    // XR_TYPE_HAPTIC_VIBRATION in OpenXR
    HapticVibration vib = {
        .amplitude = amplitude,  // 0.0 to 1.0
        .frequency = 0,          // 0 = default
        .duration = (int64_t)(duration_sec * 1e9)  // nanoseconds
    };
    apply_haptic(hand, &vib);
}

Questions to explore:

How do you handle different controller layouts (Oculus vs Index vs Vive)?
What makes a good haptic feedback pattern?
How do you implement “laser pointer” style interaction?
What’s the difference between hand tracking and controller tracking?

Learning milestones:

Controllers are detected and tracked → Basic input works
Buttons and triggers read correctly → Input mapping works
Actions abstract away hardware → Clean API
Haptics feel responsive → Feedback works

Project 11: Basic OpenXR Application

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 3: Advanced
Knowledge Area: VR APIs / Graphics Integration
Software or Tool: OpenXR SDK, OpenGL or Vulkan
Main Book: OpenXR Specification

What you’ll build: A complete OpenXR application that initializes the runtime, creates a session, handles the frame loop, and renders a simple 3D scene in VR.

Why it teaches VR: OpenXR is the cross-platform standard for VR development. Understanding it means you can target Quest, Index, Vive, Windows MR, and more from a single codebase.

Core challenges you’ll face:

OpenXR initialization → maps to instance, system, session creation
Swapchain management → maps to eye texture handling
Frame timing → maps to wait, begin, end frame protocol
View configuration → maps to stereo vs mono, view poses

Key Concepts:

OpenXR Tutorial: openxr-tutorial.com
OpenXR Specification: Khronos OpenXR 1.0 spec
Meta OpenXR Guide: Meta PC Development
VIVE OpenXR: VIVE Forum Tutorial

Resources for key challenges:

Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Projects 6-7 (Stereo rendering, distortion), Graphics API experience

Real world outcome:

$ ./openxr_demo
OpenXR instance created
  Runtime: SteamVR
  Version: 1.0.26

System found: Valve Index
  Max swapchain: 2468x2740 per eye
  Refresh rates: 80Hz, 90Hz, 120Hz, 144Hz

Session created (state: READY)
Reference space: LOCAL

[Put on headset]
Session state: VISIBLE → FOCUSED

Rendering at 90Hz...
  Left eye:  pos=(0.032, 0.0, 0.0) fov=(50°, 55°, 48°, 53°)
  Right eye: pos=(-0.032, 0.0, 0.0) fov=(55°, 50°, 48°, 53°)

[You're standing in a VR room with floating cubes!]
[Look around - head tracking works!]
[Controllers visible and tracked!]

Implementation Hints:

OpenXR initialization flow:

// 1. Create instance
XrInstanceCreateInfo instanceInfo = {XR_TYPE_INSTANCE_CREATE_INFO};
strcpy(instanceInfo.applicationInfo.applicationName, "My VR App");
instanceInfo.enabledExtensionCount = extension_count;
instanceInfo.enabledExtensionNames = extensions;
xrCreateInstance(&instanceInfo, &instance);

// 2. Get system (the HMD)
XrSystemGetInfo systemInfo = {XR_TYPE_SYSTEM_GET_INFO};
systemInfo.formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY;
xrGetSystem(instance, &systemInfo, &system_id);

// 3. Create session (connects to runtime)
XrGraphicsBindingOpenGLWin32KHR graphicsBinding = {...};
XrSessionCreateInfo sessionInfo = {XR_TYPE_SESSION_CREATE_INFO};
sessionInfo.systemId = system_id;
sessionInfo.next = &graphicsBinding;
xrCreateSession(instance, &sessionInfo, &session);

// 4. Create swapchains (eye textures)
xrEnumerateViewConfigurationViews(...);  // Get recommended sizes
XrSwapchainCreateInfo swapchainInfo = {...};
xrCreateSwapchain(session, &swapchainInfo, &swapchain);

// 5. Create reference space
XrReferenceSpaceCreateInfo spaceInfo = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
spaceInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_LOCAL;
xrCreateReferenceSpace(session, &spaceInfo, &local_space);

Frame loop:

while (running) {
    // Poll events
    XrEventDataBuffer event = {XR_TYPE_EVENT_DATA_BUFFER};
    while (xrPollEvent(instance, &event) == XR_SUCCESS) {
        handle_event(&event);
    }

    if (session_running) {
        // Wait for frame
        XrFrameWaitInfo waitInfo = {XR_TYPE_FRAME_WAIT_INFO};
        XrFrameState frameState = {XR_TYPE_FRAME_STATE};
        xrWaitFrame(session, &waitInfo, &frameState);

        // Begin frame
        XrFrameBeginInfo beginInfo = {XR_TYPE_FRAME_BEGIN_INFO};
        xrBeginFrame(session, &beginInfo);

        // Get view poses
        XrViewLocateInfo locateInfo = {...};
        locateInfo.displayTime = frameState.predictedDisplayTime;
        xrLocateViews(session, &locateInfo, &viewState, view_count, views);

        // Render each eye
        for (int eye = 0; eye < 2; eye++) {
            acquire_swapchain_image(swapchains[eye]);
            render_eye(views[eye].pose, views[eye].fov);
            release_swapchain_image(swapchains[eye]);
        }

        // End frame
        XrFrameEndInfo endInfo = {XR_TYPE_FRAME_END_INFO};
        endInfo.displayTime = frameState.predictedDisplayTime;
        endInfo.layerCount = 1;
        endInfo.layers = &projection_layer;
        xrEndFrame(session, &endInfo);
    }
}

Questions to explore:

What’s the difference between LOCAL and STAGE reference spaces?
How does OpenXR handle compositor vs application rendering?
What are OpenXR extensions and which are essential?
How do you handle session state changes (focus lost, etc.)?

Learning milestones:

Instance and session created → Basic setup works
Stereo view renders → Graphics integration works
Head tracking works → Pose queries work
Smooth experience on hardware → Frame timing correct

Project 12: Teleportation Locomotion System

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 2: Intermediate
Knowledge Area: VR Interaction / Locomotion
Software or Tool: Your OpenXR app (Project 11)
Main Book: 3D User Interfaces by LaViola, Kruijff, McMahan, Bowman, Poupyrev

What you’ll build: A teleportation system where users point with a controller, see an arc trajectory, and teleport to valid locations—the standard comfortable VR locomotion.

Why it teaches VR: Locomotion is the biggest challenge in VR. Teleportation became standard because it avoids motion sickness. Understanding why it works (and doesn’t) is key to VR UX.

Core challenges you’ll face:

Arc trajectory calculation → maps to projectile motion physics
Valid destination detection → maps to raycasting against navigation mesh
Visual feedback → maps to showing where you’ll land
Fade transitions → maps to reducing disorientation

Key Concepts:

VR Locomotion: 3D User Interfaces Chapter 5 - LaViola et al.
Motion Sickness Prevention: Oculus VR Best Practices
Projectile Motion: Basic physics
Navigation Meshes: Game AI Pro - Steve Rabin

Difficulty: Intermediate Time estimate: 1 week Prerequisites: Project 11 (OpenXR App), basic physics

Real world outcome:

[In VR]
1. Hold grip to activate teleport
2. Aim controller - parabolic arc appears
3. Arc lands on valid floor → green target marker
4. Arc lands on wall/obstacle → red X marker
5. Release grip while green → brief fade → you're there!

Features:
- Arc adjusts based on controller angle
- Turn indicator shows which way you'll face
- Smooth fade transition (150ms)
- Haptic pulse on valid target

Implementation Hints:

Parabolic arc calculation:

#define ARC_SEGMENTS 30
#define GRAVITY 9.81f
#define INITIAL_VELOCITY 10.0f

void calculate_teleport_arc(Vec3 start, Vec3 direction,
                            Vec3 *arc_points, int *hit_index,
                            Vec3 *hit_point, Vec3 *hit_normal)
{
    Vec3 velocity = vec3_scale(vec3_normalize(direction), INITIAL_VELOCITY);
    Vec3 pos = start;

    float dt = 0.1f;  // Time step per segment

    for (int i = 0; i < ARC_SEGMENTS; i++) {
        arc_points[i] = pos;

        // Check for collision with ground/obstacles
        Vec3 next_pos = pos;
        next_pos.x += velocity.x * dt;
        next_pos.y += velocity.y * dt - 0.5f * GRAVITY * dt * dt;
        next_pos.z += velocity.z * dt;

        if (raycast(pos, next_pos, hit_point, hit_normal)) {
            *hit_index = i;
            return;
        }

        // Update for next iteration
        velocity.y -= GRAVITY * dt;
        pos = next_pos;
    }

    *hit_index = -1;  // No hit
}

Destination validation:

bool is_valid_teleport_destination(Vec3 hit_point, Vec3 hit_normal) {
    // Check if surface is flat enough (walkable)
    if (hit_normal.y < 0.7f) return false;  // Too steep

    // Check if on nav mesh
    if (!point_on_navmesh(hit_point)) return false;

    // Check if there's headroom
    if (raycast_up(hit_point, 2.0f)) return false;  // Ceiling too low

    return true;
}

Teleport execution:

void execute_teleport(Vec3 destination, float target_yaw) {
    // Start fade out
    start_screen_fade(FADE_OUT, 0.15f);

    // Wait for fade
    wait_seconds(0.15f);

    // Move player
    set_player_position(destination);
    set_player_yaw(target_yaw);

    // Start fade in
    start_screen_fade(FADE_IN, 0.15f);
}

Questions to explore:

Why does teleportation cause less motion sickness than smooth locomotion?
How do you handle teleporting onto different floor heights?
What’s the difference between instant and blink teleportation?
How do you prevent players from teleporting through walls?

Learning milestones:

Arc renders from controller → Trajectory math works
Valid/invalid destinations shown → Raycasting works
Teleportation moves player → Basic system works
No disorientation on teleport → Fade transition tuned

Project 13: Grab and Throw Interaction

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: C++, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: VR Interaction / Physics
Software or Tool: Your OpenXR app, physics engine
Main Book: 3D User Interfaces by LaViola, Kruijff, McMahan, Bowman, Poupyrev

What you’ll build: A grab-and-throw system where you can pick up objects with your hands, manipulate them naturally, and throw them with velocity matching your hand movement.

Why it teaches VR: Direct manipulation is what makes VR magical. Understanding how to track hand velocity, handle physics handoff, and make throwing feel natural is core VR interaction design.

Core challenges you’ll face:

Grab detection → maps to hand proximity and grip input
Object following hand → maps to kinematic vs physics control
Velocity calculation → maps to smoothed derivative of position
Release timing → maps to when to apply throw velocity

Key Concepts:

VR Manipulation: 3D User Interfaces Chapter 6 - LaViola et al.
Physics Handoff: Game physics tutorials
Velocity Estimation: Kalman filtering or moving average
Haptic Feedback: Oculus Haptics Guidelines

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Project 10 (Controller Input), physics engine familiarity

Real world outcome:

[In VR with physics-enabled objects]
1. Move hand near a cube - it highlights
2. Press grip - cube snaps to hand
3. Move hand around - cube follows perfectly
4. Wave arm and release grip - cube FLIES!
5. Cube bounces off wall with correct physics

Throw accuracy test:
  Throw at target 5m away
  Hit rate: 8/10 (with practice)
  "Throwing feels natural!"

Edge cases handled:
  - Grabbing object from another hand
  - Objects too heavy to throw far
  - Releasing at different grip pressures

Implementation Hints:

Velocity tracking (for throw):

#define VELOCITY_HISTORY_SIZE 10

typedef struct {
    Vec3 positions[VELOCITY_HISTORY_SIZE];
    Quat rotations[VELOCITY_HISTORY_SIZE];
    float timestamps[VELOCITY_HISTORY_SIZE];
    int index;
} VelocityTracker;

Vec3 calculate_velocity(VelocityTracker *tracker) {
    // Use positions from ~60-100ms ago for best throw feel
    int current = tracker->index;
    int old = (current - 5 + VELOCITY_HISTORY_SIZE) % VELOCITY_HISTORY_SIZE;

    Vec3 delta = vec3_sub(tracker->positions[current], tracker->positions[old]);
    float dt = tracker->timestamps[current] - tracker->timestamps[old];

    return vec3_scale(delta, 1.0f / dt);
}

Vec3 calculate_angular_velocity(VelocityTracker *tracker) {
    // Similar for rotation
    // ...
}

Grab system:

typedef struct {
    RigidBody *held_object;
    Vec3 grab_offset;       // Offset from hand to object center
    Quat grab_rotation;     // Rotation offset
    VelocityTracker vel_tracker;
} HandGrabState;

void update_grab(HandGrabState *state, ControllerState *controller) {
    // Track velocity history
    record_velocity(&state->vel_tracker, controller->position, controller->orientation);

    if (!state->held_object) {
        // Check for nearby grabbable objects
        if (controller->grip > 0.8f) {
            RigidBody *nearest = find_nearest_grabbable(controller->position, 0.1f);
            if (nearest) {
                state->held_object = nearest;
                state->grab_offset = vec3_sub(nearest->position, controller->position);
                state->grab_rotation = quat_multiply(
                    quat_conjugate(controller->orientation),
                    nearest->orientation
                );
                set_kinematic(nearest, true);  // Disable physics while held
                trigger_haptic(controller, 0.5f, 0.05f);
            }
        }
    } else {
        // Update held object position
        state->held_object->position = vec3_add(
            controller->position,
            quat_rotate(controller->orientation, state->grab_offset)
        );
        state->held_object->orientation = quat_multiply(
            controller->orientation,
            state->grab_rotation
        );

        // Check for release
        if (controller->grip < 0.3f) {
            // Apply velocity for throw
            Vec3 velocity = calculate_velocity(&state->vel_tracker);
            Vec3 angular_vel = calculate_angular_velocity(&state->vel_tracker);

            set_kinematic(state->held_object, false);
            state->held_object->velocity = velocity;
            state->held_object->angular_velocity = angular_vel;

            state->held_object = NULL;
        }
    }
}

Questions to explore:

Why use velocity from 60-100ms ago instead of current?
How do you handle grabbing very heavy objects?
What’s the difference between direct attachment and physics joints?
How do you prevent objects from going through walls when thrown?

Learning milestones:

Objects can be grabbed → Detection works
Objects follow hand perfectly → Kinematic tracking works
Throws feel natural → Velocity estimation tuned
Physics continues correctly after release → Handoff works

Project 14: VR Comfort and Motion Sickness Mitigation

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C
Alternative Programming Languages: Any
Coolness Level: Level 3: Genuinely Clever
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 2: Intermediate
Knowledge Area: Human Factors / VR UX
Software or Tool: Your VR application
Main Book: Virtual Reality by Steven M. LaValle (free online)

What you’ll build: A suite of comfort options—vignette during movement, snap turning, comfort cage, and reduced FOV during locomotion—that reduce motion sickness.

Why it teaches VR: Motion sickness is VR’s biggest barrier. Understanding why it happens and how to mitigate it separates good VR developers from great ones.

Core challenges you’ll face:

Vignette implementation → maps to reducing peripheral vision during motion
Snap vs smooth turning → maps to discrete vs continuous rotation
Static reference frames → maps to cockpits, comfort cages
User adaptation → maps to settings and preferences

Key Concepts:

Vestibular System: Virtual Reality Chapter 8 - LaValle
Sensory Conflict Theory: Frontiers in VR
Comfort Best Practices: Oculus VR Best Practices Guide
Motion-to-Photon Latency: VR Wiki

Difficulty: Intermediate Time estimate: 1 week Prerequisites: Any VR project with locomotion

Real world outcome:

Settings Menu:
  ☑ Vignette during movement
    Intensity: [====----] 50%

  ○ Smooth turning
  ● Snap turning (45°)

  ☐ Comfort cage (show static reference)

  Movement speed: [======--] 75%

[With vignette enabled]
- Move forward: screen edges darken
- Stop moving: edges fade back to normal
- Result: significantly less nausea!

[Snap turning]
- Push joystick right
- Instant 45° turn (no smooth rotation)
- Much more comfortable for sensitive users

Implementation Hints:

Vignette shader:

uniform float vignette_intensity;  // 0.0 (off) to 1.0 (maximum)
uniform float movement_speed;       // Current movement speed

void main() {
    vec4 scene_color = texture(scene, tex_coord);

    // Calculate distance from center (0-1)
    vec2 centered = tex_coord - 0.5;
    float dist = length(centered) * 2.0;  // 0 at center, 1+ at edges

    // Vignette based on movement
    float vignette = 1.0 - smoothstep(0.3, 1.0, dist) * movement_speed * vignette_intensity;

    frag_color = vec4(scene_color.rgb * vignette, 1.0);
}

Snap turning:

#define SNAP_ANGLE (M_PI / 4.0f)  // 45 degrees
#define SNAP_DEADZONE 0.7f
#define SNAP_COOLDOWN 0.3f

void handle_snap_turn(VRPlayer *player, float joystick_x, float dt) {
    static float cooldown = 0;
    cooldown -= dt;

    if (cooldown <= 0 && fabsf(joystick_x) > SNAP_DEADZONE) {
        float direction = (joystick_x > 0) ? 1.0f : -1.0f;
        player->yaw += direction * SNAP_ANGLE;
        cooldown = SNAP_COOLDOWN;

        // Optional: brief fade during turn
        flash_screen(0.05f, 0.1f);  // 50ms black flash
    }
}

Comfort cage (static reference frame):

void render_comfort_cage(Mat4 view_matrix, bool player_moving) {
    if (!player_moving) return;

    // Render a faint grid or cockpit that doesn't move with the world
    // This gives the brain a stable reference

    Mat4 cage_model = mat4_identity();  // Fixed to player, not world
    cage_model = mat4_multiply(cage_model, mat4_translate(0, 0, 2));

    // Render semi-transparent cage geometry
    // ...
}

Questions to explore:

Why does peripheral vision cause more motion sickness?
What’s the vestibulo-ocular reflex and why does it matter?
How do “VR legs” develop with use?
Why are some people more susceptible than others?

Learning milestones:

Vignette responds to movement → Basic implementation
Snap turning works cleanly → No residual rotation
Settings are user-adjustable → Preference system
You can test on motion-sensitive users → Real validation

Project 15: Complete VR Experience

File: LEARN_VIRTUAL_REALITY_FROM_SCRATCH.md
Main Programming Language: C/C++
Alternative Programming Languages: Rust
Coolness Level: Level 5: Pure Magic
Business Potential: 4. The “Open Core” Infrastructure
Difficulty: Level 4: Expert
Knowledge Area: Complete VR Development
Software or Tool: All previous projects combined
Main Book: The VR Book by Jason Jerald

What you’ll build: A complete, polished VR experience—a room-scale environment with interactive objects, teleportation, grab mechanics, spatial audio, and all comfort options.

Why it teaches VR: Integration is where mastery is proven. Making all systems work together—tracking, rendering, audio, interaction—reveals the complexity of production VR.

Core challenges you’ll face:

System integration → maps to making everything work together
Performance optimization → maps to maintaining 90 FPS
Polish and juice → maps to haptics, audio, visual feedback
Cross-platform testing → maps to Quest vs PCVR differences

Difficulty: Expert Time estimate: 1-2 months Prerequisites: All previous projects

Real world outcome:

"My First VR Room"

[Enter VR]
- Comfortable tutorial explaining controls
- Room-scale environment with interesting objects
- Teleport to move around
- Pick up and throw objects (physics!)
- Spatial audio - sounds come from objects
- Comfort options accessible from wrist menu

Features:
- 90 FPS stable on Quest 2
- Works on SteamVR headsets too (OpenXR)
- All comfort options functional
- Polished with haptics and audio feedback

"This feels like a real VR app!"

Project Comparison Table

#	Project	Difficulty	Time	Depth of Understanding	Fun Factor
1	Vector/Matrix Library	Beginner	1 week	★★★☆☆	★★☆☆☆
2	Quaternion Library	Intermediate	1 week	★★★★☆	★★☆☆☆
3	IMU Sensor Reader	Intermediate	1 week	★★★★☆	★★★★☆
4	Madgwick Sensor Fusion	Advanced	2 weeks	★★★★★	★★★★☆
5	USB HID Head Tracker	Advanced	1-2 weeks	★★★★☆	★★★★★
6	Stereoscopic Renderer	Advanced	2 weeks	★★★★★	★★★★☆
7	Lens Distortion Shader	Advanced	1-2 weeks	★★★★☆	★★★☆☆
8	Timewarp/Reprojection	Expert	3-4 weeks	★★★★★	★★★★☆
9	Spatial Audio (HRTF)	Advanced	2-3 weeks	★★★★★	★★★★★
10	Controller Input System	Intermediate	1-2 weeks	★★★☆☆	★★★★☆
11	Basic OpenXR Application	Advanced	2-3 weeks	★★★★★	★★★★★
12	Teleportation Locomotion	Intermediate	1 week	★★★☆☆	★★★★☆
13	Grab and Throw	Advanced	2 weeks	★★★★☆	★★★★★
14	Comfort/Motion Sickness	Intermediate	1 week	★★★★☆	★★★☆☆
15	Complete VR Experience	Expert	1-2 months	★★★★★	★★★★★

Recommended Learning Path

Phase 1: Mathematical Foundations (2 weeks)

Start here: Projects 1 → 2

Build the math libraries you’ll use everywhere. Quaternions are essential for VR.

Phase 2: Hardware Understanding (3-4 weeks)

Projects 3 → 4 → 5

Build actual tracking hardware. Understanding IMUs and sensor fusion is core VR knowledge.

Phase 3: Visual Systems (4-6 weeks)

Projects 6 → 7 → 8

Master stereoscopic rendering, lens distortion, and timewarp. This is the visual heart of VR.

Phase 4: Audio and Input (3-4 weeks)

Projects 9 → 10

Add spatial audio and controller input. These complete the immersion.

Phase 5: Integration and Interaction (4-6 weeks)

Projects 11 → 12 → 13 → 14 → 15

Build complete VR applications with OpenXR and polished interactions.

Essential Resources Summary

Books

Book	Author	Best For
3D Math Primer for Graphics and Game Development	Dunn & Parberry	Math foundations
Virtual Reality	Steven M. LaValle	Theory (free online!)
The VR Book	Jason Jerald	Design and human factors
3D User Interfaces	LaViola et al.	Interaction design
Real-Time Rendering	Akenine-Möller et al.	Graphics techniques
Game Engine Architecture	Jason Gregory	Systems design

Online Resources

OpenXR Tutorial - Official cross-platform VR tutorial
Oculus Developer Documentation - Best practices, design guidelines
LaValle’s VR Book - Free, comprehensive theory
Madgwick Filter Paper - Original sensor fusion algorithm
Relativty Open Source VR - DIY headset project

Hardware for Learning

IMU Development: MPU-6050 ($5), BNO055 ($25)
Microcontroller: Arduino Pro Micro ($10), ESP32-S3 ($15)
VR Headset: Quest 2/3 (standalone + PCVR), Valve Index (high-end PCVR)

Summary

#	Project	Main Language
1	3D Vector and Matrix Library	C
2	Quaternion Library and Rotations	C
3	IMU Sensor Reader (Arduino/ESP32)	C/Arduino
4	Sensor Fusion with Madgwick Filter	C
5	USB HID Head Tracker	C/Arduino
6	Stereoscopic 3D Renderer	C
7	Lens Distortion Correction Shader	C/GLSL
8	Timewarp / Asynchronous Reprojection	C/C++
9	Spatial Audio Engine with HRTF	C
10	VR Controller Input System	C
11	Basic OpenXR Application	C
12	Teleportation Locomotion System	C
13	Grab and Throw Interaction	C
14	VR Comfort and Motion Sickness Mitigation	C
15	Complete VR Experience (Capstone)	C/C++

Learn Virtual Reality From Scratch

Why Virtual Reality Matters

Core Concept Analysis

The VR System Pipeline

Fundamental Concepts

Project List

Project 1: 3D Vector and Matrix Library

Project 2: Quaternion Library and Rotations

Project 3: IMU Sensor Reader (Arduino/ESP32)

Project 4: Sensor Fusion with Madgwick Filter

Project 5: USB HID Head Tracker

Project 6: Stereoscopic 3D Renderer

Project 7: Lens Distortion Correction Shader

Project 8: Timewarp / Asynchronous Reprojection

Project 9: Spatial Audio Engine with HRTF

Project 10: VR Controller Input System

Project 11: Basic OpenXR Application

Project 12: Teleportation Locomotion System

Project 13: Grab and Throw Interaction

Project 14: VR Comfort and Motion Sickness Mitigation

Project 15: Complete VR Experience

Project Comparison Table

Recommended Learning Path

Phase 1: Mathematical Foundations (2 weeks)

Phase 2: Hardware Understanding (3-4 weeks)

Phase 3: Visual Systems (4-6 weeks)

Phase 4: Audio and Input (3-4 weeks)

Phase 5: Integration and Interaction (4-6 weeks)

Essential Resources Summary

Books

Online Resources

Hardware for Learning

Summary

Sources