Project 7: Camera Timelapse + Storage Strategy

Build a headless timelapse pipeline that captures images on schedule and enforces storage retention safely.

Quick Reference

Attribute Value
Difficulty Intermediate
Time Estimate 1–2 weekends
Main Programming Language Python (Alternatives: Bash, Go, Rust)
Alternative Programming Languages Bash, Go, Rust
Coolness Level High
Business Potential Medium
Prerequisites Linux CLI, file I/O, camera module setup
Key Topics libcamera pipeline, scheduling without drift, storage retention

1. Learning Objectives

By completing this project, you will:

  1. Capture images headlessly using rpicam/libcamera tools.
  2. Implement a drift-free scheduling loop for long runs.
  3. Design a retention policy that prevents SD card exhaustion.
  4. Detect and recover from camera failures automatically.

2. All Theory Needed (Per-Concept Breakdown)

Concept 1: Camera Pipelines, Scheduling, and Storage Retention

Fundamentals

A camera pipeline captures high-bandwidth data and stores it to disk on a schedule. On Raspberry Pi, libcamera and rpicam tools manage camera hardware. A timelapse is a repeated capture at fixed intervals. If your schedule drifts, images become irregular. If you store images without limits, the SD card will fill and the system can fail. A retention policy is the set of rules that decides how many images to keep and when to delete or compress older data. The core concept is that camera capture is not just about taking pictures; it is about sustaining reliable capture over days or weeks with limited storage.

Deep Dive into the concept

The camera stack on Raspberry Pi Zero 2 W uses libcamera, which provides a modern pipeline-based camera framework. The rpicam-still tool (or libcamera-still depending on OS version) can capture single images with control over resolution, quality, and exposure. The camera is sensitive to power and cable orientation; an unstable power supply or loose ribbon cable can cause intermittent failures. When building a timelapse, you should start with known-good settings (e.g., 1280x720 JPEG at moderate quality) and then optimize for storage and quality.

Scheduling is deceptively complex. A naive loop that sleeps for interval seconds after each capture accumulates drift because capture time is not zero. If each capture takes 0.6 seconds and you sleep for 60 seconds, the interval between images becomes 60.6 seconds, and after 1000 frames you are 10 minutes late. The correct approach is “fixed schedule”: compute the next capture time based on a monotonic clock or wall clock, then sleep only until that target. Another approach is to align captures to wall clock boundaries (e.g., every minute on the minute). This requires using time.monotonic() for internal timing and adjusting for missed frames. If a capture is late, you decide whether to skip or catch up.

Storage retention is the second pillar. A timelapse produces many files; at 500 KB per image, a capture every minute yields ~720 images per day (~360 MB). On a 16 GB SD card, you can fill storage in a month. A retention policy might keep the last N days, delete older files, or compress older images into a video. Your policy should be explicit, deterministic, and tested. You must also consider filesystem wear; frequent deletes and writes can fragment the SD card. To mitigate, you can store in daily directories, delete whole directories, and avoid renaming large numbers of files.

Robustness requires monitoring for failures: camera not detected, capture command failure, or files with zero size. Your pipeline should detect these events and either retry or log and continue. It should also emit a daily summary: number of images captured, disk usage, and errors. This summary serves as the health indicator for the timelapse system.

How this fit on projects

This concept is used in §3 and §5.10. It also feeds into the capstone project where images are part of a larger telemetry pipeline.

Definitions & key terms

  • libcamera: Camera framework used on Raspberry Pi OS.
  • Timelapse: Scheduled series of images over time.
  • Drift: Accumulated timing error from naive scheduling.
  • Retention policy: Rules for keeping or deleting data.

Mental model diagram (ASCII)

Schedule -> Capture -> Store -> Retain/Delete -> Report

How it works (step-by-step, with invariants and failure modes)

  1. Initialize camera and verify with test capture.
  2. Calculate next capture time (invariant: based on fixed schedule).
  3. Capture image and validate file size.
  4. Enforce retention policy.
  5. Emit daily summary.

Failure modes:

  • Camera not detected -> capture fails.
  • Drift -> inconsistent intervals.
  • Disk full -> writes fail.

Minimal concrete example

rpicam-still -o images/$(date +%F_%H-%M-%S).jpg -t 1

Common misconceptions

  • “Sleep for N seconds is enough.” It causes drift.
  • “Disk space will be fine.” Timelapse grows quickly.
  • “Camera errors are rare.” They happen with power or cable issues.

Check-your-understanding questions

  1. Why does naive sleeping cause schedule drift?
  2. How can you detect a corrupted capture?
  3. What is a safe retention policy for a 16 GB SD card?

Check-your-understanding answers

  1. Capture time adds to sleep, accumulating error.
  2. Check for zero-size files or decode errors.
  3. Keep last N days based on size-per-day calculation.

Real-world applications

  • Construction monitoring, plant growth studies, remote security.

Where you’ll apply it

  • This project: §3.2, §5.10.
  • Other projects: Project 17.

References

  • Raspberry Pi camera documentation
  • “Making Embedded Systems” — reliability and scheduling

Key insights

Long-running capture is a scheduling and storage problem more than a camera problem.

Summary

A timelapse pipeline must schedule captures without drift, detect failures, and keep storage under control.

Homework/Exercises to practice the concept

  1. Compute storage usage for different resolutions.
  2. Compare naive sleep vs fixed schedule over 100 captures.
  3. Implement a directory-based retention policy.

Solutions to the homework/exercises

  1. Multiply average file size by captures per day.
  2. Naive sleep accumulates drift; fixed schedule stays aligned.
  3. Deleting whole directories reduces fragmentation.

3. Project Specification

3.1 What You Will Build

A headless timelapse system that captures images on schedule, logs results, and enforces storage limits.

3.2 Functional Requirements

  1. Capture images every N seconds with no drift.
  2. Log each capture with timestamp and size.
  3. Enforce retention policy (e.g., last 7 days).
  4. Generate daily summary report.

3.3 Non-Functional Requirements

  • Performance: Capture completes within 1 second.
  • Reliability: >99% successful captures in 24 hours.
  • Usability: Logs and summaries are readable.

3.4 Example Usage / Output

$ ./timelapse
Capturing every 60 seconds
Saved: images/2026-01-01_10-00-00.jpg
Daily report: 720 images, 1.4 GB used

3.5 Data Formats / Schemas / Protocols

Daily summary JSON:

{"date":"2026-01-01","captures":720,"errors":2,"disk_used_gb":1.4}

3.6 Edge Cases

  • Camera disconnected.
  • Disk nearly full.
  • Time sync changes (NTP adjustment).

3.7 Real World Outcome

A stable timelapse produces consistent images without filling the SD card.

3.7.1 How to Run (Copy/Paste)

python3 timelapse.py --interval 60 --retain-days 7 --out images/

3.7.2 Golden Path Demo (Deterministic)

export FIXED_TIME="2026-01-01T10:00:00Z"
python3 timelapse.py --simulate --captures 3

Expected output:

[2026-01-01T10:00:00Z] Captured image_0001.jpg

3.7.3 Failure Demo (Deterministic)

python3 timelapse.py --simulate --disk-free-mb 0

Expected output:

[ERROR] Disk full, retention cleanup failed

Exit code: 71

3.7.4 CLI Exit Codes

  • 0: Success
  • 70: Camera not detected
  • 71: Disk full

4. Solution Architecture

4.1 High-Level Design

Scheduler -> Capture -> Validate -> Retain -> Report

4.2 Key Components

| Component | Responsibility | Key Decisions | |—|—|—| | Scheduler | Fixed-interval timing | Monotonic vs wall clock | | Capture Engine | Invoke rpicam | Resolution + quality | | Retention Manager | Delete old data | Days vs size threshold | | Reporter | Daily summary | JSON vs text |

4.3 Data Structures (No Full Code)

stats = {"captures": 0, "errors": 0, "bytes": 0}

4.4 Algorithm Overview

Key Algorithm: Fixed Schedule Capture

  1. Compute next capture time.
  2. Sleep until target.
  3. Capture and validate file.
  4. Update retention policy.

Complexity Analysis:

  • Time: O(1) per capture
  • Space: O(1)

5. Implementation Guide

5.1 Development Environment Setup

sudo apt-get install -y rpicam-apps

5.2 Project Structure

project-root/
├── timelapse.py
├── retention.py
└── README.md

5.3 The Core Question You’re Answering

“How do you schedule high-bandwidth captures without destroying storage or missing frames?”

5.4 Concepts You Must Understand First

  1. libcamera/rpicam command usage.
  2. Fixed-schedule timing vs naive sleep.
  3. Storage retention and disk budgets.

5.5 Questions to Guide Your Design

  1. What retention period fits your SD card?
  2. What’s your strategy for missed captures?

5.6 Thinking Exercise

Compute images per day and storage usage at your chosen resolution.

5.7 The Interview Questions They’ll Ask

  1. Why does power quality affect camera stability?
  2. How do you prevent drift in periodic tasks?
  3. How do you design safe retention policies?

5.8 Hints in Layers

Hint 1: Start with low resolution.

Hint 2: Log capture times and verify drift.

Hint 3: Add retention cleanup by day directories.

5.9 Books That Will Help

| Topic | Book | Chapter | |—|—|—| | Camera pipeline | Raspberry Pi Cookbook | Ch. 10 | | Filesystems | The Linux Programming Interface | Ch. 13 |

5.10 Implementation Phases

Phase 1: Capture Bring-up (3 hours)

  • Verify camera with rpicam-hello.

Phase 2: Scheduling (4 hours)

  • Implement fixed-time scheduling.

Phase 3: Retention & Reporting (4 hours)

  • Add cleanup and summary reports.

5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale | |—|—|—|—| | Scheduling clock | Wall / Monotonic | Monotonic | Avoid NTP jumps | | Retention | Days / Size limit | Days | Simple and predictable |

6. Testing Strategy

6.1 Test Categories

| Category | Purpose | Examples | |—|—|—| | Unit Tests | Retention logic | Delete old dirs | | Integration Tests | Capture pipeline | 10 capture run | | Edge Case Tests | Disk full | Simulated low space |

6.2 Critical Test Cases

  1. 10 captures scheduled without drift.
  2. Disk full -> exit code 71.
  3. Missing camera -> exit code 70.

6.3 Test Data

Interval: 60s
Retention: 7 days

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

| Pitfall | Symptom | Solution | |—|—|—| | Naive sleep | Drift accumulates | Fixed schedule | | No retention | Disk fills | Enforce cleanup | | Camera disabled | Capture fails | Enable in config |

7.2 Debugging Strategies

  • Run rpicam-hello to verify camera.
  • Check file sizes for corruption.

7.3 Performance Traps

  • High resolution can exceed write throughput; test disk speed.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add filename overlay with timestamp.

8.2 Intermediate Extensions

  • Build daily timelapse video from images.

8.3 Advanced Extensions

  • Adaptive interval based on light level.

9. Real-World Connections

9.1 Industry Applications

  • Construction monitoring and wildlife cameras.
  • motion Linux video capture software.

9.3 Interview Relevance

  • Scheduling and storage reliability are common systems topics.

10. Resources

10.1 Essential Reading

  • Raspberry Pi camera documentation.

10.2 Video Resources

  • Timelapse tutorials with rpicam.

10.3 Tools & Documentation

  • rpicam-still CLI docs.

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain camera pipeline basics.
  • I can explain drift-free scheduling.

11.2 Implementation

  • Timelapse runs for hours without drift.
  • Retention prevents disk fill.

11.3 Growth

  • I can explain storage budgeting in interviews.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • 1-hour timelapse captured with correct intervals.

Full Completion:

  • 24-hour capture with retention and summary reports.

Excellence (Going Above & Beyond):

  • Adaptive capture based on environment conditions.