Project 7: Priority Telemetry Scheduler (The Traffic Cop)

Build a downlink scheduler that prioritizes critical telemetry, respects pass bandwidth, and produces deterministic packet transmission plans.

Quick Reference

Attribute Value
Difficulty Level 2: Intermediate
Time Estimate 1-2 weeks
Main Programming Language Python
Alternative Programming Languages C, Rust
Coolness Level Level 3: Ops Strategist
Business Potential Level 2: Mission Ops Tooling
Prerequisites Queues, scheduling, basic networking
Key Topics Downlink budgeting, priority queues, packet aging

1. Learning Objectives

By completing this project, you will:

  1. Model downlink capacity per pass and enforce a bandwidth budget.
  2. Design a priority-based scheduler for telemetry packets.
  3. Implement packet aging and drop strategies under overflow.
  4. Produce deterministic schedules and traceable logs.
  5. Understand how mission ops prioritize health vs payload data.

2. All Theory Needed (Per-Concept Breakdown)

Fundamentals A satellite cannot transmit continuously; it only has contact with the ground during passes. Each pass has a finite duration and data rate, which defines a strict byte budget. If your telemetry queue exceeds that budget, some data will never reach the ground. Therefore you must plan how to use the downlink window: prioritize health data, then critical payload, and drop or delay non-essential data. A scheduler turns a chaotic stream of packets into a deterministic plan.

Deep Dive into the concept Downlink budgeting starts with basic physics: data rate (bps) * pass duration (s) = total bits available. But real systems have overhead: packet headers, forward error correction, and idle gaps. You must account for these, or you will overestimate capacity. A good model includes packet sizes, protocol overhead, and a margin (e.g., 10-20%) for retransmissions or unexpected delays.

Pass prediction provides the duration and start/end times. For this project, you can treat passes as input data: a list of windows with duration and rate. The scheduler uses these windows to plan what to transmit. A common strategy is to separate telemetry into queues by priority: health (highest), event logs (medium), payload data (low). Then you transmit until the budget is exhausted. If you have more data than capacity, you drop low-priority packets or defer them to the next pass.

Packet aging matters: a stale payload packet might be useless after a few hours. The scheduler should consider time-to-live (TTL) for each packet and drop expired data. This prevents backlog. For health telemetry, you might always transmit the latest value rather than all historical values. This is a design decision; the scheduler should expose policies such as “latest-only” or “full history.”

Determinism is essential. If two runs with the same input produce different schedules, operators cannot trust it. Therefore your scheduler should use stable ordering within each priority: by timestamp or sequence number. It should also log why a packet was dropped (expired, over budget, superseded). These logs become operational artifacts for debugging.

How this fit on projects This concept drives Section 3.2 requirements and Section 4 architecture (scheduler), and connects to P01 packet parser.

Definitions & key terms

  • Pass window -> Period of visibility with downlink capacity.
  • Budget -> Total bytes available during a pass.
  • TTL -> Time-to-live for packet validity.
  • Priority queue -> Data structure that serves highest priority first.

Mental model diagram (ASCII)

Telemetry Queues -> Scheduler -> Pass Budget -> Downlink Plan

How it works (step-by-step, with invariants and failure modes)

  1. Compute pass budget from rate and duration.
  2. Sort packets by priority and time.
  3. Fill budget until capacity exhausted.
  4. Drop or defer remaining packets.

Invariants: total bytes <= budget; high-priority packets sent first.

Failure modes: over-budget scheduling, starvation of low priority, non-deterministic order.

Minimal concrete example

for pkt in sorted(queue, key=priority_then_time):
    if used + pkt.size <= budget: send(pkt)
    else: drop(pkt)

Common misconceptions

  • “We can send everything eventually” -> Backlog can grow without bound.
  • “Priority alone is enough” -> Aging and TTL must be enforced.

Check-your-understanding questions

  1. Why must overhead be included in the budget?
  2. What happens if you never drop expired packets?
  3. Why use stable sorting within priority?

Check-your-understanding answers

  1. Overhead reduces usable payload bytes; ignoring it overbooks the pass.
  2. The queue grows indefinitely, delaying newer data.
  3. It ensures deterministic ordering and reproducible schedules.

Real-world applications

  • CubeSat operations planning.
  • Ground system pass scheduling.

Where you’ll apply it

References

  • Space Mission Engineering (Mission operations)
  • CCSDS telemetry standards (packet overhead)

Key insights Downlink is a budget, not a pipe; scheduling is finance.

Summary A deterministic scheduler makes limited downlink usable.

Homework/Exercises to practice the concept

  • Compute pass budget for 9600 bps over 7 minutes with 20% overhead.

Solutions to the homework/exercises

  • Budget = 96004200.8 = 3,225,600 bits (403,200 bytes).

Priority Queues, Aging, and Drop Policies

Fundamentals Priority queues let you always select the most important telemetry first. But pure priority can starve low-priority data forever. Aging is a policy that increases the priority of old packets over time. Drop policies define which packets are discarded when capacity is insufficient. In spacecraft operations, these policies are critical: you must never drop health data, but you might drop old payload data if newer data is available.

Deep Dive into the concept A telemetry scheduler typically maintains multiple queues, one per priority class, or a single priority queue with a composite key. The key might include priority level, timestamp, and TTL. Aging can be implemented by gradually increasing the priority of packets as they approach expiry, or by simply dropping them when TTL is exceeded. The choice depends on mission goals: if payload data is valuable even when delayed, you might age it upward. If payload data is time-sensitive (e.g., event detection), you might drop it instead.

Drop policies are operationally significant. Common strategies include:

  • Drop oldest: remove the oldest packet when queue is full.
  • Drop newest: preserve history but lose recent data.
  • Drop by priority: always drop the lowest priority first.
  • Latest-only: keep only the most recent sample for certain telemetry types.

In spacecraft operations, health telemetry often uses latest-only: only the most recent battery and thermal values matter. For event logs, you might preserve history. For payload imagery, you might drop frames if bandwidth is limited. A robust scheduler supports different policies by packet type. This can be implemented with metadata fields such as category and ttl.

Determinism is critical. Even with aging, the scheduler must produce identical results for the same input. That means the aging function must be deterministic and based only on timestamps, not on floating randomness. It also means the ordering of packets with equal priority should be stable (e.g., by sequence number). The scheduler should log any dropped packets along with their drop reason and any policy that caused it. This gives operators insight into what data was lost.

The scheduler can also provide a “what-if” mode: simulate a pass without sending data to compute how much would be dropped. This is helpful for mission planning and for verifying policy choices. In this project, you can include a report at the end of each pass summarizing bytes sent, bytes dropped, and packets deferred.

How this fit on projects This concept drives Section 3.2 scheduling policies and Section 5.5 design questions.

Definitions & key terms

  • Aging -> Increasing priority of packets over time.
  • Drop policy -> Rule for discarding packets when capacity is limited.
  • Starvation -> Low-priority packets never transmitted.

Mental model diagram (ASCII)

Queues (P1,P2,P3) -> Aging -> Scheduler -> Drop Policy -> Downlink

How it works (step-by-step, with invariants and failure modes)

  1. Assign priority and TTL to each packet.
  2. Periodically age or expire packets.
  3. Schedule packets by priority.
  4. Drop packets based on policy when budget exceeded.

Invariants: health packets never dropped; TTL respected; ordering stable.

Failure modes: starvation, non-deterministic selection, unlogged drops.

Minimal concrete example

if pkt.ttl and now > pkt.created + pkt.ttl:
    drop(pkt, reason="EXPIRED")

Common misconceptions

  • “All data is equally important” -> Health data is always higher priority.
  • “Aging fixes everything” -> It can still starve if budget is too small.

Check-your-understanding questions

  1. Why might you use latest-only for health telemetry?
  2. What is the risk of drop-newest policy?
  3. How does aging prevent starvation?

Check-your-understanding answers

  1. Only current health matters for decisions.
  2. You might lose the most relevant data.
  3. It increases priority of older packets over time.

Real-world applications

  • Prioritized downlink in Earth observation missions.
  • Onboard data management for high-rate payloads.

Where you’ll apply it

References

  • Space Mission Engineering (telemetry scheduling)
  • Data structures texts (priority queues)

Key insights Scheduling is policy encoded in code; every drop is a mission decision.

Summary Priority + aging + drop policies are the heart of downlink planning.

Homework/Exercises to practice the concept

  • Design a drop policy for payload images when bandwidth is half of expected.

Solutions to the homework/exercises

  • Drop low-priority frames first, keep latest health packets, log drops.

3. Project Specification

3.1 What You Will Build

A telemetry scheduler that accepts a packet queue and a pass window and outputs a deterministic transmission plan with logged drops and deferrals.

3.2 Functional Requirements

  1. Pass budget: compute capacity from rate and duration.
  2. Priority scheduling: send higher priority first.
  3. Aging/TTL: expire or age packets.
  4. Drop policy: deterministic drop rules with logging.

3.3 Non-Functional Requirements

  • Determinism: stable schedule for same inputs.
  • Transparency: logs explain all drops.
  • Performance: schedule 10k packets quickly.

3.4 Example Usage / Output

$ python sched.py --pass passes.json --queue packets.json
[PASS 1] sent=120 pkts dropped=30 deferred=50

3.5 Data Formats / Schemas / Protocols

Packet JSON schema:

{"id":"pkt123","size":128,"priority":1,"created":120,"ttl":600}

3.6 Edge Cases

  • Budget smaller than one packet.
  • All packets expired.
  • Burst of high-priority packets.

3.7 Real World Outcome

A deterministic downlink plan and summary report for each pass.

3.7.1 How to Run (Copy/Paste)

python sched.py --passes passes.json --queue packets.json --seed 42

3.7.2 Golden Path Demo (Deterministic)

  • Use fixed passes.json and packets.json.
  • Output must match golden_schedule.json.

3.7.3 Failure Demo (Deterministic)

python sched.py --passes tiny_pass.json --queue packets.json

Expected: logs OVERBUDGET and drops low priority; exit code 2.

3.7.4 If CLI: Exact Terminal Transcript

$ python sched.py --passes passes.json --queue packets.json
[PASS 1] sent=120 dropped=30 deferred=50
ExitCode=0

4. Solution Architecture

4.1 High-Level Design

Packet Queue -> Policy Engine -> Scheduler -> Transmission Plan -> Report

4.2 Key Components

Component Responsibility Key Decisions
Budget Calculator Bytes per pass Overhead margin
Policy Engine TTL and drops Policy configuration
Scheduler Order packets Stable ordering
Reporter Summaries JSON logs

4.3 Data Structures (No Full Code)

class Packet:
    id: str
    size: int
    priority: int
    created: int
    ttl: int

4.4 Algorithm Overview

Key Algorithm: Priority scheduling

  1. Filter expired packets.
  2. Sort by priority then time.
  3. Fill budget until capacity exhausted.

Complexity Analysis:

  • Time: O(n log n) for sorting.
  • Space: O(n).

5. Implementation Guide

5.1 Development Environment Setup

python -m venv .venv
source .venv/bin/activate

5.2 Project Structure

project-root/
+-- sched.py
+-- passes.json
+-- packets.json
+-- README.md

5.3 The Core Question You’re Answering

“When you can only send a little, what do you send first?”

5.4 Concepts You Must Understand First

  1. Budgeting and overhead.
  2. Priority queues.
  3. Drop policies.

5.5 Questions to Guide Your Design

  1. What overhead margin should you assume?
  2. Should health data be latest-only?
  3. How do you log drops for ops review?

5.6 Thinking Exercise

Compute pass budget for 4 minutes at 19.2 kbps with 15% overhead.

5.7 The Interview Questions They’ll Ask

  1. “Why not just send packets FIFO?”
  2. “How do you prevent starvation?”
  3. “How do you handle expired data?”

5.8 Hints in Layers

Hint 1: Start with a simple priority sort.

Hint 2: Add TTL expiration and drop reasons.

Hint 3: Add overhead margin and verify budget.

Hint 4: Add summary reports per pass.

5.9 Books That Will Help

Topic Book Chapter
Mission ops Space Mission Engineering Pass planning
Scheduling Operating Systems texts Scheduling algorithms
Data structures Sedgewick, Algorithms Priority queues

5.10 Implementation Phases

Phase 1: Budgeting (2-3 days)

Goals: compute bytes per pass. Tasks: implement pass parser and budget calculation. Checkpoint: budget matches hand calculations.

Phase 2: Scheduling (3-4 days)

Goals: prioritize packets. Tasks: implement stable sorting and selection. Checkpoint: schedule matches expected order.

Phase 3: Policies (2-3 days)

Goals: add TTL and drop policies. Tasks: implement expiration and logging. Checkpoint: expired packets dropped with reason.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Policy config hard-coded / file file Easier to tune
Ordering stable sort / heap stable sort Deterministic
Drop policy oldest / lowest priority lowest priority Preserve health data

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Tests Budget calc known pass windows
Integration Tests Scheduling golden_schedule.json
Edge Case Tests tiny budget overbudget pass

6.2 Critical Test Cases

  1. Overbudget: low-priority packets dropped.
  2. Expired: TTL exceeded packets removed.
  3. Stable order: equal priority keeps time order.

6.3 Test Data

passes.json, packets.json

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Ignoring overhead Budget overrun Apply margin
Non-deterministic ordering Inconsistent outputs Stable sort
No TTL Stale backlog Expire packets

7.2 Debugging Strategies

  • Log scheduling decisions and compare to expectations.
  • Use small synthetic queues for sanity checks.

7.3 Performance Traps

Sorting large queues may be heavy; consider heap if needed.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add CSV report for each pass.

8.2 Intermediate Extensions

  • Implement aging to prevent starvation.

8.3 Advanced Extensions

  • Add optimization for maximum science value under budget constraints.

9. Real-World Connections

9.1 Industry Applications

  • Downlink scheduling for Earth observation satellites.
  • Mission ops planning for short passes.
  • OpenSatKit telemetry schedulers.
  • cFS data management apps.

9.3 Interview Relevance

  • Demonstrates scheduling and prioritization in constrained systems.

10. Resources

10.1 Essential Reading

  • Space Mission Engineering (ops and scheduling).
  • CCSDS 133.0-B-1 (packet overhead).

10.2 Video Resources

  • Mission ops training videos.

10.3 Tools & Documentation

  • Python heapq for priority queues.

11. Self-Assessment Checklist

11.1 Understanding

  • I can compute pass budgets with overhead.
  • I can explain drop policies and their impact.
  • I can justify my prioritization scheme.

11.2 Implementation

  • Scheduler matches golden outputs.
  • Drops are logged with reasons.
  • Deterministic ordering under ties.

11.3 Growth

  • I can propose a science-value optimization.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Compute pass budget and select packets by priority.

Full Completion:

  • TTL and drop policies with logging.

Excellence (Going Above & Beyond):

  • Implement aging and science-value optimization.