Project 2: Scoped Storage Inbox and Safe File Vault

Build a crash-safe boundary that validates Android-visible files, imports trusted copies into private Termux state, and exports deliberate sanitized copies without treating shared storage as an application root.

Quick Reference

Attribute Value
Difficulty Level 1-2 - Beginner to intermediate
Time Estimate 8-14 hours
Language Python (alternatives: Rust, Go, Bash for a reduced version)
Prerequisites Project 1 capability model, filesystem basics, hashing, JSON, transactions
Key Topics Android storage classes, noexec, path security, atomic commit, SQLite manifests, idempotency, redacted export

1. Learning Objectives

By completing this project, you will:

  1. Classify executable code, secrets, mutable state, cache, imports, exports, and package-owned files by trust and lifetime.
  2. Explain why Android shared storage is an interchange surface rather than a safe Unix program tree.
  3. Defend a named inbox against traversal, symlinks, surprising file types, oversized input, and archive expansion.
  4. Stream input through hashing and validation without loading an arbitrary file into memory.
  5. Use private staging and one explicit atomic visibility point.
  6. Make import replay safe across process death with content identity, operation identity, and a transactional manifest.
  7. Export sanitized copies without exposing live private databases, keys, or internal paths.
  8. Demonstrate recovery at multiple forced interruption points.

2. Theoretical Foundation

2.1 Core Concepts

Android offers storage according to purpose. App-private internal storage holds data meaningful only to one application and is the right default for sensitive or operational state. Shared storage holds media, documents, and other user-visible files meant to cross application boundaries. The Storage Access Framework gives user-mediated access to selected documents and directories. Scoped storage limits broad ambient access on modern Android releases.

Inside Termux, HOME and PREFIX are private application storage. They provide the filesystem behavior most Unix tools expect and normally keep other apps out. Paths under ~/storage commonly point into Android-visible storage prepared for exchange. The official Termux execution documentation warns that shared/public storage uses emulated filesystem semantics, is mounted noexec, cannot reliably provide ordinary symlinks and Unix metadata, and can be modified by other authorized actors. Executing a file from there is both fragile and a security risk.

The primary design question is therefore not “where is there space?” It is “who can change these bytes, who must read them, and what happens if they disappear?”

Data Class Recommended Location Reason
Executable code Private HOME or package-owned PREFIX Requires trusted content and executable semantics
Secrets and keys Private state with restrictive access Must not be visible to other apps
Database and queue Private state Needs transactions, locks, and durable consistency
Rebuildable cache Private cache subtree Can be deleted under a documented policy
User-supplied import Named shared inbox until validated Explicit untrusted boundary
User-facing export Named shared export or SAF destination Deliberate copy for exchange
Installed templates/manuals Package-owned PREFIX Replaced through package lifecycle

Classic and Play track caveat

This guide’s concrete Termux directory examples follow the classic F-Droid/GitHub track. If optional classic companion apps are installed later, the main Termux app and every plug-in must come from the same signing lineage; never mix F-Droid and GitHub APK families. The current Google Play track has a different built-in integration surface, including selected SAF/storage tools. Detect the active track with Project 1 and adapt the ingress mechanism rather than installing a classic add-on into Play.

The core vault architecture does not depend on a specific track: shared bytes are untrusted input, private state remains authoritative, and user-granted access is checked at runtime.

The inbox-validate-stage-commit pattern

The vault admits data through a pipeline whose intermediate states are observable:

 Android-visible inbox                Termux-private filesystem
 +-----------------------+            +------------------------------+
 | untrusted candidate   |            | staging/ operation-7f31      |
 +-----------+-----------+            | objects/ content-addressed   |
             |                        | manifest database            |
             v                        +---------------+--------------+
       open defensively                                ^
             |                                         |
    type / size / path / quota                          |
             |                                         |
       stream + digest ---------------------------------+
             |                    private temporary copy
          validate                                      |
             |                                          |
             +---- reject/quarantine     flush -> rename -> manifest

 Export is a new, sanitized copy. The private object is never shared in place.

Each stage has a narrow responsibility:

  1. Discover: Accept an explicit candidate under one authorized inbox; do not scan the whole device.
  2. Open: Refuse surprising object types and untrusted link behavior.
  3. Bound: Enforce raw size, expected expansion, count, nesting, and total quota.
  4. Identify: Stream a cryptographic digest while copying.
  5. Validate: Parse content into a typed internal representation.
  6. Stage: Store the candidate on the private destination filesystem.
  7. Commit: Flush/close and atomically publish the complete object.
  8. Record: Transactionally write provenance and final disposition.
  9. Dispose: Retain, archive, or delete the shared original according to explicit policy.

Atomicity, durability, and cross-filesystem reality

Atomicity means readers observe either the old complete state or the new complete state. A rename can be an atomic visibility point when source and destination are on the same filesystem and the surrounding implementation respects platform semantics. Moving directly from shared storage into private storage crosses filesystems, so the operation is a copy—not an atomic rename. The safe approach is to stream from shared input into a private temporary object, verify it, then rename within the private filesystem.

Durability is separate. An atomic name change does not automatically prove that every byte and directory entry survived sudden power loss. For the learning project, define the durability guarantee explicitly: flush file content, close it, publish through a same-filesystem rename, and commit the manifest transaction only when the final object is addressable. Where the runtime exposes directory synchronization, investigate it and document device/filesystem limits. Do not claim a stronger guarantee than you tested.

SQLite provides a transactional commit point for manifests and operation states. It does not make an external file write part of the same transaction. Your protocol must order file and database transitions so recovery can reconcile either side.

Content identity and idempotency

An operation ID identifies an attempt. A content digest identifies exact bytes. A logical import identity may include the digest, declared object type, target collection, and parser version. They answer different questions:

  • “Did this request already run?” → operation identity.
  • “Have these exact bytes already committed?” → content identity.
  • “Should these bytes create another logical record here?” → import policy.

If Android kills the importer after object publication but before the final manifest update, replay must find and reconcile the complete object rather than copy it again. If the kill occurs during staging, replay must recognize an incomplete leased operation and either resume or quarantine it. A uniqueness rule over the chosen logical identity prevents duplicates.

Path and object security

Rejecting ../ in a string is not enough. Between validation and open, a path can change. A symlink can point outside the inbox. An archive can contain absolute paths, traversal segments, link entries, duplicate names, case collisions, device files, or a tiny compressed stream that expands beyond storage.

The stronger model anchors operations to a trusted directory and validates the object actually opened:

  • Candidate must be under the authorized inbox after canonical resolution.
  • Opened object must be a regular file when the profile requires one.
  • Symbolic links and hard-link-like archive entries are rejected unless a narrowly specified format needs them.
  • File size is checked before and during streaming.
  • Final bytes are hashed and validated; metadata alone is not trusted.
  • Archive extraction writes only to a fresh private staging directory and enforces file-count, depth, per-file, total-size, and ratio limits.
  • File names are normalized according to a documented collision policy; user names never become executable paths.

This reduces TOCTOU exposure but does not magically remove every race in high-level language APIs. State the remaining assumptions and test with hostile fixtures.

Provenance, retention, and export

Every accepted object should have provenance: import ID, content digest, original display name, source class, observed size/type, validation profile, parser version, import time, and disposition. Avoid storing an absolute shared path if it reveals user information or will become stale. Preserve only what supports audit and recovery.

Export is not “make the vault world-readable.” It is a new object designed for another audience. A report exporter can omit internal IDs, precise location, raw network names, secrets, private paths, and debugging evidence. It writes to a temporary export, validates the final schema, publishes a user-visible copy, and records which redaction profile produced it.

2.2 Why This Matters

File ingestion is a universal trust-boundary problem. The same design appears in document upload services, backup systems, offline mobile capture, package managers, and media pipelines. Learning it on a phone makes filesystem differences, process interruption, user-controlled names, and limited storage impossible to ignore.

2.3 Historical Context / Background

Older Android workflows often treated shared storage as a broadly writable SD card. Modern Android moved toward purpose-based and scoped access, while Termux retained a private executable userspace. The resulting split is deliberate: private storage supports trusted application behavior, while shared storage and SAF support user-mediated exchange. Robust tools embrace that split instead of recreating an unsafe desktop filesystem assumption.

2.4 Common Misconceptions

  • “chmod +x makes shared storage safe for scripts.” The mount and trust boundary still make execution unreliable and dangerous.
  • “A normalized path is a safe object.” The object may be a link or change after the check.
  • “Rename is always atomic.” Cross-filesystem moves are copy operations.
  • “SHA-256 means the content is safe.” A digest identifies bytes; it does not validate meaning.
  • “Private storage is backed up.” App uninstall, device loss, or source migration can remove it.
  • “Exactly once means no retries.” Safe replay is usually achieved through idempotency and reconciliation.

3. Project Specification

3.1 What You Will Build

Build a pocketctl vault domain with these operations:

  • plan-import: Read-only classification and resource estimate.
  • import: Stream, hash, validate, privately stage, publish, and record an object.
  • show: Display safe provenance and state.
  • recover: Reconcile expired staging operations and ambiguous commits.
  • export: Create a redacted copy in an authorized destination.
  • verify: Rehash a private object and compare it with its manifest.
  • gc-plan: Report safe cleanup candidates without deleting them.

The reference fixture is a field-report JSON document. Add archives only after the single-file contract is complete.

3.2 Functional Requirements

  1. Named boundaries: Only configured inbox/export roots are accepted.
  2. Read-only plan: Planning reports policy decisions without creating durable state.
  3. Streaming import: Copy and hash under hard memory and size limits.
  4. Typed validation: Imported JSON must satisfy a versioned internal schema.
  5. Private staging: Temporary objects reside on the private destination filesystem.
  6. Atomic publication: Readers never observe a partial final object.
  7. Manifest transaction: Provenance and state transitions are durable and queryable.
  8. Safe replay: Repeating the same logical import creates one object and record.
  9. Visible recovery: Expired or ambiguous operations are recovered or quarantined with reasons.
  10. Sanitized export: The exported copy is independently generated and validated.

3.3 Non-Functional Requirements

  • Performance: Memory usage remains bounded while importing a file at the configured maximum.
  • Reliability: Kill tests at five transitions never expose partial final content.
  • Security: No executable, link escape, traversal, or oversized expansion crosses into the vault.
  • Usability: Rejections explain rule, observed value, allowed value, and safe next action.
  • Auditability: Every committed object and recovery decision has immutable provenance fields.

3.4 Example Usage / Output

ImportOperation:
    operation_id
    state = PLANNED | STAGING | VALIDATED | PUBLISHED | COMMITTED | QUARANTINED
    lease_owner?
    lease_expires_at?
    candidate_identity
    content_digest?
    staging_name?
    final_object_id?
    validation_profile
    failure_code?

VaultObject:
    object_id = content-addressed identifier
    digest
    byte_count
    media_type
    schema_version
    created_at
    verification_state

ExportRecord:
    export_id
    source_object_id
    redaction_profile
    output_digest
    destination_class
    committed_at

3.5 Real World Outcome

Create an Android-visible PocketOps-Inbox and place report-042.json inside it using a file manager or another app. Run plan-import. The terminal shows the display name, observed size, expected media type, authorized inbox, estimated private-space requirement, parser profile, and intended original-file disposition. It performs no durable change.

Run the real import. A progress-free machine path streams the file while the human view prints completed phases only. The object is copied into a private temporary area, hashed, parsed, published into a content-addressed object tree, and recorded once in the manifest. The shared original remains or is removed according to the selected explicit policy; default behavior keeps it.

Run the same import again. The tool returns ALREADY_IMPORTED, names the original import ID, and creates neither a second object nor a second logical record. Then repeat with a fault injector that terminates the importer during copy, after validation, after publication, and before manifest commit. recover reports each abandoned operation and either safely resumes, reconciles, or quarantines it.

Finally, export the report with a public redaction profile. Open the exported JSON from an Android file manager. It contains the allowed field report but no private vault path, operation lease, raw location, secret, or internal database identifier.

$ pocketctl vault import ~/storage/shared/PocketOps-Inbox/report-042.json
IMPORT  report-042.json
Size    18.2 KiB                 OK
Type    application/json         OK
Digest  sha256:8c19...b72a       VERIFIED
Stage   private/tmp/import-7f31  COMMITTED
Vault   objects/8c/19/8c19...    READY
Record  import_id=7f31           CREATED

Result: IMPORTED (safe to replay)

You can see the result in three places: the terminal contract, the private manifest through vault show, and the Android-visible sanitized export. The private object itself remains inaccessible to ordinary file-manager browsing.

3.6 Edge Cases

  • Empty file, file exactly at maximum, and file growing during read.
  • Candidate replaced between planning and import.
  • Symlink, directory, socket-like object, broken link, or inaccessible object.
  • Traversal name and canonical path outside inbox.
  • Two concurrent imports of identical content.
  • Same bytes imported under different declared logical types.
  • Valid JSON with unknown schema version or extreme nesting.
  • Archive with absolute names, links, case collisions, many tiny files, or extreme ratio.
  • Storage exhaustion before copy, before publication, and before manifest commit.
  • Export destination revoked, removed, or filled during write.
  • Uninstall/source migration restoring manifest without corresponding object files.

4. Solution Architecture

4.1 High-Level Design

 Android file manager / SAF / shared inbox
                    |
                    v
          +---------------------+
          | boundary validator  |
          | root/type/size/link |
          +----------+----------+
                     |
            bounded byte stream
                     v
          +---------------------+
          | hash + parser       |
          | schema + policy     |
          +-----+-----------+---+
                |           |
             reject      private stage
                |           |
          quarantine        v
                     flush + atomic publish
                            |
                 +----------+----------+
                 | manifest transaction|
                 +----------+----------+
                            |
                    authoritative vault
                            |
                  redaction + export copy
                            v
                  Android-visible export

4.2 Key Components

Component Responsibility Key Decision
Boundary policy Authorize inbox/export roots and object classes Default deny outside named roots
Candidate opener Inspect and open the actual object Reject links/special files for v1
Stream guard Enforce byte, time, and memory limits Check limits continuously, not only metadata
Content validator Parse and normalize declared types Hash identity does not imply semantic validity
Staging store Hold private incomplete objects Same filesystem as final object tree
Object publisher Publish a complete immutable object One rename is the visibility point
Manifest repository Journal operations and provenance Unique logical import constraint
Recovery engine Reconcile leases, stages, objects, and rows Never delete ambiguous evidence silently
Exporter Create redacted user-facing copies Never share the live private object

4.3 Data Structures

Use the structures in Section 3.4 plus a declarative policy:

profile: field-report-v1
allowed-source-class: named-shared-inbox
allowed-types: [application/json]
max-bytes: bounded-value
max-structure-depth: bounded-value
symlink-policy: reject
original-disposition: retain
logical-identity: digest + collection + schema-version
export-profile: public-field-report-v1

This is an illustrative configuration shape, not a complete runnable configuration.

4.4 Algorithm Overview

Import protocol

  1. Resolve and validate the authorized inbox configuration.
  2. Open the candidate and verify object type and containment.
  3. Create an operation row with a lease.
  4. Stream bytes to a private staging object while enforcing limits and computing a digest.
  5. Parse the staged copy and validate the declared schema.
  6. Check the logical uniqueness rule.
  7. Flush, close, and atomically publish the immutable object.
  8. Transactionally record the object, provenance, and committed operation.
  9. Apply the explicit original-file policy.
  10. Return only after the manifest can reproduce the committed result.

Recovery protocol

  1. Find expired nonterminal operations.
  2. Inspect staging, final object, and manifest state.
  3. If final object and digest are complete, reconcile the manifest.
  4. If only incomplete staging exists, resume where proven safe or quarantine.
  5. If state is contradictory, preserve evidence and require intervention.

Complexity analysis

  • Import time: O(n) for n input bytes, plus parser cost.
  • Import space: O(n) private durable space and O(1) bounded streaming memory for non-archive input.
  • Deduplication lookup: O(log m) with an indexed manifest of m objects.

5. Implementation Guide

5.1 Development Environment Setup

Complete Project 1 first and record the active Termux track. On classic, prepare shared-storage access through official Termux mechanisms and keep every optional plug-in from the main app’s same F-Droid or GitHub signing family. On Play, use the track’s current built-in SAF/storage capability where appropriate rather than mixing APK families.

Use private test directories beneath HOME for the vault and manifest. Create a dedicated shared inbox and export directory; do not point a learning tool at all of Downloads. Seed synthetic reports only—no real location, messages, or credentials.

5.2 Project Structure

scoped-storage-vault/
├── docs/
│   ├── data-classification.md
│   ├── import-protocol.md
│   └── recovery-table.md
├── schemas/
│   ├── field-report-v1.schema.json
│   └── public-export-v1.schema.json
├── src/
│   ├── boundary/
│   ├── streaming/
│   ├── validation/
│   ├── manifest/
│   ├── recovery/
│   └── export/
├── fixtures/
│   ├── safe/
│   └── hostile/
├── tests/
└── README.md

5.3 The Core Question You’re Answering

“How can untrusted, user-visible files cross into private application state without becoming code, corruption, or duplicate work?”

Before implementing, classify every byte in the system. If a file has no declared trust class and owner, it has no valid path yet.

5.4 Concepts You Must Understand First

  1. Android private/shared/scoped storage: Which actor can access each class?
  2. Termux execution constraints: Why shared storage is noexec and semantically different.
  3. Atomic visibility: What a same-filesystem rename can guarantee.
  4. Durability: Why flush, close, publish, and manifest ordering matter.
  5. TOCTOU: Why checking a path string does not freeze an object.
  6. Idempotency: How attempt, content, and logical identities differ.
  7. Content validation: Why MIME hints and hashes are not parsers.

5.5 Questions to Guide Your Design

  • What exact directories are authorized for input and output?
  • Which profile determines type, maximum bytes, depth, and retention?
  • At what line in the protocol does the object become visible to readers?
  • How does recovery decide between resume, reconcile, quarantine, and delete?
  • Which provenance fields remain useful after the shared original moves?
  • How are two concurrent identical imports serialized or deduplicated?
  • What does “sanitized” mean for each export audience?

5.6 Thinking Exercise

Trace these interruption points on paper:

  1. After operation row creation but before staging.
  2. Mid-copy.
  3. After validation but before publication.
  4. After publication but before manifest object row.
  5. After object row but before response.

For each, draw existing files and rows, identify what replay sees, and choose a recovery action. Then repeat for two concurrent imports with the same content digest.

5.7 The Interview Questions They’ll Ask

  1. “What is a TOCTOU race?”
  2. “When is rename atomic, and when is a move actually copy-plus-delete?”
  3. “Why should executable code stay out of Android shared storage?”
  4. “How do you make a file import idempotent?”
  5. “How do you defend archive extraction from traversal and zip bombs?”
  6. “What is the difference between atomicity and durability?”

5.8 Hints in Layers

Hint 1: Finish single-file import first

Archives multiply path, quota, count, and expansion risks. Do not add them until recovery for one JSON file is proven.

Hint 2: Stage on the destination

Copy into a private temporary object on the same filesystem as final storage, then publish there.

Hint 3: Journal before side effects

Create an operation identity before copying so every abandoned temporary object has an owner.

Hint 4: Reconcile, do not guess

Recovery should compare digest, object presence, and manifest state. Contradiction becomes quarantine, not silent deletion.

5.9 Books That Will Help

Topic Book Chapter
File I/O and metadata “The Linux Programming Interface,” Michael Kerrisk Ch. 4, 5, and 14
Directories and links “The Linux Programming Interface” Ch. 18
Transactions and recovery “Designing Data-Intensive Applications,” Martin Kleppmann Ch. 7
Secure file handling “Secure Programming HOWTO,” David A. Wheeler Input validation and filesystem sections

5.10 Implementation Phases

Phase 1: Boundary and Single-File Plan (2-3 hours)

Goals

  • Define data classes, roots, policies, and manifest states.
  • Build a read-only plan for one JSON profile.

Tasks

  1. Write the data-classification table.
  2. Create safe and hostile path fixtures.
  3. Define operation/object tables and uniqueness rules.
  4. Produce deterministic plan output with no durable mutation.

Checkpoint: Traversal, link, wrong type, and oversized candidates are rejected before staging.

Phase 2: Commit and Recovery (4-6 hours)

Goals

  • Implement bounded stream/hash/validate/publish.
  • Make every interruption state recoverable or quarantined.

Tasks

  1. Add private staging and continuous size limits.
  2. Validate the staged copy against the schema.
  3. Publish with one same-filesystem visibility point.
  4. Add fault injection around every durable transition.
  5. Implement expired-lease recovery and reconciliation.

Checkpoint: Five kill tests create one logical object or one visible quarantine, never a partial final file.

Phase 3: Export, Quotas, and Acceptance (2-5 hours)

Goals

  • Generate safe Android-visible output.
  • Prove quota, concurrency, replay, and privacy behavior.

Tasks

  1. Define public redaction and export schemas.
  2. Add concurrent-identical-import tests.
  3. Inject storage exhaustion at multiple phases.
  4. Add verify and a non-destructive garbage-collection plan.
  5. Open the export in another Android application.

Checkpoint: Export is independently valid and contains none of the seeded private canaries.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Object naming User filename; random ID; content address Content address plus manifest display name Stable deduplication without trusting names
Original disposition Delete; move; retain Retain by default Avoid surprising data loss during learning
Manifest Flat JSON; SQLite SQLite Transactions, uniqueness, and indexed recovery
Archive support Immediate; deferred Defer until single-file protocol passes Reduces attack surface and complexity
Ambiguous recovery Delete; assume success; quarantine Quarantine with evidence Preserves safety and auditability
Export Share live file; copy Generate a sanitized copy Prevents aliasing mutable private state

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Validate pure rules Root containment, state transitions, redaction
Property Explore hostile names and structures Traversal strings, Unicode, case collisions, depth
Integration Exercise private filesystem and SQLite Publish, uniqueness, verify, recovery
Fault injection Prove interruption behavior Kill at each transition, disk-full simulation
Concurrency Prevent duplicate logical imports Two identical candidates at once
Android acceptance Prove real shared/private boundary File manager import/export, revoked access

6.2 Critical Test Cases

  1. Happy single-file import: One immutable object and one committed manifest row.
  2. Replay: Second request returns original result without duplicate state.
  3. Mid-copy kill: Incomplete staging remains owned by an expired operation.
  4. Post-publication kill: Recovery reconciles the object into the manifest.
  5. Traversal and link: Candidate is rejected before content parsing.
  6. Concurrent duplicate: One logical commit wins; the other returns the committed identity.
  7. File growth: Streaming maximum stops an object that grew after metadata check.
  8. Disk full: Last complete vault state remains valid.
  9. Malformed/deep JSON: Parser applies structural bounds and quarantines evidence safely.
  10. Export canaries: Private path, secret, raw location, and operation lease do not appear.
  11. Permission revoked: Tool returns a storage capability error, not “file missing.”
  12. Track adaptation: Play storage ingress does not trigger classic add-on installation advice.

6.3 Test Data

safe/report-small.json          valid field-report-v1
safe/report-max.json            exactly configured byte limit
hostile/empty                   zero bytes
hostile/deep.json               structure over nesting limit
hostile/growing.fixture         expands while being streamed
hostile/link-outside            symbolic link escaping inbox
hostile/name-../report.json     traversal display name
hostile/archive-bomb.fixture    extreme declared expansion
canaries/private.json           contains SECRET_CANARY and LOCATION_CANARY

Use synthetic fixtures. Never seed real keys, clipboard contents, or precise location.

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Running code from shared storage Permission error or modified code execution Keep code in HOME/PREFIX; import data only
Trusting extension/MIME hint Invalid content enters parser Validate bounded bytes and schema
Cross-filesystem rename assumption Partial/slow “move” after kill Copy to private stage, then rename privately
Path-string-only validation Link escapes inbox Validate the opened object relative to trusted root
Manifest after response Replay duplicates object/row Commit recoverable state before success response
Cleanup by age alone Active import is deleted Require expired lease and reconciled ownership
Exporting live database Other app observes or alters state Generate a separate sanitized artifact

7.2 Debugging Strategies

  • Query the operation state before inspecting raw temporary files.
  • Compare staged and final digests; never infer completeness from file name.
  • Inspect which filesystem contains staging and final objects.
  • Reproduce parsing with the immutable private stage, not a changing shared source.
  • Use a non-destructive recover --plan view before applying cleanup.
  • Verify storage permission and track through Project 1 before changing filesystem logic.

7.3 Performance Traps

  • Reading whole files or archives into memory.
  • Hashing the shared source once, then copying it in a separate pass that can see different bytes.
  • Running content validation repeatedly instead of persisting parser version and result.
  • Keeping a long database transaction open during the entire file copy.
  • Extracting archives before computing a strict resource plan.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add CSV input with a strict column and row limit.
  • Add vault diff for two normalized field reports.
  • Add an export receipt containing digest and redaction profile.

8.2 Intermediate Extensions

  • Add encrypted backup export with a separate verified restore exercise.
  • Add a user-mediated SAF import adapter while retaining the same core protocol.
  • Add archive support with hostile traversal, links, count, depth, and ratio fixtures.

8.3 Advanced Extensions

  • Use directory-relative file APIs where available to reduce path races.
  • Add content quarantine review and promotion without exposing raw secrets.
  • Design a multi-profile retention engine with legal hold and explicit erasure evidence.

9. Real-World Connections

9.1 Industry Applications

  • Document ingestion: Upload services validate, quarantine, normalize, and commit user files through the same pattern.
  • Offline field capture: Mobile clients keep local authoritative state and export exchange artifacts.
  • Artifact stores: Content-addressed storage separates identity from user-controlled names.
  • Backup systems: Atomic manifests and replay-safe operations distinguish complete from partial backups.
  • Media pipelines: Shared input is decoded under resource and type limits before entering trusted storage.
  • Termux app/packages: Define the private execution and shared-storage constraints.
  • SQLite: Provides local transactional manifests and uniqueness rules.
  • restic/borg concepts: Illustrate content addressing, manifests, and verified storage, though this project is not a reimplementation.
  • Android Storage Access Framework: Demonstrates user-mediated document access rather than ambient filesystem authority.

9.3 Interview Relevance

This project provides concrete examples for filesystem races, atomic commit, transactional recovery, content addressing, untrusted file handling, archive security, and mobile storage boundaries. It is especially useful for backend upload pipelines, desktop sync clients, backup tools, and security-focused interviews.

10. Resources

10.1 Essential Reading

10.2 Video Resources

  • Official Android Developers sessions on scoped storage and the Storage Access Framework.
  • Database transaction and crash-recovery lectures from a university systems course.

10.3 Tools and Documentation

11. Self-Assessment Checklist

11.1 Understanding

  • I can classify every project file by trust, owner, and lifetime.
  • I can explain why shared storage is an interchange boundary, not an execution root.
  • I can distinguish atomic visibility from durability.
  • I can explain attempt, content, and logical import identities.
  • I can trace every recovery state after process death.
  • I understand how classic signing lineage and Play track differences affect setup advice.

11.2 Implementation

  • Import is streaming and hard-bounded.
  • Candidate containment and object type are validated defensively.
  • Staging and final objects are on the private destination filesystem.
  • Readers never observe a partial final object.
  • Concurrent replay creates one logical object and record.
  • Recovery never silently deletes ambiguous evidence.
  • Export is a sanitized copy with its own digest and schema.

11.3 Growth

  • I documented one remaining filesystem assumption I could not fully prove.
  • I can defend my original-file retention policy.
  • I can explain this import protocol in a systems or security interview.

12. Submission / Completion Criteria

Minimum Viable Completion

  • One JSON profile imports from a named shared inbox into private storage.
  • Size, path, type, and schema checks run before publication.
  • One same-filesystem atomic publication point exists.
  • Replay returns one logical object and manifest record.
  • One sanitized export opens from another Android application.

Full Completion

  • All minimum criteria plus transactional operation journal, five interruption tests, concurrent replay, storage exhaustion, verification, recovery/quarantine, provenance, redaction canaries, and track-aware setup notes.

Excellence (Going Above and Beyond)

  • SAF adapter, encrypted backup/restore, securely bounded archive handling, property-based path fixtures, and an independent review of the recovery protocol.

This guide was expanded from TERMUX_ANDROID_APPS_TOOLS_MASTERY.md. See the expanded project index for the complete learning path.