Project 32: BEAM Artifact Inspector and Reproducibility Auditor

Build a safe release auditor that inventories .beam files without loading them, explains chunk-level and semantic differences between builds, and enforces policies for debug information, provenance, and deterministic artifacts.

Quick Reference

Attribute Value
Difficulty Level 4: Expert
Suggested Seniority Staff Elixir/BEAM engineer
Time Estimate 22-34 hours
Main Programming Language Elixir with Erlang/OTP tooling
Alternative Programming Languages Erlang
Coolness Level Level 5: Pure Magic
Business Potential Level 2: Micro-SaaS / Pro Tool
Prerequisites Mix releases, modules and exports, binaries, hashing, compilation, supply-chain vocabulary
Key Topics BEAM chunks, :beam_lib, safe inspection, semantic manifests, provenance, reproducible builds, artifact policy

1. Learning Objectives

By completing this project, you will:

  1. Describe a BEAM file as a container of named chunks rather than an opaque binary.
  2. Use :beam_lib to inspect files without loading or executing their modules.
  3. Distinguish raw byte identity, chunk identity, and normalized semantic identity.
  4. Detect malformed, truncated, duplicated, and unexpected module artifacts.
  5. Explain how source paths, compile options, debug information, toolchain versions, and dependency drift affect reproducibility.
  6. Produce a signed-or-hashed manifest that maps release files to semantic and raw digests.
  7. Compare two release trees and classify differences as expected, policy-relevant, or unexplained.
  8. Enforce artifact policy without claiming that static chunk inspection proves code is safe.

2. All Theory Needed (Per-Concept Breakdown)

BEAM Files, Chunks, and Safe Static Inspection

Fundamentals

A compiled Erlang or Elixir module is stored in a BEAM file, a binary container with a header and named chunks. Chunks can carry executable code, atom and literal tables, exports, imports, attributes, compile information, documentation, line data, and optional abstract/debug information. Exact chunk presence and encoding vary with compiler and OTP version. Erlang’s :beam_lib module reads, validates, strips, compares, and extracts supported chunks from a filename or binary. Inspection is different from code loading: :code.load_binary and related APIs make a module executable in the VM, while :beam_lib can analyze untrusted artifacts as data. A safe auditor never loads a candidate merely to call module_info or __info__.

Deep Dive

The first invariant of an artifact inspector is that the inspected bytes remain data. A BEAM module can define load-time behavior and native integration declarations, and loading code changes the running node’s code server state. Even if no function is called intentionally, loading an untrusted artifact is outside the scope of a static inventory. Use :beam_lib.info/1, :beam_lib.chunks/2, validation functions, and direct hashing over file bytes. Run the auditor in a separate process or OS sandbox for defense in depth when artifacts are hostile, but do not confuse process isolation inside one VM with protection from a malicious parser-triggered runtime flaw.

Start with container identity. Derive the path relative to the release root, raw byte size, raw cryptographic digest, and recognized chunk list. Parse the module identity reported by BEAM metadata and compare it to filename expectations. Release layouts may contain the same module name in more than one application or version directory; this is operationally dangerous because code-path order determines which can load. The auditor should index module identity globally and report duplicates with both paths.

Chunk extraction needs an allowlist and resource limits. Request only the chunks needed for the current report instead of decoding every optional structure. Place maximum file size, decompressed literal/debug data, nesting, and total-manifest limits around inspection. Classify :beam_lib errors into malformed container, missing requested chunk, unsupported version/format, I/O failure, and policy condition. A module without debug information is not malformed; a requested optional chunk may legitimately be absent.

Exports and imports form a useful interface inventory. Exports reveal callable function/arity pairs, including compiler-generated entries that policy may normalize. Imports reveal external module/function/arity dependencies referenced by bytecode. These do not provide a complete call graph: dynamic dispatch, apply, protocols, generated modules, and native code complicate that claim. Report them as static tables, not proof of runtime reachability.

Attributes and compile information reveal provenance clues. Source paths, compiler options, behavior declarations, version attributes, and custom metadata may be present. These fields can leak build-host paths or make byte-for-byte builds diverge. They are also version-dependent and should be decoded conservatively. Documentation chunks can provide module/function docs and metadata without loading the module, but absence is common in stripped releases. Abstract code or debug information can enable richer auditing and easier reverse engineering; whether it is allowed in production is a policy choice, not a universal vulnerability.

Chunk hashes let the report localize differences. Hash the exact raw file first, then each chunk as stored or returned by a stable extraction layer. Preserve the extraction method and toolchain version in the manifest because normalized encodings can change. A semantic summary may sort exports/imports and remove declared volatile fields, but it must never replace the raw digest. Keep three levels separate: raw artifact digest for integrity, chunk digest for localization, and semantic digest for controlled comparison.

Malformed or surprising artifacts should not stop the entire release scan by default. Record a per-file error, continue under a bounded error budget, and return a policy failure at the end. A strict mode may stop on first failure for gatekeeping. Never silently skip unreadable .beam files; the manifest must account for every path selected by the release traversal.

How this fits on the project

This concept defines file discovery, safe :beam_lib usage, resource limits, module/interface inventory, chunk hashes, and per-file error classification.

Definitions and key terms

  • BEAM file: Compiled module container consumed by the BEAM code loader.
  • Chunk: Named section containing code or metadata.
  • Raw digest: Hash of the exact file bytes.
  • Chunk digest: Hash localizing identity to one chunk representation.
  • Semantic summary: Normalized, documented interpretation of selected metadata.
  • Code path collision: More than one artifact defining the same module identity.

Mental model diagram

release/lib/app/ebin/example.beam
              |
       read as bytes only
              |
       +------+-------------------------------+
       | BEAM container                       |
       | header | atoms | code | exports       |
       | imports | attributes | compile_info   |
       | docs? | debug/abstract? | line data?  |
       +------+-------------------------------+
              |
       [:beam_lib extraction]
        /       |          \
 raw digest  chunk map  semantic inventory
        \       |          /
          per-module manifest

NEVER: candidate.beam -> code.load -> call module_info

How it works

  1. Traverse the release under explicit include/exclude rules.
  2. Read and hash each selected file with size limits.
  3. Validate container and enumerate available chunks through :beam_lib.
  4. Request supported interface and provenance chunks.
  5. Normalize only documented fields for semantic comparison.
  6. Index module names and report collisions or filename mismatches.
  7. Preserve raw, chunk, and semantic identities separately.
  8. Invariant: no inspected module enters the node’s code server.
  9. Failure modes include corrupt containers, resource bombs, unsupported chunks, duplicate modules, missing optional metadata, and accidental code loading.

Minimal concrete example

MODULE MANIFEST, NOT RUNNABLE CODE
path: lib/billing-2.4.0/ebin/Elixir.Billing.Invoice.beam
module: Elixir.Billing.Invoice
raw_sha256: 32ab...
chunks_present: [atoms, code, exports, imports, attributes, compile_info, docs]
exports: [{create,2}, {total,1}, {__info__,1}]
debug_info: absent
policy_findings: []

Common misconceptions

  • Static inspection requires loading a module to call its metadata functions.
  • Exports and imports form a complete runtime call graph.
  • Debug information in production is always a vulnerability or always harmless.
  • A semantic digest is sufficient for artifact integrity verification.

Check-your-understanding questions

  1. Why preserve a raw digest when a semantic digest exists?
  2. Why can duplicate module identities matter even if files live in different applications?
  3. What should happen when an optional docs chunk is absent?

Check-your-understanding answers

  1. Normalization intentionally ignores bytes; only the raw digest proves exact artifact identity.
  2. Code-path order may select one definition unpredictably or differently across releases.
  3. Record absence as inventory; fail only if an explicit policy requires documentation.

Real-world applications

  • Release content and provenance inventories
  • Supply-chain policy gates
  • Debug-information exposure audits
  • Duplicate module and dependency collision detection

Where you will apply it

  • Project 32 scanner, chunk extractor, module index, manifest, and policy engine

References

Key insight

An artifact auditor should learn everything it can from BEAM files while never making them executable in its own node.

Summary

Treat BEAM modules as chunked containers, account for every artifact, inspect selected metadata under limits, separate raw and normalized identities, and never load candidates for convenience.

Homework/Exercises

  1. Design the manifest behavior for two paths declaring the same module.
  2. Classify abstract-code absence under strict and permissive policies.

Solutions

  1. Keep both entries, emit a release-level collision finding, report code-path precedence if known, and never discard one silently.
  2. It is normal inventory under a permissive policy; strict policy may require absence or presence depending on the organization’s debuggability/exposure standard.

Reproducible Builds, Provenance, and Difference Classification

Fundamentals

A reproducible build produces equivalent artifacts from declared identical inputs. Byte-for-byte equality is the strongest simple result, but build paths, compiler options, source metadata, debug information, dependency resolution, environment variables, native toolchains, and ordering can introduce differences. A reproducibility auditor compares two release manifests at several levels: file-set differences, raw digests, per-chunk digests, and normalized semantic summaries. It also records provenance such as Elixir/OTP versions, dependency lock digest, source revision, build command class, target architecture, and declared environment. The tool must distinguish “same semantics after a documented normalization” from “identical bytes” and must label unexplained differences rather than normalize them away until the report turns green.

Deep Dive

Reproducibility begins before compilation. Define the input closure: source revision and dirty state, dependency lock, fetched package checksums, Elixir and OTP versions, Mix environment and target, relevant compiler flags, release configuration, native compiler/runtime versions, target operating system and architecture, locale/timezone where relevant, and explicitly allowed environment values. Secrets should not enter the manifest, but their presence as undeclared build inputs should be detected through policy where possible.

The release tree includes more than BEAM files. Configuration templates, scripts, native libraries, priv assets, metadata, and application resource files can all affect behavior. The core project specializes in BEAM modules but records every release file’s path, size, and raw digest. Non-BEAM differences are classified at the file level. This prevents a green module report from hiding a changed NIF library or runtime script.

Comparison proceeds from coarse to fine. First compare relative path sets. A missing module, extra application version, or renamed asset is immediately visible. For common files, compare raw digests. If equal, stop. For differing BEAM files, compare chunk sets and chunk digests. Then decode selected stable metadata and generate a semantic diff: exports added or removed, imports changed, attributes changed, debug information toggled, compile options or source paths changed, code chunk changed, documentation changed. If only an approved volatile field differs, classify it as normalized-equivalent while still reporting raw inequality.

Normalization is a policy surface and needs a version. Removing an absolute build-root prefix from a source path may be justified when the relative source is retained. Dropping every compile-info field is not; it can hide an optimization or parse-transform change. Sort unordered collections only when the format treats them as unordered. Record before/after normalized values and the rule ID responsible. A normalization rule must never modify artifacts, only their comparison representation.

Some differences are inherently target-specific. Native code, target architecture, and OTP compiler output may differ across platforms. A cross-target comparison can still check semantic interfaces and expected file-set policy, but should not claim bit reproducibility. Define comparison profiles: strict same-toolchain byte identity, same-platform semantic diagnosis, and cross-target interface inventory. Each profile states what success means.

Provenance manifests themselves need integrity. Canonically order entries, version the schema, hash the manifest, and optionally sign the digest outside the core project. Include tool version and normalization profile. A manifest generated after artifacts are deployed proves what was observed, not necessarily how they were built. A stronger pipeline creates it in the trusted build and verifies it before deployment.

Triaging a mismatch should produce an actionable chain. “Raw digest differs” is insufficient. The report might say: same file path, same chunk set, code and exports equal, compile-info differs only in source root, debug chunk equal, normalized semantic digest equal under rule PATH-ROOT-1. Another mismatch might say code chunk and imports changed while source revision is declared equal—an unexplained policy failure requiring dependency/toolchain investigation.

Reproducibility tests should rebuild in two different absolute directories and compare. A second test changes one declared input, such as compiler version or feature configuration, and confirms the auditor detects it. A third modifies the final artifact after build and ensures raw integrity fails even when semantic parsing still succeeds. Do not optimize normalizations based solely on observed differences; every rule needs a principled reason and a negative test proving it does not conceal meaningful change.

How this fits on the project

This concept defines build provenance, release-tree manifests, comparison profiles, normalization rules, difference explanations, and CI exit policy.

Definitions and key terms

  • Input closure: Complete declared set of values capable of influencing a build.
  • Provenance: Evidence identifying source, dependencies, toolchain, target, and process context.
  • Normalization profile: Versioned rules for semantic comparison of known non-semantic variance.
  • Byte reproducibility: Exact file-byte equality across builds.
  • Semantic equivalence: Equality under a declared limited interpretation, not a substitute for raw identity.
  • Unexplained difference: Mismatch not justified by declared inputs or approved normalization.

Mental model diagram

source + lock + toolchain + config + target
       |                            |
       v                            v
   build A                      build B
       |                            |
       v                            v
  manifest A                    manifest B
       \                            /
        +------ comparison profile-+
                    |
       path set -> raw hash -> chunk hash -> semantic diff
                    |
       identical | normalized-equivalent | expected-target-diff | unexplained

How it works

  1. Capture declared provenance before or during each build.
  2. Walk every release file in stable relative-path order.
  3. Hash raw files and inspect BEAM metadata safely.
  4. Generate a canonical versioned manifest.
  5. Compare path sets, raw hashes, chunks, then semantic summaries.
  6. Apply only profile-approved, visible normalization rules.
  7. Classify each difference and determine policy result.
  8. Invariant: no normalization can turn raw inequality into a claim of byte identity.
  9. Failure modes include undeclared input drift, dependency drift, path leakage, over-normalization, target mismatch, tampering, and non-canonical manifests.

Minimal concrete example

DIFF EXPLANATION
module=Elixir.Billing.Invoice
raw_equal=false
code_chunk_equal=true
exports_equal=true
compile_info_equal=false
difference=source_path
normalization=PATH-ROOT-1
result=normalized_equivalent
claim=NOT byte-reproducible

Common misconceptions

  • A successful compile from the same Git SHA proves reproducibility.
  • Ignoring all compile metadata is harmless.
  • Semantic equivalence proves an artifact was not tampered with.
  • Cross-architecture releases should normally be byte-identical.

Check-your-understanding questions

  1. Why inventory non-BEAM files in a BEAM-focused auditor?
  2. When is path normalization defensible?
  3. What does a differing code chunk under identical declared provenance imply?

Check-your-understanding answers

  1. Runtime behavior also depends on native libraries, scripts, configs, and priv assets.
  2. When it removes only the declared build-root prefix, preserves meaningful relative identity, and is visible/versioned.
  3. There is undeclared input/toolchain drift, nondeterminism, or artifact modification requiring investigation.

Real-world applications

  • CI reproducible-build gates
  • Release promotion verification
  • Software bill-of-material evidence
  • Incident comparison of deployed artifacts

Where you will apply it

  • Project 32 provenance collector, canonical manifest, comparator, normalizer, and CI policy

References

Key insight

Reproducibility is a precise claim about declared inputs and artifact identity, not a green build from the same source revision.

Summary

Capture build inputs, inventory the whole release, compare raw and semantic levels separately, make every normalization visible, and treat unexplained code differences as evidence—not noise to suppress.

Homework/Exercises

  1. Define strict, same-platform semantic, and cross-target comparison profiles.
  2. Design a negative test for a source-path normalization rule.

Solutions

  1. Strict requires identical path sets and bytes; same-platform semantic diagnoses raw differences under limited normalization; cross-target compares expected interfaces/file policies without byte claims.
  2. Change both the root prefix and meaningful relative source path; the rule may remove only the root and must still report the relative-path change.

3. Project Specification

3.1 What You Will Build

Build beam-audit, a CLI/Mix task that scans a Mix release or arbitrary artifact directory, accounts for every file, safely inspects every .beam, detects duplicate modules, records interface and provenance metadata, enforces artifact policy, and compares two canonical manifests with layered difference explanations.

3.2 Functional Requirements

  1. Traverse release files under stable relative paths and explicit exclusions.
  2. Hash every selected file and inspect BEAM files without loading them.
  3. Record module identity, chunk presence, exports, imports, attributes, compile information, docs/debug presence, and per-chunk digests where supported.
  4. Detect module-name collisions, malformed files, and filename/module mismatches.
  5. Enforce versioned policies for debug information, source-path leakage, docs, and unexpected chunks or applications.
  6. Generate a canonical manifest with provenance and tool version.
  7. Compare manifests at path, raw, chunk, and semantic levels.
  8. Apply visible versioned normalization profiles.
  9. Return distinct results for clean, policy violation, incomplete scan, and unexplained reproducibility failure.

3.3 Non-Functional Requirements

  • No candidate module is loaded or executed.
  • File, decoded-data, and total-manifest resource limits are enforced.
  • One malformed file does not disappear from the report.
  • Manifest ordering and digests are deterministic.
  • Every comparison claim states whether it is raw or normalized.
  • The scanner handles large releases with bounded concurrency and memory.

3.4 Example Usage / Output

$ beam-audit scan _build/prod/rel/shop --manifest shop-a.json
release=shop files=418 beam_modules=173 malformed=0
duplicate_modules=0 debug_info_present=0 source_path_leaks=0
manifest_sha256=fd81... result=PASS

$ beam-audit compare shop-a.json shop-b.json --profile same-platform-v1
raw_equal_modules=171 raw_different_modules=2
normalized_equivalent=1 semantic_difference=1
result=FAIL unexplained_difference

3.5 Data Formats / Schemas / Protocols

MANIFEST HEADER
{schema_version, auditor_version, created_at, release_root_label,
 source_revision, lock_digest, elixir_version, otp_version,
 mix_env, target, architecture, normalization_profile}

FILE ENTRY
{relative_path, kind, size, raw_sha256, inspection_status, findings}

BEAM ENTRY ADDITIONS
{module, chunks, chunk_digests, exports, imports, attributes_summary,
 compile_summary, docs_present, debug_info_state, semantic_sha256}

3.6 Edge Cases

  • Truncated or corrupt BEAM container
  • Optional chunk absent under a permissive policy
  • Extremely large literal or debug chunk
  • Duplicate module in two application versions
  • Symlink escaping the release root
  • Same semantic metadata but different raw file bytes
  • Same source revision with changed dependency lock
  • Cross-architecture comparison under strict profile
  • Unknown future chunk name
  • Native library changed while all BEAM modules remain equal

3.7 Real World Outcome

3.7.1 How to Run

$ MIX_ENV=prod mix release
$ beam-audit scan _build/prod/rel/example --manifest build-a.json
$ beam-audit compare build-a.json build-b.json --profile strict-v1

3.7.2 Golden Path Demo

Build the same fixture from two different absolute checkout paths. The raw comparison identifies modules containing path-dependent compile metadata. The versioned semantic profile removes only the root prefix and reports normalized equivalence without calling the artifacts byte-identical. A deliberately modified code chunk remains an unexplained failure.

3.7.3 Exact Terminal Transcript

$ beam-audit explain build-a.json build-b.json Elixir.Example.Parser
module=Elixir.Example.Parser
raw_sha256 equal=false
chunk Code equal=true
chunk ExpT equal=true
chunk Attr equal=true
chunk CInf equal=false
CInf.source build_a=/tmp/a/lib/parser.ex build_b=/tmp/b/lib/parser.ex
normalization PATH-ROOT-1 => lib/parser.ex == lib/parser.ex
classification=normalized_equivalent
byte_reproducible=false semantic_profile_match=true

3.8 Scope Boundary Versus Projects 1-13

Project 9 practices hot code upgrades and release transitions. This project never changes running code and never loads candidate artifacts. It studies compiled BEAM containers, provenance, integrity, policy, and reproducibility. It is also distinct from Project 10 telemetry because all primary evidence comes from release files rather than runtime metrics.

4. Solution Architecture

4.1 High-Level Design

release root + provenance
          |
   [safe path walker]
       /         \
 non-BEAM       BEAM
 raw hash    [bounded beam_lib inspector]
       \         /
       [file/module entries] --> [policy engine]
                  |
          [canonical manifest]
                  |
      manifest A <--> manifest B
                  |
       [layered comparator]
 path -> raw -> chunks -> semantics -> normalization explanation

4.2 Key Components

Component Responsibility Key decision
Path Walker Account for selected release files Never follow escaping symlinks
Raw Hasher Establish exact identity Stream large files under limits
BEAM Inspector Extract supported chunks Never load code; request minimum chunks
Module Index Detect collisions and mismatches Preserve every defining path
Policy Engine Evaluate artifact rules Separate inventory from organization policy
Manifest Writer Canonicalize and hash report Version schema and ordering
Comparator Localize differences Raw claim always precedes normalization
Normalizer Explain approved variance Versioned, narrow, visible rules

4.3 Data Structures (No Full Code)

  • Provenance: source, lock, toolchain, environment class, target, and build labels.
  • FileEntry: relative path, size, raw digest, type, status, findings.
  • BeamEntry: module identity, chunks, interfaces, provenance summary, semantic digest.
  • PolicyFinding: rule ID, severity, evidence, path/module, remediation.
  • Difference: level, before/after evidence, classification, normalization rule.
  • ComparisonProfile: allowed targets, required equality levels, normalizers, policy thresholds.

4.4 Algorithm Overview

Walk paths in sorted order, hash each file, and submit BEAM files to a bounded inspector. Build a global module index and evaluate policies after all files are known. Canonicalize entries and write the manifest. Comparison merges sorted path lists in O(F), then analyzes differing common BEAM entries by chunk and semantic metadata. Module collision indexing is O(M) average with a map for M modules.

Peak memory should be proportional to manifest metadata plus bounded concurrent decoded chunks, not total release bytes. Stream hashes and discard decoded chunk data after producing compact summaries.

5. Implementation Guide

5.1 Development Environment Setup

Use a current Elixir/OTP pair and create fixture releases with combinations of debug info, source paths, docs, duplicate modules, and corrupted files. Run comparison builds in distinct absolute directories.

5.2 Project Structure

beam_audit/
├── lib/
│   ├── mix/tasks/beam.audit.ex
│   └── beam_audit/
│       ├── path_walker.ex
│       ├── hasher.ex
│       ├── inspector.ex
│       ├── chunk_adapter.ex
│       ├── module_index.ex
│       ├── policy.ex
│       ├── manifest.ex
│       ├── comparator.ex
│       ├── normalizer.ex
│       └── renderer.ex
├── test/fixtures/releases/
└── test/

5.3 The Core Question You Are Answering

“How can I prove what is inside a BEAM release and explain why two builds differ without trusting or executing either artifact?”

5.4 Concepts You Must Understand First

  1. BEAM modules, exports/imports, code loading, and application code paths.
  2. Binary containers, chunks, hashing, and canonical serialization.
  3. Compiler/debug metadata and build input provenance.
  4. Raw identity versus normalized semantic comparison.
  5. Safe traversal, symlinks, and resource-limited parsing.

5.5 Questions to Guide Your Design

  1. Which chunks are required, optional, or toolchain-specific?
  2. What is the threat boundary for untrusted artifacts?
  3. Which manifest fields prove exact bytes and which are interpretations?
  4. What may each normalization rule remove without hiding behavior changes?
  5. How does a cross-target profile differ from strict reproducibility?

5.6 Thinking Exercise

Imagine two BEAM files with equal code chunks and exports but different compile information and raw hashes. List what you can claim, what you cannot claim, and which evidence would determine whether the difference is expected path variance or undeclared toolchain drift.

5.7 The Interview Questions They Will Ask

  1. “What is stored in a BEAM file, and why does chunk presence vary?”
  2. “Why is calling module_info unsafe for an untrusted artifact auditor?”
  3. “What is the difference between byte reproducibility and semantic equivalence?”
  4. “How would you detect duplicate modules in a release?”
  5. “Which build inputs belong in provenance?”
  6. “How do you prevent normalization from hiding a meaningful change?”

5.8 Hints in Layers

Hint 1: Account before decode — Produce a raw file entry even if BEAM inspection later fails.

Hint 2: Ask for less — Extract only chunks required by enabled inventory and policy rules.

Hint 3: Preserve three identities — Raw file, per-chunk, and semantic digests answer different questions.

Hint 4: Make normalizers testify — Every normalized difference names a rule and displays before/after evidence.

5.9 Books That Will Help

Topic Book Precise reading
VM artifacts The BEAM Book — Erik Stenman “The BEAM File Format” and “Code Loading” chapters/sections
Erlang compilation Programming Erlang, 2nd ed. — Joe Armstrong Chapters on compiling and running programs and code loading
OTP releases Designing for Scalability with Erlang/OTP Release handling and deployment chapters
Supply-chain framing Building Secure and Reliable Systems — Google Chapters on software supply chain and verifiable artifacts

5.10 Implementation Phases

Phase 1: Safe Inventory (5-8 hours)

  • Walk and hash all release files deterministically.
  • Inspect basic BEAM identity and chunk lists without loading.
  • Add malformed-file and module-collision reporting.

Phase 2: Semantic Manifest (5-8 hours)

  • Extract exports, imports, attributes, compile and docs/debug summaries.
  • Add chunk digests, resource limits, and versioned adapters.
  • Canonicalize and hash the manifest.

Phase 3: Policy and Comparison (6-10 hours)

  • Add artifact policy rules and difference hierarchy.
  • Implement narrow visible normalization profiles.
  • Explain raw, chunk, and semantic differences.

Phase 4: Reproducibility Harness (6-8 hours)

  • Build fixtures from distinct roots and toolchain inputs.
  • Add tamper, cross-target, and over-normalization negative tests.
  • Document precise claims and CI exit statuses.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Inspection load module / :beam_lib :beam_lib only Keeps candidate data non-executable
Identity semantic digest only / raw+chunk+semantic Preserve all three Prevents normalized claims replacing integrity
Error handling skip malformed / manifest error entry Error entry and policy result Accounts for every artifact
Normalization implicit / versioned visible rules Versioned visible rules Makes equivalence auditable
Scope BEAM only / whole release with BEAM depth Whole release with BEAM depth Detects native and runtime asset drift

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Container Validate BEAM parsing normal, stripped, corrupt, truncated
Inventory Protect module/interface facts exports, imports, absent optional chunks
Safety Prove no loading and bounded paths on-load fixture, symlink escape, huge chunk
Policy Validate rule evidence debug info, source paths, duplicate module
Reproducibility Compare controlled builds different roots, lock, toolchain, target
Canonicalization Stabilize manifests randomized discovery order
Tamper Protect raw integrity post-build byte modification

6.2 Critical Test Cases

  1. An artifact with load-time behavior is inspected without entering the code server.
  2. A malformed BEAM receives a manifest error entry and nonzero policy result.
  3. Missing optional debug/docs chunks are classified according to policy, not parser failure.
  4. Duplicate module identities across two application paths are reported together.
  5. Random filesystem discovery order yields the same manifest digest.
  6. Builds from two roots expose path variance without hiding raw inequality.
  7. A changed code chunk remains unexplained under the path normalizer.
  8. A native library change fails whole-release raw comparison.
  9. Cross-architecture strict comparison refuses an invalid success claim.
  10. Resource limits stop an oversized decoded chunk while the scan completes accountably.

6.3 Test Data

Compile minimal modules with docs/debug information enabled and stripped, source roots varied, exports altered, and duplicate module names. Mutate copied files for corruption and tamper cases. Include a release with a small sample native asset so whole-tree comparison is tested.

6.4 Definition of Done

  • Every selected release file appears in the manifest, including failures.
  • Candidate modules are never loaded or executed.
  • BEAM identity, chunk presence, interfaces, and policy evidence are documented.
  • Duplicate modules and malformed containers are detected.
  • Raw, chunk, and semantic digests remain separate.
  • Manifests are canonical and deterministic under randomized traversal.
  • Every normalizer is versioned, visible, and has a negative test.
  • Comparison output never labels normalized equivalence as byte identity.
  • Provenance states declared source, lock, toolchain, target, and profile.
  • Resource, symlink, and incomplete-scan policies are enforced.

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Problem Why it happens Fix Quick test
Inspector executes candidate Uses code loading for metadata Use :beam_lib; assert module remains unloaded On-load marker fixture
Missing file disappears Parser error drops entry Create raw entry before decode Scan truncated BEAM
False reproducible claim Semantic digest replaces raw hash Report raw identity first Change ignored metadata byte
Meaningful drift hidden Normalizer drops whole compile info Narrow field-specific rule Change compiler option too
Duplicate module missed Index scoped only per app Build release-global module index Two apps define same module
Memory spike Decodes many large chunks concurrently Limit concurrency and decoded size Oversized debug fixture

7.2 Debugging Strategies

  • Use an explain command for one file/module instead of dumping full manifests.
  • Compare chunk sets before decoding semantic values.
  • Print the exact normalization rule and before/after value.
  • Check code server loaded modules before and after safety fixtures.
  • Recreate builds with one controlled input changed at a time.

7.3 Performance Traps

  • Reading every release file fully into memory to hash it
  • Decoding abstract/debug chunks when no enabled rule needs them
  • Unbounded Task concurrency over thousands of modules
  • Retaining full chunk binaries inside the final manifest
  • Recomputing global collision indexes per file

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add an ASCII table of applications, versions, and module counts.
  • Add a policy requiring no duplicate modules.
  • Export an interface-only manifest.

8.2 Intermediate Extensions

  • Compare application resource files with module inventory.
  • Add a policy for unexpected NIF/native library files.
  • Integrate manifest verification into release promotion.

8.3 Advanced Extensions

  • Sign manifest digests with an external trusted signing service.
  • Generate an SBOM mapping packages to release files and module identities.
  • Run inspection in an OS-level sandbox for hostile artifact intake.
  • Compare abstract-code structure under a carefully versioned profile.

9. Real-World Connections

9.1 Industry Applications

Teams need to answer which exact modules reached production, whether two deployments contain the same bytes, why a rebuild differs, whether debug/source metadata leaked, and whether a dependency collision changed code-path behavior. A BEAM-specific auditor converts opaque release directories into reviewable evidence.

  • Erlang/OTP beam_lib and code server
  • Mix releases and Erlang release tooling
  • Hex package checksums as upstream dependency evidence
  • SBOM and artifact-signing systems as downstream manifest consumers

9.3 Interview Relevance

This project demonstrates staff-level knowledge of compiler artifacts, code-loading risk, deterministic serialization, supply-chain evidence, and careful claims. Strong candidates clearly separate raw integrity from semantic interpretation.

10. Resources

10.1 Essential Official Reading

10.2 Books and Deeper Study

  • The BEAM Book by Erik Stenman — BEAM file format and code loading sections.
  • Programming Erlang, 2nd edition, by Joe Armstrong — compilation and code loading chapters.
  • Designing for Scalability with Erlang/OTP by Francesco Cesarini and Steve Vinoski — release handling and deployment sections.

10.3 Verification Checklist Before Sharing the Project

  • Prove inspection leaves candidate modules unloaded.
  • Show one normalized-equivalent but byte-different explanation.
  • Show one code-chunk difference that normalization cannot hide.
  • Account for native and non-BEAM release files.
  • Keep examples as manifest records, comparison transcripts, schemas, and pseudocode rather than full runnable code.