Validation¶
Status — most of this repo is still Tier 2. Tier 1 covers the FileVault unwrap chain (P7), encrypted-volume file reading (P8), and exactly those core read paths the third-party dfVFS corpus exercises (P9): directory listing, path resolution, inode numbers, file byte assembly, extended attributes, symlink targets and resource forks.
Everything else remains Tier 2, including parts of the very sections P9 touches: decmpfs compression (no compressed file exists in the dfVFS corpus), inode metadata — timestamps, mode, ownership, which dfVFS asserts only for presence, never by value — and all of snapshots (P5), the space manager and reaper (P6), fusion detection, and sealed volumes.
A per-phase heading below says Tier 2 where P9 raised only part of what that phase covers; the heading is the conservative claim and P9's table is the precise one. Read them together, and prefer the narrower.
How to read the evidence tiers¶
- Tier 1 — an independent third party authored both the artifact and the answer key, or the data is real-world.
- Tier 2 — real engine/tool output whose ground truth is derivable from the documented construction or confirmed by an independent oracle, but we chose the scenario (so it can miss real-world quirks).
- Tier 3 — we authored both the fixture and the expected answer with nothing independent vouching (maximal self-deception risk; labelled, never read as Tier 1).
Oracle independence and corpus tier are orthogonal. The tier of a claim is
min(oracle independence, corpus provenance): an independent oracle run against a
corpus we minted ourselves is Tier 2; only an independent oracle on real-world
data reaches Tier 1.
Independent oracles¶
| Oracle | Independence | Validates | Install |
|---|---|---|---|
macOS (hdiutil attach -readonly, diskutil apfs list, stat, xattr, ls -lR@) |
Apple's own driver | directory tree, file bytes (post-decmpfs), timestamps, xattrs, snapshots | present |
The Sleuth Kit fsstat/fls/istat (v4.12.1) |
separate C codebase | container/volume geometry, inode listing + metadata | installed |
fsapfsinfo (libfsapfs) |
independent | NXSB/APSB fields, volumes, btree/omap, snapshots | build (LGPL — oracle only) |
apfsck (apfsprogs) |
structural fsck | checksum/omap/btree/spaceman structural integrity | build |
| apfs-fuse | independent | decmpfs decode + encrypted-volume unwrap | build (GPL — oracle only) |
apfs / exhume_apfs crates |
independent Rust | spot field cross-check | crates.io |
Cross-extractor check: macOS cp of a decmpfs file vs apfs-core::extent::read_data
must be byte-identical (same SHA-256), so neither extractor's assumptions are
load-bearing alone.
Corpora (mintable on a macOS host)¶
| Corpus | Mint command | Tier |
|---|---|---|
| Plain APFS | hdiutil create -size 64m -fs APFS -volname APFSTEST -layout GPTSPUD apfstest.dmg |
2 |
| Snapshots | attach → tmutil localsnapshot / diskutil apfs → detach |
2 |
| decmpfs | ditto --hfsCompression src dst on the attached volume |
2 (macOS oracle) |
| Clones | cp -c (clonefile) on the attached volume |
2 |
| Encrypted | hdiutil create -encryption -stdinpass -fs APFS … |
2 |
| Sealed system volume | real macOS SSV image (env-gated, gitignored) | 1 |
| Real macOS images | env-gated, gitignored | 1 |
Verbatim mint commands are recorded in issen/docs/corpus-catalog.md and
tests/data/README.md. Carving/recovery is validated against an independent
oracle (real images / pre-delete capture + apfsck), not only records we deleted
ourselves.
Validated capabilities¶
Object map + B-tree navigation + volume-superblock resolution (P2) — Tier 2¶
Exercised, not validated, by P9. Reaching any dfVFS file requires this machinery to work, so P9 is indirect evidence it does. The specific claims below — checksum-before-trust, cycle guards, the fixture's own geometry — are asserted against a fixture we minted and remain Tier 2. Working is not the same as validated against an independent expectation.
Corpus: tests/data/apfs_container_chain.bin — blocks 0–344 (1.38 MiB) of a
real APFS container partition minted by Apple's own hdiutil
(hdiutil create -size 128m -fs APFS -volname APFSORACLE -layout GPTSPUD). The
slice holds the complete checkpoint ring and the full
NXSB → container omap → omap B-tree → volume superblock (APSB) chain. Because the
carve starts at NXSB block 0, a physical block address maps directly to
paddr * 4096 in the fixture. Provenance + MD5 in tests/data/README.md.
What is exercised (core/tests/{omap,btree,btree_descend,volume_resolve}.rs):
| Claim | Evidence | Oracle |
|---|---|---|
omap_phys_t header decode (om_flags/om_tree_type/om_tree_oid/om_snapshot_tree_oid) |
nx_omap_oid = 343 → om_tree_oid = 344, om_flags = OMAP_MANUALLY_MANAGED |
offsets verified verbatim vs the Apple reference + libfsapfs; raw-image decode |
btree_node_phys_t header + TOC iterate (fixed + variable KV) |
omap root @344: btn_flags = 0x7 (ROOT|LEAF|FIXED), 1 fixed entry omap_key(1026,2) → omap_val(paddr 342) |
Apple reference + libfsapfs spec (4-byte kvoff_t, 8-byte kvloc_t, btree_info footer) |
| Root→leaf descent with cksum + cycle guards | real omap walk yields exactly (1026, 2, 342); a corrupted node fails Fletcher-64 (ChecksumMismatch); a self-referential index node trips CycleGuard |
construction; checksum-before-trust |
Virtual nx_fs_oid → physical APSB paddr |
volume_superblock_addrs() == [342]; block 342 carries APSB magic 0x42535041 + valid Fletcher-64 |
Apple diskutil apfs list and libfsapfs fsapfsinfo (run on the committed fixture) each report exactly one volume (APFSORACLE) — matching the single resolved APSB |
Independence: fsapfsinfo (libfsapfs, a separate C codebase) reads the exact
committed apfs_container_chain.bin and reports Number of volumes: 1, volume
identifier fa8b74aa-…, name APFSORACLE — matching Apple diskutil. The
B-tree TOC and value offsets were verified verbatim against the Apple APFS
Reference and the libfsapfs format spec before implementation, not from memory.
Tier rationale: independent oracle (libfsapfs/diskutil) on a self-minted
corpus ⇒ Tier 2 (min(independent, self-minted)). A Tier-1 lift needs the same
resolution run on a real-world / third-party image (env-gated, future work).
Fusion container detection + fail-loud (P1/P2 robustness) — Tier 2¶
A Fusion (SSD+HDD) container resolves physical addresses tier-aware (a tier bit in the high address, via the fusion middle tree). This reader does not yet implement that translation (full support is P8), so reading a Fusion image without it would silently mis-address blocks — the worst failure class. Per the design (risk #6), detection + loud rejection lands early.
ApfsContainer::open checks the NXSB nx_incompatible_features NX_INCOMPAT_FUSION
bit (offset 64, value 0x100, Apple APFS Reference) on the bootstrap superblock
— before walking the checkpoint ring or resolving any address — and returns
ApfsError::UnsupportedFusion rather than proceed.
What is exercised (core/tests/container_open.rs):
| Claim | Evidence | Oracle |
|---|---|---|
| Fusion container is rejected at open | the real apfs_nxsb_head.bin with NX_INCOMPAT_FUSION set (+ re-stamped Fletcher-64) opens to UnsupportedFusion, never a mis-read |
construction (documented spec bit) on a real container |
| non-Fusion container is unaffected | the unmodified real container still opens and resolves its geometry | real Apple-minted fixture (Tier 2) |
Volume superblock + fs-record dispatch + inode metadata + name→inode (P3) — Tier 2¶
Partially raised by P9.
name→inoderesolution and inode numbers are Tier 1 against the dfVFS corpus. Inode metadata — timestamps, mode, ownership — is not, and dfVFS cannot close it: its timestamp tests are nineassertIsNotNonepresence checks with no pinned values anywhere in the file. Asserting against them would validate that a field is populated, not that it decodes correctly.(Recorded because the method names
testAccessTime/testModificationTimeread like value oracles and are not. A test name is a claim about a test, not about the world — read the body.)Closing this needs a corpus whose timestamps are documented independently: macOS
statoutput captured alongside a minted image raises the oracle but not the corpus, so it reaches Tier 2 only.
Corpus: tests/data/apfs_fstree.bin — blocks 0–373 (1.46 MiB) of a real APFS
container partition minted by Apple's own hdiutil
(hdiutil create -size 128m -fs APFS -volname APFSP3 -layout GPTSPUD), populated
with a known directory tree before carving. The slice reaches the full chain
NXSB → checkpoint → container omap → APSB (block 371) → volume omap (366/367) →
the virtual file-system tree leaf node (block 365). Provenance + the known-tree
table + MD5 in tests/data/README.md.
What is exercised (core/tests/{volume,fsrecord,inode,dir}.rs):
| Claim | Evidence | Oracle |
|---|---|---|
apfs_superblock_t (APSB) decode |
block 371: magic APSB, apfs_fs_index = 0, apfs_omap_oid = 366, apfs_root_tree_oid = 1028 (virtual), volname APFSP3 |
offsets verbatim vs libfsapfs fsapfs_volume_superblock; TSK pstat reports APSB block 371, oid 1026, xid 6, volume APFSP3 |
j_key dispatch + xf_blob TLV walk |
top-4-bit type / low-60-bit oid split; xfields decode NAME + DSTREAM with 8-byte value alignment | Apple reference + libfsapfs format spec |
j_inode_val_t metadata |
Beth.txt (inode 20): parent 18, size 38, mode 0644, uid/gid 99, create 1782060082608648902, mod …686902, access …733745215; root (2): mode 0755, uid 501, 3 children |
TSK istat -o 40 -B 371 — every field (size, mode, uid/gid, child/link count, all four ns-timestamps) matches per inode |
| name→inode path navigation | open_path resolves /top.txt→22 (15 B), /Dir1/Beth.txt→20 (38 B), /Dir1/Sub/secret.bin→21 (26 B); /→root(2); // and trailing / normalize; missing components error loudly |
TSK fls -r -o 40 -B 371 lists the identical tree + inode numbers; macOS stat confirmed sizes pre-detach |
Inode-offset correction (vs the design doc). The design doc placed the inode
timestamps/mode following the libfsapfs asciidoc table (access@48, flags@56,
nchildren@64, mode@86, xfields@98). The real on-disk j_inode_val_t — verified
empirically and reconciled against TSK istat (timestamps + mode + uid for inode
20/2) — has these fields 8 bytes earlier: access@40, internal_flags@48,
nchildren/nlink@56, bsd_flags@68, owner@72, gid@76, mode@80, xfields@92. The
asciidoc table inserts a phantom 8-byte gap; the empirical layout matches Apple's
struct and the istat oracle exactly. inode.rs uses the corrected offsets.
fs-tree is virtual. The fs-tree's node oids resolve through the volume
object map (ObjectMap::resolve) at the volume xid; dir::for_each_fs_record
walks it with checksum-before-trust, a visited-set cycle guard, and a depth cap.
The fixture's fs-tree is a single leaf, so the index-node descent and cycle guard
are exercised on a synthetic two-level virtual tree (valid Fletcher-64), mirroring
the P2 btree_descend.rs pattern — every functional assertion uses the real
fixture + TSK oracle.
Tier rationale: independent oracle (TSK fls/istat, a separate C codebase)
on a self-minted corpus ⇒ Tier 2. A Tier-1 lift needs the same navigation run
on a real-world macOS image (env-gated, future work).
File byte read + decmpfs + xattr + symlink (P4) — Tier 2¶
Partially raised by P9. File byte assembly, extended attributes, symlink targets and resource forks are Tier 1 against the dfVFS corpus. decmpfs is not — that corpus contains no compressed file, so every decmpfs claim still rests on a fixture we minted. A third-party image with an LZVN/LZFSE-compressed file would close it.
Corpus: tests/data/apfs_content.bin — the first 442 blocks (1.73 MiB,
4096-byte blocks) of a real APFS container partition minted by Apple's own
hdiutil (hdiutil create -size 128m -fs APFS -volname APFSP4 -layout GPTSPUD),
populated with known content covering every read path: a plain file, a
sparse file (a 64 KiB hole + a tail extent), a transparently-compressed
file (ditto --hfsCompression → decmpfs type 8, LZVN, resource fork), a file
with custom xattrs, and a symlink. The carve reaches the full chain
NXSB → checkpoint → container omap → live APSB (block 438, xid 14) → volume omap
(433/434) → fs-tree leaf (block 432) and every file's data extents +
resource fork (blocks 345–426). Provenance + MD5 in tests/data/README.md.
What is exercised (core/tests/{extent,compression,xattr}.rs):
| Claim | Evidence | Oracle |
|---|---|---|
| plain-file extent assembly | /plain.txt (35 B, single extent phys 347) reads back byte-identical |
macOS shasum -a 256 = 289af0a0… (matches read_data) |
| sparse-hole handling | /sparse.bin (69632 B): FILE_EXTENT records are a 64 KiB hole (phys_block_num 0) + a 4 KiB tail @65536; the hole reads back as zeroes |
macOS cp SHA-256 = fe0fc4fa…; list_extents surfaces the raw hole+tail |
| nested-file read | /Dir1/Beth.txt (33 B, phys 400) |
macOS SHA-256 = ee7c2682… |
| decmpfs type-8 LZVN (resource fork) | /compressed.txt (180000 B): com.apple.decmpfs header (type 8) + a com.apple.ResourceFork stream xattr (dstream 24, 1526 B, three LZVN chunks); decoded transparently by read_data |
macOS cp SHA-256 = 3f58a418… — identical to our decoded output (the cross-extractor check) |
| decmpfs other types | inline 1/9, zlib 3 (+0xFF-stored), LZVN 7 (real chunk + 0x06-stored), LZFSE 11, resource-fork zlib 4 / LZFSE 12 / uncompressed 10 | forensicnomicon::decmpfs::classify + flate2/lzvn/lzfse_rust; round-trip / real-chunk |
| decmpfs refusals (fail-loud) | bad magic, truncated, unknown type, dedup type 5, LZBitmap 13/14, missing fork, length mismatch all return a named ApfsError::Decmpfs — never fabricated bytes |
construction |
| extended attributes | /plain.txt carries com.example.tag="forensic-marker-P4", user.note="second custom attr" (both embedded) |
macOS xattr -l — names + values match list_xattrs |
| symlink target | /symlink_to_beth → com.apple.fs.symlink (embedded) = "Dir1/Beth.txt" |
macOS readlink = Dir1/Beth.txt |
SHA-256 oracle reconciliation (the byte-identical cross-extractor check). Each
file's content, as assembled by apfs_core::extent::read_data, hashes identically
to the macOS-read bytes captured before detach:
| File | Bytes | apfs_core::read_data SHA-256 |
macOS shasum -a 256 |
Match |
|---|---|---|---|---|
/plain.txt |
35 | 289af0a0…abf86b |
289af0a0…abf86b |
✅ |
/sparse.bin |
69632 | fe0fc4fa…cc822a |
fe0fc4fa…cc822a |
✅ |
/Dir1/Beth.txt |
33 | ee7c2682…cfeb96 |
ee7c2682…cfeb96 |
✅ |
/compressed.txt (decompressed) |
180000 | 3f58a418…3abc78 |
3f58a418…3abc78 |
✅ |
decmpfs codec reuse (not reinvented). core/src/compression.rs is thin glue
over the fleet's validated codec stack — forensicnomicon::decmpfs::classify for
the type→algorithm/storage map, flate2 (zlib 3/4), our length-tolerant
lzvn (lzvn-core, types 7/8 — the decoder hfsplus-forensic validated 25/25 on
real macOS Tahoe data, tolerant of the trailing bytes that lzfse_rust's strict
path rejects), and lzfse_rust (types 11/12). The only APFS-specific logic is
locating the payload (inline xattr vs com.apple.ResourceFork stream).
Symlink-target source (verified, not assumed). Empirically the symlink target
is the embedded com.apple.fs.symlink xattr value (flags 0x6 =
EMBEDDED | 0x4, value "Dir1/Beth.txt\0"), confirmed against the raw fixture
and macOS readlink. (APFS does also store it as a data stream; the xattr form is
what ditto/ln -s produced here and what the reader resolves.)
Tier rationale: independent oracle (macOS cp/xattr/readlink, Apple's own
driver, plus the documented construction) on a self-minted corpus ⇒ Tier 2
(min(independent, self-minted)). The decmpfs decoder additionally inherits
hfsplus-forensic's real-macOS validation of the same codec stack. A Tier-1 lift
needs the same reads run on a real-world macOS image (env-gated, future work).
Snapshots + point-in-time volume view (P5) — Tier 2 (+ env-gated populated path)¶
What P5 reads. A volume's snapshots live in a single B-tree located by the
APSB's physical apfs_snap_meta_tree_oid block number (offset 152;
libfsapfs names it snapshot_metadata_tree_block_number, o_subtype 0x10
SNAPMETATREE). The tree is variable-KV and holds two record kinds dispatched by
the j_key top-4-bit type: metadata (SNAP_METADATA 1, keyed by the
snapshot xid, value j_snap_metadata_val_t) and name (SNAP_NAME 11,
keyed by name, value j_snap_name_val_t { snap_xid }). mount_snapshot reads a
snapshot's sblock_oid (a physical APSB block) and grafts the live volume's
object map onto the frozen superblock: a snapshot's own apfs_omap_oid is 0,
so its fs-tree is read through the live omap, resolving oids at the snapshot's
xid (ok_xid ≤ snapshot_xid). The existing P3/P4 navigation then reads the
volume exactly as it stood at snapshot time.
Constant verification (before coding). The j_snap_metadata_val_t field
offsets were taken verbatim from libfsapfs fsapfs_snapshot_metadata_btree_value
and independently cross-confirmed against dissect.apfs j_snap_metadata_val;
both agree exactly: extentref_tree_oid@0, sblock_oid@8, create_time@16,
change_time@24, inum@32, extentref_tree_type@40, flags@44, name_len@48, name@50.
A RED unit test caught an early off-by-2 (name placed at @48 instead of @50,
mis-reading flags@44 as ending at @46) — the offsets above are what snapshot.rs
implements and what the unit test now asserts.
What is exercised (core/tests/snapshot.rs + core/src/snapshot.rs units):
| Claim | Evidence | Oracle / tier |
|---|---|---|
| snap-meta tree located + walked on real Apple bytes | the P4 fixture's APSB (block 438) carries apfs_snap_meta_tree_oid = block 340, a real o_subtype 0x10 btree node whose btn_nkeys is 0; the walk reads it, verifies its Fletcher-64 checksum, and returns zero snapshots |
the documented empty tree + Apple-authored checksum (Tier 2, real structure) |
j_snap_metadata_val_t decode |
every field decoded at the verified offsets | libfsapfs + dissect.apfs agreement (constant verification) |
j_snap_name_key_t decode + name→xid resolve |
resolve_snapshot_xid returns None on the empty real tree; returns the right xid on a populated vector |
construction + the libfsapfs snap_xid value layout |
| point-in-time seam | mount_snapshot reads sblock_oid as an ApfsVolume and grafts the live omap (a snapshot's own apfs_omap_oid is 0); the frozen root_tree_oid/xid are preserved. A unit test pins the graft (omap_oid overridden to the live volume's); mounting a non-APSB block (NXSB @0) fails loudly with UnexpectedObjectType |
real P4 APSB + omap_oid==0 contract (Tier 2) |
| walk control flow (leaf dispatch, index-node virtual child descent through the volume omap, checksum-mismatch fail-loud, visited-set cycle guard) | spec-faithful hand-built APFS micro-images: real obj_phys headers, real Fletcher-64, real variable-KV TOC/key/value + fixed-KV omap layout |
Tier 3 (walk control flow only; every offset/decode it relies on is Tier-2-validated on real data above) |
Populated changing-file path (env-gated) — VALIDATED on real snapshots. The
v1-vs-v2 point-in-time read (read changing.txt at the earliest snapshot →
v1 SHA-256; at the live volume → v2 SHA-256; assert each matches what macOS
wrote, and that they differ) ran from
core/tests/snapshot.rs::populated_fixture_point_in_time_read against a real
Apple-minted fixture and passed — byte-identical to the macOS oracle
(v1 da27342b…, v2 cfd0476c…; snapshot XIDs 6094/6096). This is the strongest
P5 evidence: an independent oracle (macOS itself) on real snapshots, and it
caught a real bug the synthetic seam test missed — the committed mount_snapshot
did not graft the live omap, so point-in-time open_path read block 0 (the NXSB)
and failed. The fixture is a whole GPT disk image (a Tart VM disk.img); the
test opens it through a partition-offset view and derives the live APSB from
the container omap. Gated on APFS_P5_FIXTURE (+ APFS_P5_PART_OFFSET,
APFS_P5_FILE, APFS_P5_V1_SHA256, APFS_P5_V2_SHA256); skips cleanly when
absent. See tests/data/README.md for the mint recipe and recorded values.
Performance (keyed descent, two layers). Navigation was quadratic on real volumes; two keyed-descent changes fixed it, each measured on the real P5 fixture:
- omap (
btree::find_leaf):omap.resolvereads one root→leaf path instead of scanning the whole omap B-tree on every node resolution. Cut the point-in-time validation run from ~4962 s (~83 min) to ~5 s.- fs-tree (
dir::for_each_fs_record_for_oid): the fs-tree walk now prunes to the target object id — at each index node it descends only children whose key range can cover that oid (child_may_contain_oid), instead of visiting every node and filtering. Every navigation entry point (lookup_child,load_inode,list_dir,list_extents,list_xattrs) keys by one object id, so all benefit.Measured (env-gated
keyed_nav.rs::keyed_navigation_prunes_real_fs_tree, real macOS multi-level fs-tree): resolving + reading/Users/admin/changing.txttouched 25 955 distinct blocks before → 26 after the fs-tree pruning, same byte-identical bytes (the macOSAPFS_P5_V2_SHA256oracle). With both layers the point-in-time test runs in ~0.01 s (from ~83 min). The committed fixtures are single-leaf, so the index-pruning branch is validated on the real multi-level tree (above) plus a CI-visible cross-check that the keyed walk returns the same record set as the full unpruned walk onapfs_fstree.bin(dir::tests::keyed_walk_matches_full_walk_on_real_fs_tree) and a boundary-case unit test of the prune predicate (child_pruning_selects_only_covering_subtrees).Minting blocker (host capability, documented). Creating an APFS snapshot on an arbitrary (DMG) volume requires the
com.apple.developer.vfs.snapshotentitlement: under SIP,fs_snapshot_create(2)returnsEPERMeven as root,diskutil apfshas noaddSnapshotverb, andtmutil localsnapshotonly snapshots the Time-Machine-eligible system data volume (never the DMG). The only entitled creator on a stock host istmutil, which cannot target a DMG. So the populatedchanging.txtfixture is minted on a host where snapshot creation is permitted (SIP-relaxed dev box, or a TM-registered volume); the exact mint commands are recorded intests/data/README.mdandissen/docs/corpus-catalog.md. The location, empty-case, decode, point-in-time seam, and walk control flow are all validated now on real Apple bytes + spec-faithful vectors per the table above.
Tier rationale: independent structure (Apple's own snap-meta tree + Fletcher-64)
on a self-minted corpus ⇒ Tier 2 for the location/empty/seam claims; the populated
v1-vs-v2 path lifts to Tier 2 with an independent oracle (diskutil apfs
listSnapshots + macOS-written SHA-256) once the env-gated fixture is supplied.
Space manager (allocation bitmap) + reaper (P6) — Tier 2¶
What P6 reads. The space manager (spaceman_phys_t, ephemeral
nx_spaceman_oid, resolved via the checkpoint map) answers "is this physical
block free?": block → chunk → chunk-info (CIB) → SPACEMAN_BITMAP. The reaper
(nx_reaper_phys_t, ephemeral nx_reaper_oid) surfaces objects logically
deleted but still physically present (pending_objects).
Bit polarity (the inverted-bit trap). A set bit = allocated, clear =
free (apfsck count_chunk_free: free = total − popcount(bitmap));
ci_bitmap_addr == 0 ⇒ whole chunk free. Offsets/polarity were taken before
coding from linux-apfs apfs_raw.h + apfsprogs spaceman.c, cross-checked
against libfsapfs (which leaves the chunk fields Unknown — so polarity rests on
apfsprogs' runtime-self-validated code, flagged below).
What is exercised (core/tests/{spaceman,reaper,container_open}.rs + reaper.rs units):
| Claim | Evidence | Oracle / tier |
|---|---|---|
is_block_free correct on real data |
on apfs_container_chain.bin every block 0..=344 (live objects: NXSB@0, spaceman@11, reaper@12, CIB@331, bitmap@332, omap@343, APSB@342) reports allocated; every block ≥ 345 reports free |
the carve boundary (Tier 2) |
| bit polarity verified by an independent count | counting allocations via is_block_free over all 32 758 blocks yields 345 == block_count − free_count, the spaceman's own accounting (independent of how each bit is read — a flipped polarity changes this total) |
spaceman free_count (Tier 2) |
| CAB tier fails loud | sm_cab_count > 0 (multi-TB indirection, not yet implemented) returns UnsupportedSpacemanCab, never a mis-resolved chunk |
construction |
| reaper empty on a clean container | the fixture's live reaper (block 12) has nr_head == 0, nr_oid == 0 ⇒ pending_objects returns no entries |
real fixture (Tier 2) |
| reaper walk + in-progress object | synthetic nx_reaper_phys_t → nx_reap_list_phys_t (two entries, chained via a checkpoint mapping) yields both (oid, type); an in-progress nr_oid is reported with no lists |
spec-faithful hand-built blocks (Tier 3; the offsets they use are Tier-2-validated above) |
| ephemeral resolution | ApfsContainer::spaceman_paddr()/reaper_paddr() resolve nx_spaceman_oid/nx_reaper_oid to blocks 11/12 via the live checkpoint map |
real fixture (Tier 2) |
Flag (honest tiering). The 1=allocated polarity is confirmed by one decoder
family (apfsprogs/linux-apfs) whose code self-checks against ci_free_count on
every real image, not by two independently-reasoned decoders — libfsapfs does not
decode these fields. The independent free-count cross-check above (345 from the
bitmap == 345 from the spaceman accounting) is what raises this to Tier 2 on our
real fixture. The populated reap-list walk is Tier 3 (no committed fixture has a
queued reaper; a real deleted-volume image would lift it).
FileVault unwrap chain: password → KEK → VEK (P7) — Tier 1 (chain only)¶
What P7 does. Recovers a volume encryption key from an APFS-native encrypted
volume given its password: container keybag (AES-128-XTS, keyed on the container
UUID, tweaked by absolute sector) → per-volume records → volume keybag (same
scheme, keyed on the volume UUID) → wrapped-KEK object (BER TLV: 0x83
wrapped key, 0x84 iterations, 0x85 salt) → PBKDF2-HMAC-SHA256 → AES key
unwrap (RFC 3394) for the KEK, then again for the VEK.
Why this reaches Tier 1 where the fixture alone does not. The committed
fixture is Tier 2: real Apple-written bytes, but we chose the scenario and set
the password, so a test against it grades our own homework. The Tier-1 claim
rests on a different artifact entirely — an examiner's real-world encrypted
external disk, minted by macOS on hardware we never touched, carrying a
password never disclosed to the implementation's author, acquired independently
with ddrescue. Identifiers are case material and are recorded outside this
repository.
The oracle is cryptographic, not a comparison. Nothing here is checked by
diffing against our own expectation. RFC 3394 key unwrap carries a 64-bit
integrity check value and fails closed, and the chain must pass it twice —
password→KEK, then KEK→VEK. A wrong salt, iteration count, TLV parse, XTS tweak,
or UUID keying yields bytes that cannot satisfy it. The check was written into
the artifact by Apple and is implemented by aes-kw, so the confirming authority
is independent of this codebase in both directions.
| Claim | Evidence | Oracle / tier |
|---|---|---|
| the chain recovers a volume key from a real-world encrypted disk | the examiner's own password produced UNLOCKED, 32-byte key, on a real acquired image |
RFC 3394 ICV, twice (Tier 1) |
| a wrong password is refused, never answered with bytes | same image, deliberately wrong password: REFUSED at 1/1 volumes, structures intact |
RFC 3394 ICV (Tier 1) |
| the same chain works on committed bytes in CI | fixture unlocks with its recorded password; wrong password refused | fixture (Tier 2) |
| a non-APFS input is rejected loudly, not answered | a file of zeros exits 2 with "no NXSB container superblock" | construction |
| the chain is usable on real evidence at all | 12,288 bytes read (one superblock, two keybag areas) from a 2 TB image | measured (Tier 1) |
| VEK value stability across refactors | pinned SHA-256 of the fixture's VEK | regression pin (Tier 3 — see flag) |
Superseded: an earlier revision of this section flagged that Tier 1 covered the unwrap chain only, because AES-XTS decryption of volume blocks was not implemented. It now is, and is validated at Tier 1 in P8 below.
The pinned VEK fingerprint is explicitly Tier 3: it pins the current implementation's output and proves the answer stopped changing, never that it was right. Its value was demonstrated rather than assumed — a mutation returning a constant key left the two-path equality assertion green, and only the pin caught it. The Tier-1 real-world unlock is what carries the correctness claim; the pin guards against silent drift.
Encrypted volume file reading: tree nodes + extents (P8) — Tier 1¶
What P8 does. Decrypts the two datasets APFS protects with the VEK: B-tree
nodes (tweak = the node's own ov_paddr, applied only when the omap entry
carries OMAP_VAL_ENCRYPTED) and file extents (tweak = crypto_id + block
index, stored in j_file_extent_val_t because APFS may relocate an extent
without re-encrypting it). AES-128-XTS over 512-byte sectors, VEK split as data
key || tweak key. Nodes are decrypted before the Fletcher-64 check, since
that checksum covers plaintext.
The corpus is not ours, and that is the whole point. Every self-minted
fixture is capped at Tier 2 by construction: a marker we wrote, on a volume we
encrypted, with a password we chose, is ground truth we authored. The Tier-1
claim rests on dfvfs-apfs-encrypted.dmg — third party artifact, third party
password (apfs-TEST), third party expected tree, inode numbers and file
contents. See tests/data/filevault/README.md for the full provenance table.
| Claim | Evidence | Oracle / tier |
|---|---|---|
| directory tree reads correctly off an encrypted volume | root lists .fseventsd, a_directory, a_link, passwords.txt |
dfVFS expected_sub_file_entry_names (Tier 1) |
| the right inode is resolved through the encrypted tree | /a_directory/another_file → inode 21 |
dfVFS APFSFileEntryTestEncrypted (Tier 1) |
| file contents decrypt byte-exactly | bytes equal This is another file.\n |
dfVFS generator + their committed test_data/another_file (Tier 1) |
| a second file decrypts, so one lucky extent cannot carry the claim | passwords.txt begins place,user,password |
the same generator heredoc (Tier 1) |
| the image is genuinely encrypted | no answer-key filename appears in the raw bytes | construction |
| a wrong password is refused on the third-party image | apfs-WRONG fails AES-KW |
RFC 3394 ICV (Tier 1) |
| the self-minted fixture also reads back | its marker decrypts byte-for-byte | our own ground truth (Tier 2) |
Controls, reported as they ran. Corrupting the node tweak turns the Tier-1
test red; corrupting the extent tweak turns it red only after the assertion
was changed from size to bytes — pinning len() == 22 let a corrupted extent
tweak pass, because decryption cannot change a file's length. That near-miss is
the reason the byte-exact form is the one that ships.
the_same_file_is_unreadable_without_the_key is not claimed as a passing
control: two mutations (a bogus key, and disabling the checksum guard) both left
it green, because refusing is the natural outcome of nearly any breakage here.
It stands as a regression guard against a future change that returns data for a
locked volume, not as evidence that one cannot.
Still open. Not validated against a volume paused mid-conversion, and not against a volume converted from CoreStorage — apfs-fuse notes that block id and XTS id diverge on exactly those, so it is a known unknown rather than an assumption of correctness.
Core reader against a third-party corpus (P9) — Tier 1¶
What changed. P2–P6 were validated only against fixtures this project
minted. That caps them at Tier 2 by construction: the decoder and the answer key
share an author, so they share his blind spots. dfVFS's apfs.raw supplies both
an artifact and an answer key we did not write, which lifts the core read paths
without altering a line of decoder.
| Claim | Evidence | Oracle / tier |
|---|---|---|
| directory listing | root = .fseventsd, a_directory, a_link, passwords.txt |
dfVFS expected_sub_file_entry_names (Tier 1) |
| nested listing | a_directory holds exactly a_file, a_resourcefork, another_file |
dfVFS number_of_sub_file_entries + generator (Tier 1) |
| path → inode | a_directory 16, a_file 17, another_file 19, a_link 20 |
dfVFS APFSFileEntryTest._IDENTIFIER_* (Tier 1) |
| file byte assembly | two files match their generator heredocs byte-for-byte | dfVFS generator (Tier 1) |
| extended attributes | myxattr = My extended attribute |
dfVFS asserts name and value (Tier 1) |
| symlink targets | a_link → a_directory/another_file |
the generator's ln -s (Tier 1) |
| resource forks | a_resourcefork fork = My resource fork\n |
the generator's ..namedfork/rsrc write (Tier 1) |
Controls. These validate existing behaviour, so there is no RED and fabricating one would be dishonest. They were instead shown capable of failing: corrupting extent assembly, truncating an xattr value, and dropping a directory entry each turn them red.
Corpus asymmetry (do not assume parity). dfVFS's two APFS images are built
differently: the plaintext one carries a resource fork, a myxattr extended
attribute, and a symlink to another_file; the encrypted one has none of the
first two and links to a_file, with different inode numbers throughout.
Extended attributes and resource forks are therefore Tier 1 on plaintext
only. Symlink targets are validated on both, which matters because APFS stores
them in an embedded xattr — so that path is exercised through decryption as
well.
Scope. Tier 1 here covers the read paths the dfVFS corpus exercises. Snapshots (P5), the space manager (P6) and sealed volumes are untouched by it and remain at their existing tiers — a third-party corpus raises what it actually contains, not the repository average.