Validation — refs-core / refs-forensic¶
The honest tier: Tier-2 at best (structural), no ground-truth corpus¶
ReFS is undocumented. Microsoft publishes no on-disk specification. Every
structural fact this reader encodes is third-party reverse engineering
(libyal libfsrefs, Prade's academic
TSK fork + paper). There is no ground-truth forensic corpus for ReFS.
Consequently:
- Structural metadata (boot VBR, superblock, version, block headers) is Tier-2 at best — validated by self-minting a volume on the live Windows ReFS driver and cross-checking against the reverse-engineered references. We authored both the fixture and the expected answers, so they can share a blind spot with a wrong reference. Structural metadata cannot reach true Tier-1 on this filesystem — the only "oracle" (libfsrefs) is itself reverse-engineered (the LZNT1-trap risk: a wrong reader and a wrong reference can agree and ship green).
- File content can reach Tier-1 in a later phase, by hashing reconstructed
file bytes against
Get-FileHashon the live Windows driver (the only authoritative source of true file bytes). The mint already captured those SHA-256 hashes (seetests/data/README.md).
This is stated plainly, per the fleet Evidence-Based Rigor discipline: we do not claim Tier-1 for any structural finding.
Reproduce from a clean clone — no Tier-1 path exists¶
ReFS has no downloadable third-party corpus (Microsoft publishes no on-disk
spec; the only references — libfsrefs, Prade's TSK fork — are reverse-engineered
and cannot serve as a Tier-1 answer key). So, unlike the other filesystem repos,
there is no download URL that reconstitutes a Tier-1 validation here. The only
real-volume oracle is a Tier-2 self-mint on the live Windows ReFS driver, and
re-minting it produces a new, different Tier-2 volume — it can never reconstitute
a Tier-1 validation (a re-mint is self-authored by definition).
The two env-gated real-volume oracles are gitignored (16 MiB / 256 MiB slices)
because the source VHD was detached and lost, so they are not byte-for-byte
re-mintable; a fresh mint via the generators in tests/data/README.md yields a
different v3.14 volume the tests re-verify against, not the identical bytes.
| Oracle | env var | md5 | committed? | mint command | run command |
|---|---|---|---|---|---|
16 MiB partition head (refs_partition_head.bin) |
REFS_TIER2_ORACLE |
1a38a7ff099bcd1d58cce9f8e29c9db2 |
No (gitignored) | tests/data/README.md → "How the oracle was minted" (mint4.ps1) |
see below |
256 MiB container head (refs_container_head256.bin) |
REFS_TIER2_ORACLE256 |
8d09e81af8b939151fe0ae81d90b4623 |
No (gitignored) | tests/data/README.md → "P3 generator" (mint5.ps1 / slice256.ps1) |
see below |
Always-on (committed, no env var, no mint): the P0 boot/superblock path runs
against the committed refs_boot_superblock.bin and the small refs_v314_*.bin
metadata pages — cargo test -p refs-core and cargo test -p refs-forensic
exercise the whole structural + audit path with zero setup.
Run commands for the Tier-2 real-volume oracles (after minting a v3.14 volume
per tests/data/README.md and slicing the head):
# 16 MiB head: boot/superblock, Minstore object tree, directory reachability, clean sweep
REFS_TIER2_ORACLE=/abs/path/to/refs_partition_head.bin \
cargo test -p refs-core --test boot full_partition_head_env_gated
REFS_TIER2_ORACLE=/abs/path/to/refs_partition_head.bin \
cargo test -p refs-forensic --test integrity real_volume_clean_sweep_emits_nothing_false_env_gated
# 256 MiB head: container resolve, descent (no-fabrication), F-CARVE CoW residue
REFS_TIER2_ORACLE256=/abs/path/to/refs_container_head256.bin \
cargo test -p refs-core --test container \
real_volume_container_resolves_root_directory_page_env_gated
REFS_TIER2_ORACLE256=/abs/path/to/refs_container_head256.bin \
cargo test -p refs-forensic --test carve \
real_volume_stale_directory_page_carved_env_gated
P0 — boot VBR + superblock + version detection¶
Validated against a Tier-2 self-minted ReFS v3.14 volume (Windows 11 Pro,
build 26200, Dev Drive on a 60 GB dynamic VHD — the only ReFS path on Win11
client). Ground truth is fsutil fsinfo refsinfo on the live Windows driver.
Every boot-VBR offset was byte-verified against the fixture and reconciled
against the driver's fsutil output:
| Field | Offset | Reader value | fsutil / driver |
Match |
|---|---|---|---|---|
| FS signature | 3 (8) | "ReFS\0\0\0\0" |
— (identity) | ✅ |
| FSRS identifier | 16 (4) | "FSRS" |
— (identity) | ✅ |
| Number of sectors | 24 (8) | 125,698,048 | 0x077e0000 = 125,698,048 |
✅ |
| Bytes per sector | 32 (4) | 512 | Bytes Per Sector 512 |
✅ |
| Sectors per cluster | 36 (4) | 8 | (512 × 8 = 4096 = Bytes Per Cluster) |
✅ |
| Major version | 40 (1) | 3 | REFS Volume Version 3.14 |
✅ |
| Minor version | 41 (1) | 14 | REFS Volume Version 3.14 |
✅ |
| Volume serial | 56 (8) | 0x4e32fc4432fc3317 |
0x4e32fc4432fc3317 |
✅ |
| Container size | 64 (8) | 67,108,864 | — | (RE) |
| SUPB block | cluster 30 (0x1e000) |
signature SUPB, self-describing block # = 30 |
— (RE, libfsrefs) | (RE) |
(RE) = reverse-engineered structural fact with no independent driver field to
reconcile against — Tier-2 by nature.
The env-gated test (REFS_TIER2_ORACLE) re-runs the same parse against the full
16 MiB minted partition head, not just the committed 128 KiB slice.
Fail-loud gates (never silently misparse)¶
Per RESEARCH.md, the reader fails loud with the offending value rather than guessing:
- Bad FS signature / FSRS identifier → error carries the offending bytes.
- A v1 (or any non-v3) major version is rejected by
require_v3()naming the realmajor.minor— a v1 volume (Server 2012 / 8.1) has a materially different layout, so parsing it as v3 would silently misparse. - Bad
SUPBblock signature → error carries the offending bytes and the byte offset. - Truncated / hostile buffers never panic (bounds-checked LE readers, saturating
arithmetic,
#![forbid(unsafe_code)]).
refs-forensic — the audit layer (F-INTEGRITY + F-CARVE)¶
The analyzer emits graded forensicnomicon::report::Findings. Each finding is an
observation ("consistent with …"), never a verdict. What is validated, and
against what, stated plainly per the Evidence-Based Rigor discipline:
F-INTEGRITY — structural anomalies (the solid, validatable deliverable)¶
Validated on the real resident v3.14 metadata (a clean volume must emit
nothing false — the anti-LZNT1-trap regression guard, run over all 39 real MSB+
pages via REFS_TIER2_ORACLE) plus crafted corruption of the real bytes:
| Code | Detects | Validated on |
|---|---|---|
REFS-BOOT-SIGNATURE-INVALID |
boot VBR signature ≠ ReFS\0\0\0\0 (fail-loud value) |
real SUPB region + byte-flipped signature |
REFS-SELF-BLOCK-MISMATCH |
a metadata block whose self-recorded block # ≠ its location (relocated/tampered) | real SUPB (self-block 30) + a planted MSB+ page |
REFS-METADATA-CRC-MISMATCH |
a stored CRC that fails over a known coverage range | self-contained CRC-32C over an explicit span |
REFS-CHECKPOINT-DIVERGENCE |
the superblock names zero / torn checkpoint copies | real SUPB with a zeroed checkpoint-count field |
REFS-IMPOSSIBLE-GEOMETRY |
cluster/geometry beyond bounds (allocation-bomb guard) | real boot VBR with an absurd sectors-per-cluster |
REFS-ORPHANED-OR-UNRESOLVED |
a child reference that resolves to no resident page (reserved for a directory-walk caller — not emitted on a partial slice, where non-residence is normal) | vocabulary unit test |
CRC honesty (which codes are validated vs deliberately NOT fabricated). ReFS's
own whole-block checksum coverage range is undetermined in the
reverse-engineered references (libfsrefs marks it TODO; an empirical
brute-force over the real SUPB self-reference did not reproduce it — see
tests/data/README.md). So refs-forensic never
auto-fabricates a REFS-METADATA-CRC-MISMATCH on a clean block. The code is
emitted only via audit_crc_range(block, offset, start, end, stored) — a
caller-supplied, KNOWN coverage range (validated here with a self-contained CRC-32C
whose answer key is the crate's own crc32c, an independent Tier-1 algorithm
oracle). Automatic whole-block CRC verification stays absent until a future phase
pins ReFS's range. This is the LZNT1-trap avoidance the fleet standards mandate:
we ship the mechanism, not a guessed range that would flag every clean block.
F-CARVE — CoW metadata-residue recovery¶
ReFS is allocate-on-write: an updated metadata page is written NEW at a higher
block number and the tree re-points at it, leaving the OLD MSB+ page behind.
recover_residue surfaces MSB+ directory pages that are not the current version
(the highest self-block for their table id — the CoW-monotonic discriminator,
corroborated by the object table's resolvable 0x600 → 80_384 mapping) and
carves the directory-entry rows they still hold that the current version dropped.
Validated on real resident bytes (REFS_TIER2_ORACLE256): on the 256 MiB
v3.14 oracle, the current 0x600 root directory (self-block 80_384, cluster
14848) no longer carries the System Volume Information entry, but an older CoW
copy (self-block 70_656, cluster 5120) still does — F-CARVE surfaces that
stale page and its carved System Volume Information entry. A genuine, resident
CoW-residue recovery, not synthetic. Also validated on synthetic stale/current
page pairs.
Oracle-blocked (stated plainly, never fabricated). The minted user files
(dir_a / known1.txt / nested / big.bin) live in a non-resident band beyond
the 256 MiB slice, and the source VHD was detached and lost (see
tests/data/README.md P4 note), so their
deleted-recovery end-to-end is not reproducible. F-CARVE returns what IS
resident and the real-volume test asserts the non-resident files do not appear
— the carver never fabricates the oracle-blocked band.
Robustness (Paranoid Gatekeeper)¶
#![forbid(unsafe_code)]; every field read through bounds-checked, saturating LE
helpers; a lying block/row/count never panics or over-reads (malformed-input tests
sweep truncations and garbage across both audit_image and recover_residue).
100% gate-effective line coverage on both crates (uncovered lines are
defence-in-depth guards annotated // cov:unreachable).
Later phases (not yet built)¶
File-content extraction (data runs → the Tier-1 content-hash gate against
Get-FileHash on the live Windows driver), USN Change Journal parsing, and a
pinned ReFS CRC coverage range (turning audit_crc_range into an automatic
whole-block sweep) follow in subsequent, individually-validated increments — each
version-gated (a reader validated on v3.14 may break on v3.4/v3.9).