ReFS Forensic Reader — Research-First Report (refs-core + refs-forensic)¶
Read-only Research-First deliverable.
Overriding caveat (threaded throughout): ReFS is undocumented — Microsoft publishes no on-disk specification. Every structural fact below is from third-party reverse engineering → Tier-2 at best. ReFS is also version-fragmented: the on-disk format changed materially between v1.x (Server 2012/8.1) and v3.x (Server 2016+/Win10+/Win11), and each Windows release tweaks it further. A reader validated against one build can silently misparse another. Windows auto-upgrades a volume to the newest format version on mount (Nordvik et al.) → the newest v3.x is what matters forensically. This version fragmentation — not algorithmic depth — is the single hardest thing.
1. Authoritative sources (reverse-engineered — NO official Microsoft spec)¶
De-facto spec (single most important reference):
- libfsrefs (Joachim Metz / libyal) — https://github.com/libyal/libfsrefs. C
library + fsrefsinfo, LGPLv3+, experimental, read-only.
- Its format doc: documentation/Resilient File System (ReFS).asciidoc
(https://github.com/libyal/libfsrefs/blob/main/documentation/Resilient%20File%20System%20(ReFS).asciidoc)
— the closest thing to a spec; honest about its own incompleteness (many fields
TODO/Unknown; the C library is v1-only, v2/v3 unsupported in code — though the
documentation + a contributor's PR go further, see §2).
Academic / peer-reviewed (strongest secondary — deeper than libfsrefs on v3.x + CoW): - Nordvik, Georges, Toolan, Axelsson — "Reverse engineering of ReFS" (Digital Investigation, 2019) — https://www.sciencedirect.com/science/article/pii/S1742287619301252. Documents v1.2 (Server 2012) + v3.2; block-sharing (copy → shared content blocks; edit → new runs, unchanged blocks shared) — relevant to carving. - Prade, Groß, Dewald — "Forensic Analysis of the Resilient File System (ReFS) Version 3.4" (FSI: Digital Investigation, 2020) — https://www.sciencedirect.com/science/article/pii/S266628172030010X / DFRWS PDF. The deepest v3.x source — reverse-engineers v3.4, introduces virtual addresses, documents CoW, proposes deleted-file/old-version recovery, extends TSK, implements a page carver. Source: Paul Prade's TSK fork (https://faui1-gitlab.cs.fau.de/paul.prade/refs-sleuthkit-implementation). - "Forensic analysis of ReFS journaling" (DFRWS APAC 2021) — ReFS Logfile + Change Journal; Log Record + USN_RECORD_V3 (distinct from NTFS).
Unverified — dropped: a "Williballenthin / FireEye-Mandiant ReFS" reverse- engineering piece could not be confirmed; the load-bearing secondary literature is Nordvik + Prade, not a Mandiant blog. (Paragon Software's ReFS driver is closed-source commercial — behavioral reference only.)
What IS known of the structures (from the libfsrefs asciidoc):
- Boot / FS-recognition at offset 0: 3-byte jump, signature "ReFS\x00\x00\x00\x00"
at offset 3, "FSRS" at offset 16, then sector count, sector size,
sectors-per-cluster, Major/Minor format version at offsets 40/41 (where you read
the version), volume serial (56), container/band size (64).
- Page-based metadata. Block signatures by level: "SUPB" superblock (L0),
"CHKP" checkpoint (L1), "MSB+" Ministore B+-tree (L2+). Block/page typically
16 KB (confirm per volume via cluster fields / fsutil).
- Minstore B+-tree — the core structure (ReFS is a key-value B+-tree store).
Everything (dirs, object table, volume info) is a Minstore tree of key→value records.
Node header + tree header (36 bytes) + records.
- Metadata block header differs by version: a format-v1 header is 48 bytes
starting with a volume-relative block number; a format-v3 metadata block reference
is 48 bytes carrying four block numbers (redundancy) + a checksum descriptor
(type/data offset/data size). This four-copy + checksum-descriptor layout is a
concrete v1↔v3 divergence.
- Object table indirection — maps logical object IDs → metadata block locations
(Prade's "virtual addresses" in v3.x add another indirection: resolve virtual →
physical before reading). Walked before file records resolve.
- Directory object = a Minstore B+-tree of directory records; entry types 0 =
FS-metadata file, 1 = File, 2 = Directory. Attributes reuse NTFS-like type codes
(0x10 $STANDARD_INFORMATION, 0x30 $FILENAME, 0x80 $DATA, 0xa0 $INDEX_ALLOCATION,
0xc0 $REPARSE_POINT, 0xe0 $EA). Data runs map file data to metadata-block
ranges (logical offset + run size in blocks). Filenames are UCS-2 + unpaired
surrogates (not strict UTF-16 — a robustness quirk the spec calls out).
- Nature: integrity-streams (checksummed metadata, optional file-data integrity),
allocate-on-write / copy-on-write — metadata never overwritten in place; a new
page is allocated and the tree re-pointed → stale metadata pages recoverable (the
basis of deleted-file/old-version carving).
Version fragmentation (the biggest problem), per the libfsrefs version table:
| Version | Windows release |
|---|---|
| 1.2 | Server 2012 / 8.1 (legacy) |
| 3.0 | Server 2016 (preview) |
| 3.1 | Server 2016 |
| 3.2 | Win10 1703 |
| 3.3 | Win10 1709 |
| 3.4 | Win10 1803 / Server 2019 |
| 3.5–3.6 | Win11 (preview) |
| 3.7 | Win11 / Server 2022 |
| 3.9 | Win11 Enterprise Insider (25236) — adds LZ4/ZSTD compression |
A v1 reader and a v3 reader parse genuinely different layouts. (XenoPanther version gist = build→version cross-ref.)
2. Existing implementations (build-vs-reuse)¶
Pure-Rust ReFS parser: effectively NONE — a genuine greenfield. The only Rust hits
are unrelated (copy_on_write uses ReFS reflink Win APIs; the windows crate's
REFS_COMPRESSION_FORMATS binding).
Reference readers / oracles:
- libfsrefs (fsrefsinfo) — most mature RE reader + structural oracle.
Completeness: shipped C library is v1-only ("finishing planned"), BUT its
PR #10 records a contributor parsing
v3.1/v3.4/v3.7/v3.9 (all dirs/files + content from data runs, with the
UCS-2+surrogate caveat) — so the documentation reaches v3.x even where the C code
doesn't. Specimens: dfirlabs refs-specimens.
- Prade's TSK fork (GitLab) — a second, independent v3.4 reader + page carver,
validated vs the Windows driver → a second oracle (breaks the LZNT1-trap correlation
with libfsrefs).
- TSK does NOT support ReFS (confirmed; mainline supports NTFS/FAT/exFAT/APFS/UFS/
ext/HFS/ISO9660/YAFFS2). ReFS-in-TSK exists only as Prade's academic fork.
- Commercial (behavioral references only): X-Ways, EnCase, Magnet AXIOM read ReFS;
Paragon ships a ReFS driver for Linux/macOS. Cross-check behavior, not spec.
Recommendation: BUILD pure-Rust clean-room refs-core from the libfsrefs asciidoc
+ Prade/Nordvik papers (no Rust prior art; C libs are LGPL and forbid-unsafe rules out
FFI). Use fsrefsinfo as the structural cross-check + Prade's TSK fork as a second,
independent structural oracle.
LZNT1-trap risk is acute here: libfsrefs is itself reverse-engineered → a wrong-reader + wrong-oracle can agree and ship green. Content-hash validation against the live Windows ReFS driver is mandatory, not optional — Windows' own driver is the only authoritative source of true file bytes. Structural metadata cannot reach Tier-1 on this filesystem.
3. Real sample data + oracle (Tier-1 plan)¶
Mint on the Parallels Windows 11 VM. Win11 client creates ReFS via Dev Drive
(Format-Volume -DevDrive, format D: /DevDrv) and, on a VHDX, via plain
Format-Volume -FileSystem ReFS. The version stamped is build-dependent (Win11/Server
2022 → v3.7; newer Insiders → v3.9); C: cannot be a Dev Drive. Record the stamped
version — it's the whole ballgame.
$vhd = "C:\cases\refs-test.vhdx"
New-VHD -Path $vhd -SizeBytes 8GB -Dynamic
Mount-VHD -Path $vhd
$disk = Get-VHD -Path $vhd | Get-Disk
Initialize-Disk -Number $disk.Number -PartitionStyle GPT
$part = New-Partition -DiskNumber $disk.Number -UseMaximumSize -AssignDriveLetter
Format-Volume -Partition $part -FileSystem ReFS -NewFileSystemLabel "REFSTEST" -Confirm:$false
$drv = ($part.DriveLetter + ":")
fsutil fsinfo refsinfo $drv # ReFS version (e.g. 3.7) + cluster/metadata block size
New-Item -ItemType Directory "$drv\dir_a\nested" | Out-Null
"hello refs" | Out-File "$drv\dir_a\known1.txt" -Encoding utf8
fsutil file createNew "$drv\dir_a\nested\big.bin" 5242880
Get-ChildItem -Recurse -File $drv | Get-FileHash -Algorithm SHA256 |
Select-Object Path,Hash | Export-Csv C:\cases\refs-truth.csv -NoTypeInformation
Remove-Item "$drv\dir_a\known1.txt" # optional: exercise CoW deleted-file recovery
Dismount-VHD -Path $vhd # then copy the .vhdx off; parse the partition offline
Oracles:
# (a) libfsrefs fsrefsinfo — STRUCTURAL, Tier-2 (reverse-engineered)
git clone https://github.com/libyal/libfsrefs && cd libfsrefs
./synclibs.sh && ./autogen.sh && ./configure && make
./fsrefsinfo /path/to/refs-partition.raw
./fsrefsinfo -H /path/to/refs-partition.raw
# reconcile vs Prade's TSK fork (fsstat/fls/usnjls) as an independent 2nd structural oracle
# (b) mount on Windows + Get-FileHash — CONTENT oracle, Tier-1 (LZNT1-trap breaker)
Explicit tiering: content = Tier-1 (Windows driver); structure = Tier-2 only
(libfsrefs/Prade, both RE). ReFS is the one filesystem where structural metadata
cannot reach true Tier-1 — state so in docs/validation.md. Corpora: rare;
dfirlabs refs-specimens; otherwise self-mint. Document per fleet provenance standard.
4. Scope/difficulty + phased order¶
Why hard — a different axis than ZFS: not algorithmic depth (ordinary B+tree, simple checksums) — undocumented + version-fragmented: reverse-engineering from incomplete third-party notes, correctness a per-version/per-build moving target. Moderate algorithmic complexity, HIGH reverse-engineering + version-testing burden — harder to validate than to write.
Target ONE version first: ReFS v3.x — the v3.4 / v3.7 line (Win10 1803+/Server 2019 → Win11/Server 2022). Rationale: most forensically relevant (Server, Hyper-V VM storage, Storage Spaces, backup targets, Win11 Dev Drives); Windows auto-upgrades on mount; v3.4 is best-documented (Prade paper + TSK code + carver). De-scope v1.2 to a later phase (materially different, low-value).
MVP refs-core scope (v3.x read-only):
1. Boot/FS-recognition at offset 0 → FSRS + Major/Minor version (gate: fail loud
with the actual version bytes on any untested version — never silently misparse).
2. Superblock (SUPB) → checkpoint (CHKP) → locate object/container table.
3. Minstore B+-tree traversal (generic key→value node walker — the reusable engine).
4. Object-table + virtual-address indirection (logical→physical before reading).
5. Page checksum validation (checksum-descriptor type/offset/size).
6. Directory-object walk → file records → attributes → data runs → file content
extraction (the Tier-1 content-hash gate).
7. UCS-2-with-unpaired-surrogates filename decoding.
What libfsrefs already solved (we follow): metadata-block layout, level signatures, Minstore node/tree headers, attribute type codes, data-run format. Prade adds v3.4 virtual addresses + CoW.
Version-coverage RISK (own explicitly): a reader validated on one build (v3.7) may break on v3.4/v3.9 (Minstore tweaked per release; v3.9 adds LZ4/ZSTD). Mitigation: version-gate hard, keep a per-version test image for every claimed version, mark unclaimed versions unsupported-fail-loud (show the version bytes).
Phases:
- refs-core — Phases 1→7, single version (v3.7 or v3.4). Panic-free bounds-checked;
fuzz target per parsed structure (boot, metadata-block, minstore node, object-table
entry, data-run); Tier-1 content-hash vs the Windows driver + Tier-2 structural
reconcile vs fsrefsinfo and Prade's TSK, in docs/validation.md. Widen versions
(v3.4, v3.9) as separate individually-validated increments.
- refs-forensic — over refs-core (or at the raw page level per the reader/
analyzer-split principle): (a) deleted-file/old-version recovery via CoW stale
metadata pages (Prade's page carver = model + oracle); (b) integrity-stream/
page-checksum analysis (mismatch = corruption/tampering); (c) USN Change Journal
+ Logfile (USN_RECORD_V3, Log Record). Findings → forensicnomicon::report::Finding
(REFS-PAGE-CHECKSUM-MISMATCH, REFS-DELETED-FILE-CARVED, REFS-USN-…).
Tiering (restated): every value-producing path in refs-core (file content, data-
run resolution) needs Tier-1 content validation (Windows driver hashes — the only
independent oracle); structural metadata is Tier-2 and must be cross-checked vs
both libfsrefs and Prade's TSK fork (avoid a single-RE-oracle blind spot).
Gaps/throttling: Andrea Fortuna intro blog (404), ScienceDirect full texts (403 paywall — DFRWS open PDFs captured instead), one MS versions page (404) — none load- bearing. The "Williballenthin/FireEye-Mandiant ReFS" reference is unverified — dropped.