nm-dedupe

Store node_modules once

GET Free · macOS · Apple Silicon and Intel · 2.5 MB

Every project keeps its own copy of the same packages. nm-dedupe turns identical files across your node_modules folders into APFS clones that share their disk blocks. npm, yarn and your bundler see ordinary files, and editing a dependency in one project never touches another.

Storage

12 projects1.66 GB used

Unique filesDuplicatesAvailable

Recommendations

Files scanned

73,070

Duplicate files

53,436 (73 %)

Freed, measured

1.11 GB

Dry run · real run

2 s · 22 s

Peak memory

under 45 MB

Benchmark of 12 typical projects (React, Next, Vite, TypeScript, ESLint, Jest, Express, Webpack) on an isolated APFS volume, reproducible with bench/sample-projects.sh.

How a file gets replaced

Clone beside the fileWith clonefile(). Permissions, group, timestamps and extended attributes are copied onto the clone.

Compare every byteTimestamps are never trusted on their own, because npm, tar and rsync reset them.

Swap atomicallyWith renamex_np(). The original stays under a temporary name until the run ends.

Verify, then deleteThe original is deleted only if it still holds the same bytes. If anything wrote to it, it goes back in place.

Safety tests on real APFS

Crashes before and after every swap, kill -9 at random momentsEvery file keeps its content and metadata.

Other processes writing during a runIn place, through already open files, with restored timestamps. No write is lost.

Read-only folders, immutable files, ACLs, hard links, symlinksLeft alone.

Tabs, newlines, Unicode and 255-byte file namesHandled like any other file.

Compared to pnpm

	npm	pnpm	npm + nm-dedupe
Disk usage	a copy per project	one copy	one copy
Package manager	npm	pnpm only	any
Editing a dependency	changes one project	changes all projects	changes one project

Install

Terminal

# install the universal binary
$ curl -fsSL https://github.com/michaellocher/nm-dedupe/releases/latest/download/nm-dedupe-macos-universal.tar.gz | sudo tar -xz -C /usr/local/bin

# see what would be freed, change nothing
$ nm-dedupe -n ~/Developer

# clone duplicates and report the freed space
$ nm-dedupe ~/Developer

# run after every npm install and npm ci
$ nm-dedupe hook >> ~/.zshrc

Or build from source with cargo install --git https://github.com/michaellocher/nm-dedupe

Limits

APFS onlyClones work within one volume on macOS.

ProvenancemacOS sets com.apple.provenance per app, so a clone keeps the source's value. Every other attribute is kept exactly.

Files held openA program that writes to a file after the run has ended writes to the replaced original, as after npm install.

Time Machine snapshotsLocal snapshots keep the old blocks until they expire.

Not notarizedAfter a browser download, run xattr -d com.apple.quarantine nm-dedupe.