Hugging Face cache copies: fix broken snapshot symlinks

tutorialtroubleshootingworkflow

TL;DR: A Hugging Face cache snapshot can contain links to files outside the snapshot directory. Copying that directory while preserving its links can produce a broken model export. Inspect the links, choose a complete-cache transfer or a materialized directory, and validate the destination before deleting anything from the source.

What you’ll have after this guide:

  • A read-only report of broken and externally resolved file links.
  • A reproducible demonstration of why a snapshot-only copy can fail.
  • A clear choice between preserving cache structure and exporting regular files.

Honest take: For a portable model folder, copy the referenced file contents and verify the result; a snapshot directory name alone is not a backup.

Why does a copied model folder exist but fail to load?

A copied Hugging Face snapshot can preserve symbolic links whose targets were left behind in the original cache. The filenames still appear in a directory listing, yet opening the linked weights fails because the referenced blob is missing at the destination.

The Hub cache guide describes the relationship between revision snapshots and shared blobs. A valid source snapshot can refer outside its own directory while remaining entirely inside a healthy cache. That layout is useful for avoiding duplicate storage; it needs an explicit transfer policy when you move only part of the tree.

This problem is different from a shard index naming a file that was never downloaded. It is also different from a Git LFS pointer containing text instead of weights. Start with the exact path passed to your loader. Determine whether the entry is an ordinary file, a working link, or a dangling link before changing the inference runtime.

The published shard preflight deliberately rejects external links under its self-contained-directory policy. A shared Hub cache may be healthy while failing that policy. The fix is to choose the intended deployment layout, not to call every cache link corrupt.

Which transfer layout should you choose?

Choose a complete cache transfer when you need the cache structure itself, and a materialized directory when you need a portable collection of files. Keeping the original layout and exporting a self-contained directory solve different operational problems.

Transfer choiceWhat must arrive togetherMain tradeoff
Preserve a cache treeReferenced blobs and snapshot links with their relationshipsShared storage remains useful, but a partial copy can break links
Copy one snapshot with links preservedEvery referenced target at the expected relative locationEasy to make an incomplete export
Materialize a snapshotThe referenced file contents become regular destination filesExport can duplicate bytes that the cache shared
Download into a selected local directoryThe files selected for that directoryDownload selection still must include the files the loader needs

The Hub download guide documents local-directory downloads as an alternative to treating the cache as a deployment format. A downloaded directory is still only as complete as its selected files; an allowlist that excludes tokenizer or configuration files can leave an unusable application bundle.

Do not delete the source cache immediately after a copy command exits successfully. Validate the destination independently first. A successful copy reports that the requested operation completed; it does not prove you requested all files or preserved the intended model revision.

Use a filesystem inventory that reports links separately from ordinary files. The helper below uses Python’s standard library, performs no Hub login, downloads no weights, and executes no model code.

Save it as cache_links.py and run it against the directory you intend to move:

python3 cache_links.py /path/to/model-directory
import json
import sys
from pathlib import Path


def inventory(root):
    root = Path(root).resolve(strict=True)
    if not root.is_dir():
        raise ValueError("expected a directory")
    result = {"regular_files": 0, "symlinks": 0,
              "broken_links": [], "external_links": [],
              "directory_links": [], "logical_bytes": 0}
    for path in sorted(root.rglob("*")):
        name = path.relative_to(root).as_posix()
        if path.is_symlink():
            result["symlinks"] += 1
            try:
                target = path.resolve(strict=True)
            except (OSError, RuntimeError):
                result["broken_links"].append(name)
                continue
            if not target.is_relative_to(root):
                result["external_links"].append(name)
            if target.is_dir():
                result["directory_links"].append(name)
            elif target.is_file():
                result["logical_bytes"] += target.stat().st_size
        elif path.is_file():
            result["regular_files"] += 1
            result["logical_bytes"] += path.stat().st_size
    return result


if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise SystemExit("usage: cache_links.py DIRECTORY")
    report = inventory(sys.argv[1])
    print(json.dumps(report, indent=2))
    raise SystemExit(1 if report["broken_links"] else 0)

broken_links identifies links whose targets cannot be resolved by this process. external_links identifies resolved links outside the supplied root. An external link is a portability dependency, not automatically an error. A link can be external and healthy in the source cache, then become broken after a snapshot-only copy.

The helper exits with status 1 when it finds broken links. Other access errors may also stop the script rather than producing a complete report. Keep the directory still while inspecting it: a downloader or cleanup process changing targets during traversal can make a point-in-time report stale.

Directory links are listed but not recursively inventoried through the link. This is intentional. The helper is designed for the usual file-link snapshot pattern, not as a general backup validator for arbitrary symlinked directory graphs. If directory_links is nonempty, inspect those dependencies separately before using the byte total for planning.

What does the reported byte count mean?

logical_bytes sums the sizes of the ordinary files and resolved linked files visited by the helper. It is an export-content estimate for that traversal, not a physical disk-allocation measurement or a count of unique shared blobs.

If two filenames point at the same blob, each name contributes the target’s size. A materialized export may need those contents at both destination names. Conversely, filesystem compression, sparse files, hard links, and deduplication can make physical allocation differ from logical lengths. The helper does not inspect those allocation mechanisms.

This distinction is useful before filling a transfer disk. A source snapshot may look small when counting only link entries, while a materialized directory needs room for the referenced contents. Do not calculate a universal storage multiplier: the amount of sharing and the selected revisions determine the difference.

The diagnostic requires an existing computer and Python, with no GPU requirement and no paid API. A real model export needs enough destination storage for the selected files and sufficient temporary headroom for the transfer method. We did not measure throughput, cloud storage charges, or the disk needs of a named model here.

Can you reproduce the broken-copy problem safely?

Create a disposable fixture with one blob containing four bytes, then put a relative link inside a snapshot. This mirrors the structural dependency without downloading any model weights or assuming a model-specific file format.

mkdir cache-copy-demo
cd cache-copy-demo
python3 - <<'PY'
from pathlib import Path
import shutil
cache = Path('cache')
blob = cache / 'blobs' / 'fixture'
blob.parent.mkdir(parents=True)
blob.write_bytes(b'ABCD')
snapshot = cache / 'snapshots' / 'fixture-revision'
snapshot.mkdir(parents=True)
(snapshot / 'weights.bin').symlink_to('../../blobs/fixture')
shutil.copytree(snapshot, 'links-only', symlinks=True)
shutil.copytree(snapshot, 'portable', symlinks=False)
print((snapshot / 'weights.bin').read_bytes())
print(Path('links-only/weights.bin').is_symlink())
print(Path('links-only/weights.bin').exists())
print(Path('portable/weights.bin').read_bytes())
PY

Observed on Linux with Python 3.13.5, November 12, 2026:

b'ABCD'
True
False
b'ABCD'

The link-only copy still contains a symbolic link, but its target cannot be resolved. The materialized copy contains the original four bytes as a regular file. These are filesystem fixture results, not a real model restore or an inference test.

The published inventory helper was also run against all three fixture directories:

Fixture directoryRegular filesLinksBroken linksLogical bytes
Original snapshot0104
Link-preserving copy0110
Materialized export1004

The original snapshot reports one external target because the blob lives outside the snapshot root. That result is expected and valid inside this fixture cache. The copy becomes broken because the relative target is interpreted from its new location, where the matching blob directory does not exist.

How do you create a materialized export from a healthy snapshot?

Copy a stable, inspected source into a new destination with file links dereferenced. Python’s shutil documentation defines copytree behavior: symlinks=False copies linked contents rather than preserving the links themselves.

For a source you control and have already inspected, the core operation is:

python3 - <<'PY'
from pathlib import Path
import shutil
source = Path('/path/to/healthy/snapshot')
destination = Path('/path/to/new/export-directory')
if destination.exists() or destination.is_symlink():
    raise SystemExit('choose a new destination; existing path preserved')
shutil.copytree(source, destination, symlinks=False)
PY

Choose a new destination so the operation cannot silently blend files from different revisions. Do not add dirs_exist_ok=True merely to make a failed transfer stop complaining. A partially populated directory is useful evidence; inspect it or select another new destination rather than treating a merge as a clean export.

Dereferencing follows source links. It is appropriate for a trusted, known snapshot layout, not for blindly exporting an arbitrary directory that could point into unrelated personal files. Inspect external targets first and limit the source to the specific revision you intend to transfer. This helper and copy recipe are not a security boundary against another process changing links during the operation.

After copying, run the inventory against the destination. Then check expected configuration, tokenizer, and weight files using the loader’s documented requirements. Absence of broken links does not prove completeness, valid weight serialization, or that the model architecture is supported.

How do you verify the exported contents independently?

Compare the destination against an independently saved identity record for the intended revision. A file count and a directory name cannot establish that the bytes match the intended source.

Use the SHA-256 manifest workflow for a separately saved baseline, and the Git LFS pointer check when a suspiciously small weights file may contain only a pointer. The checks answer different questions: link resolution, byte identity, and pointer detection should not be collapsed into a single “model valid” label.

A real restore test also includes the consuming application. If you claim an offline deployment works, test that application with its declared files and environment under the intended network policy. We did not perform an offline model load here. The fixture demonstrates a filesystem failure before any inference framework is involved.

Honest take: when should you avoid this export recipe?

Avoid materializing the snapshot when you actually need to preserve a shared cache across many revisions and already have a transfer process that carries its referenced blobs correctly. Copying every revision into separate ordinary folders can sacrifice the storage sharing that made the cache useful.

Also stop when the source has unresolved links, unexpected directory links, permission failures, or ongoing downloads. Repair or finish the source state first. Enabling “ignore dangling links” would conceal the very omission you are trying to detect.

The huggingface_hub library is Apache-2.0-licensed; its current latest release was v2.4.0 when checked November 12. The scripts here use only the Python standard library and do not require installing that package. The library’s license does not grant a model’s redistribution rights: inspect the model repository’s own license before sharing an exported directory.

For a portable handoff, favor a new materialized destination, a saved revision identifier, a separately retained integrity manifest, and a real loader check. Keep the original source until those checks pass.

Sources

Last updated November 12, 2026. Synthetic filesystem fixtures only; no model weights, model loading, GPU benchmark, or storage-price measurement.

Was this article helpful?

What self-hosting actually costs

Real cost breakdowns for self-hosted AI: hardware floors, power, maintenance hours, and the honest comparison against paying for it. No spam, unsubscribe anytime.