modelmap: See Any Hugging Face Model Without Downloading It

⬅️ Back to Tools

modelmap: Architecture Maps for Any Hugging Face Model

The question everyone who runs local models re-asks every week: will this thing fit on my GPU? modelmap answers it, then goes further. Paste any Hugging Face model id and you get the whole network drawn out, animated, and explorable down to every projection. The trick is that no weights are ever downloaded.

1. It builds a real map from almost nothing. modelmap fetches config.json, about 2 KB, instantiates the model on PyTorch’s meta device, and runs a hooked fake forward pass to capture actual execution order and tensor shapes. An 8B dense model and a 671B MoE cost the same few seconds, and the graph ships as roughly 10 KB gzipped.

2. Flow mode makes the architecture move. An amber pulse replays the traced forward pass while a HUD narrates each step with true shapes, repeated layers compress into stacks with a layer 12 / 36 counter, and micro-views zoom into one block’s choreography: norm, attention, merge, norm, MLP, merge.

3. The fit planner answers the VRAM question with knobs attached. Choose GPUs, tensor or pipeline parallel split, context, batch, and weight precision, and it stacks weights plus KV cache plus activations against the card, reports KV-limited max context, and estimates roofline throughput per GPU preset. Drag context to 128k and watch the cache outweigh the model.

4. Fine-tuning gets its own tab. Full, LoRA, or QLoRA with rank choices, AdamW vs 8-bit optimizer states, ZeRO sharding, gradient checkpointing, and flash attention toggles end in per-GPU memory breakdowns, largest micro-batch, and training tok/s.

5. Quantized checkpoints work too. owner/name:Q4_K_M opens a GGUF variant with the config rebuilt from the GGUF header, so every module shows its actual quant type and bits per weight, and the cost lens reports true on-disk bytes.

6. Compare two models and it leads with derived takeaways instead of two silent trees: KV cache per token 144 KB vs 56 KB, active vs total parameters for MoE, RoPE vs learned positions, dropped biases. Point it at a base and its fine-tune and it correctly reports no structural change.

7. ⚡ live mode runs real inference in your browser. TinyLLama-v0 is 9 MB and does about 4 ms per token on CPU: genuine next-token probabilities, a per-head attention heatmap, a logit lens that shows predictions sharpen layer by layer, and head ablation where you silence one attention head and watch probabilities shift.

8. It meets you where you already work. There is a REST API with OpenAPI docs, a Python API, a CLI (dump · cost · plan · diff · train), an MCP server so Claude Code can ask whether Qwen3-32B fits on two A100s at 32k context, a Chrome extension that adds a view-in-modelmap button to Hugging Face pages, and embeddable iframes plus README badges.

9. Honest limits. The hosted server refuses trust_remote_code repos by design because that flag executes arbitrary Python; the workaround is running modelmap dump locally and dropping the .graph.json onto the site, which renders fully client-side. The free-tier host needs around 30 seconds for the first uncached model on a cold instance. Live mode covers only small llama-family and GPT-2 checkpoints up to 700 MB. And modelmap maps structure and cost, not quality: pair it with an arena leaderboard when you are choosing between models.

Install & first run

uvx --index https://download.pytorch.org/whl/cpu modelmap

One command, nothing permanent: it pins the CPU torch wheel instead of the 2.5 GB CUDA one and opens http://127.0.0.1:7860. Until the package lands on PyPI, install from GitHub by adding --from modelmap@git+https://github.com/lizhaoliu/modelmap.

uvx --index https://download.pytorch.org/whl/cpu --from modelmap@git+https://github.com/lizhaoliu/modelmap modelmap plan Qwen/Qwen3-8B --gpus 2 --gpu-memory 24 -T 32768

That prints the fit verdict straight to your terminal. Or skip the terminal entirely: open modelmap.cc, hit fit? on any gallery card, and drag the context slider.

Worth your time if: you run, fine-tune, or teach LLMs and want structure and VRAM math without pulling gigabytes of weights. Skip it if you only want benchmark scores.

Related TMFNK Content

• llmfit: Find Which LLM Models Run on Your Hardware The same will-it-fit question, answered as a scored lookup across models rather than a per-model planner.
• hfviewer: Visualize Any Hugging Face Model Another lens into Hugging Face repos when model cards are not enough.
• Scaling Has More Than One Dial: Jie Tang on Why Parameter Count Isn’t Enough Tang’s total-vs-active parameter distinction is exactly what modelmap’s cost lens makes visible per token.

Crepi il lupo! 🐺