← Back to BLACKWIRE EMBER BUREAU AI RESEARCH Terminal window showing DeepSeek V3 JAX code and training logs on a MacBook Air screen

The author’s terminal displays the 190‑line JAX implementation and real‑time training metrics on a MacBook Air.

DEEPSEEK V3 JAX IMPLEMENTATION RUNS ON A MACBOOK AIR IN 190 LINES

*A 190‑line pure JAX replica of DeepSeek‑V3 trains from scratch on a MacBook Air. The demo proves high‑end transformer tricks can run on consumer silicon, but it remains a proof‑of‑concept, not a production model.*

By EMBER Bureau - BLACKWIRE  |  November 14, 2026, 23:00 CET  |  DeepSeek V3, JAX, MacBook Air, transformer, AI research

A developer has stripped the DeepSeek‑V3 transformer down to 190 lines of pure JAX and trained it on a consumer‑grade MacBook Air. The experiment, posted on Hacker News, demonstrates that sophisticated attention tricks—MLA, decoupled RoPE, MoE routing and multi‑token prediction—can run without heavyweight frameworks or GPUs. In under three hours the model processed 98 million tokens, achieving validation loss comparable to the original paper while using only 24 GB of RAM. The result is a stark reminder that the barrier to entry for cutting‑edge AI research is slipping, even as the models themselves grow more complex.

How did DeepSeek V3 run on a MacBook Air?

The author compiled the entire DeepSeek‑V3 architecture in 190 lines of pure JAX, avoiding Flax, Equinox and PyTorch. Training occurred on an Apple Silicon MacBook Air M5 with 24 GB of RAM. The run used bf16 compute on f32 master weights, pre‑fill and decode with the MLA cache. In 2.6 hours the model completed 12,000 steps, processing roughly 98 million tokens—about 4.5 epochs over a 207‑book detective‑fiction corpus. Median throughput hit 10.7 k tokens per second. No external GPU was required; the experiment proves that a modest laptop can execute the full DeepSeek‑V3 pipeline when the code is stripped to its essentials.

What are the key architectural shortcuts in the 190‑line JAX code?

The implementation follows the DeepSeek‑V3 technical report but compresses each token’s cache to 144 numbers per layer via Multi‑head Latent Attention (MLA), a reduction from the usual 480. It uses a decoupled RoPE where a single rotary key is shared across heads. DeepSeekMoE is realized with one always‑on expert plus 16 routed experts, top‑4 per token, and an auxiliary‑loss‑free balancing bias that nudges routing toward idle experts without altering weight updates. A sequence‑wise balance loss (α = 1e‑4) adds a small complementary term. Multi‑token prediction (MTP) adds a module that predicts the token after next, weighted λ = 0.3 in the loss. All components are plain functions operating on dicts of arrays.

The bias evens out the experts without costing anything, which is the paper's claim.

What performance metrics did the demo achieve?

The model contains 36.2 M parameters, with 13.6 M active per token. Validation loss fell from 5.02 at step 250 to 3.51 by step 2000. Expert load with bias settled between 5.7 % and 6.9 % per expert (even 6.25 %). Without bias the spread widened to 0.09 %‑18.6 % at the same step. Turning the bias on or off left validation loss virtually unchanged (3.921 vs 3.923). The author notes the bias evens out expert usage without costing accuracy, matching the paper’s claim. Sample output from the prompt “The fog came in off the” reads: “The fog came in off the bay, and the great trees were spreading and the stars were shining.”

How can others reproduce the experiment?

Reproduction requires an Apple Silicon Mac, Python 3.12 and uv. The repository pins every dependency; jax‑metal 0.1.1 works only with jax 0.4.34. After cloning, create a virtual environment, install requirements, and verify with pytest (39 passed). The dataset—207 public‑domain books—can be built via prepare_data.py (≈120 MB download, 7 minutes) or fetched directly as pre‑hashed train.bin, val.bin and tokenizer.json from the v1.0 release. The checkpoint (138 MB pickle) is available at the release URL. Users are warned that the pickle can execute code, so only load from trusted sources. Running on CPU is possible, but the author provides a smoke_metal.py script to test jax‑metal support.

Frequently asked questions

What hardware was used to train DeepSeek V3?

The model was trained on an Apple Silicon MacBook Air M5 with 24 GB of memory.

How many parameters does the DeepSeek V3 demo model contain?

The demo model has 36.2 million parameters, with 13.6 million active per token.

Where can the trained checkpoint be downloaded?

The checkpoint (ckpt.pkl, 138 MB) is available from the v1.0 release on the GitHub repository’s releases page.

DeepSeek V3 in 190 lines proves that the most elaborate transformer components are not exclusive to data‑center clusters. By exposing the full pipeline in readable JAX functions, the author invites scrutiny, replication and extension on modest hardware. As AI research democratizes, the next wave will likely see more hobbyists turning laptops into testbeds for ideas that were once the domain of megacorp labs. The question now is not whether the code runs, but how quickly the community can turn these demos into robust, secure tools.

Sources: Hacker News post "Show HN: DeepSeek V3 in ~200 lines of pure Jax" and the linked GitHub repository.