NVIDIA Nemotron 3.5 Bolt E4B Review: A Local Agent in 4GB of VRAM

Bolt E4B's target is in its name: conversation and tool calls happen on the local device, not through a cloud model. NVIDIA positions it as the language component for voice assistants, interactive characters and robot interfaces; in the demo, a Reachy Mini robot runs locally on an 8GB Jetson Orin Nano, with Bolt handling dialogue and Cosmos handling vision.

1. Released by: NVIDIA, October 23, 2026
2. Parameters and base: 8B total, 4B effective, post-trained from Google's Gemma 4 E4B-it
3. Training: Reinforcement learning with verifiable rewards (RLVR) and from human feedback (RLHF), using NeMo-RL and NeMo-Gym; datasets and the full recipe are public
4. Context and languages: 64K context, 35 supported languages, on a base model pre-trained on 140+ languages
5. Modes: A conversational mode switches to concise, speech-ready replies; reasoning mode can be turned on or off
6. Price and licence: Weights are a free download under the OpenMDW-1.1 licence, with commercial use allowed
7. Download and runtime: The Q4_K_M GGUF file is about 6.18GB, with a BF16 version also available; channels include Hugging Face, Ollama, LM Studio, Unsloth, Canonical and Jetson AI Lab, running on llama.cpp
8. What 4GB means: 4GB is NVIDIA's target memory footprint, not the file size; measured VRAM use varies with hardware and context length, as shown below

NVIDIA's research page publishes 21 benchmarks, all BF16 scores measured in its own NeMo-Gym harness, with the same reasoning mode used across models within each benchmark. NVIDIA's headline summary: average scores about 40% above the previous Nemotron 3 Nano 4B and about 10% above same-size peers, with up to about 80% fewer output tokens in reasoning-mode comparisons. The comparable scores below list leads and shortfalls side by side.

1. Character dialogue, RoleBench: Bolt E4B 67.0, Gemma 4 E4B 49.9, Qwen 3.5 4B 53.7, Nano 4B 12.0
2. Role adherence, RoleMRC: Bolt E4B 80.6, Gemma 4 E4B 74.6, Qwen 3.5 4B 65.8, Nano 4B 47.0
3. Multi-turn dialogue, MultiChallenge: Bolt E4B 53.1, Gemma 4 E4B 42.1, Qwen 3.5 4B 51.8, Nano 4B 35.3 (NVIDIA BF16)
4. Instruction hierarchy, IHEval: Bolt E4B 84.8, Gemma 4 E4B 67.0, Qwen 3.5 4B 60.6, Nano 4B 51.3
5. Instruction following, IFBench: Bolt E4B 63.8, Gemma 4 E4B 42.8, Qwen 3.5 4B 58.9, Nano 4B 60.1 (NVIDIA BF16)
6. Function calling, BFCL-v3: Bolt E4B 68.5, Gemma 4 E4B 66.4, Qwen 3.5 4B 70.9, Nano 4B 61.1
7. Multi-turn customer tasks, Tau2-Bench: Bolt E4B 55.1, Gemma 4 E4B 38.5, Qwen 3.5 4B 75.1, Nano 4B 29.2
8. Long context, RULER 64K: Bolt E4B 87.0, Gemma 4 E4B 86.5, Qwen 3.5 4B 95.1, Nano 4B 91.5
9. Long prompts, LongBench v2: Bolt E4B 50.5, Gemma 4 E4B 49.3, Qwen 3.5 4B 59.9, Nano 4B 31.4
10. Cross-conversation recall, LongMemEval: Bolt E4B 72.1, Gemma 4 E4B 70.9, Qwen 3.5 4B 66.8, Nano 4B 25.2

Speed and memory figures are NVIDIA's published Q4_K_M measurements on llama.cpp at batch size 1, about 3K input tokens and 50 output tokens: RTX 5070 uses 3.5GB of VRAM, 548ms to first token, 133.4 tokens/s; RTX Spark uses 3.1GB, 930ms, 67.0 tokens/s; Jetson AGX Orin uses 4.2GB, 3,687ms, 24.0 tokens/s; Jetson Orin Nano 8GB uses 4.2GB, 5,785ms, 15.5 tokens/s. With 32K context retained, the RTX 5070 figure rises to 4.3GB and Jetson Orin Nano to 4.9GB. The DFlash speculative-decoding drafter lifts throughput up to 21% above Gemma 4 E4B's original MTP drafter.

Two independent leaderboards give external figures to set against the vendor numbers; their methods differ from NVIDIA's own runs, so their scores are not folded into the vendor average. On Scale Labs' MultiChallenge small-model leaderboard, Nemotron 3.5 Bolt E4B (Thinking) ranks 1st at 56.03±6.20, marked as new; that score is above NVIDIA's own 53.1 and comes from Scale's independently run evaluation. On Artificial Analysis' IFBench evaluation page, the reasoning version of Bolt E4B scores 61.1%, below NVIDIA's own 63.8; on Artificial Analysis' model page for this model, the summary Intelligence, Speed, Cost and Verbosity scores are currently listed as not available.

The scorecard conversion is stated openly: one representative score from NVIDIA's BF16 table per dimension, divided by 10 into a 10-point scale; the total is the mean of the five dimension scores. The converted scores only restate benchmark figures, with no editorial judgement added; independent leaderboard scores are listed separately and excluded from the total.

1. Dialogue and character, 6.7: RoleBench 67.0 converted, from NVIDIA's BF16 table
2. Instruction hierarchy, 8.5: IHEval 84.8 converted, from NVIDIA's BF16 table
3. Function calling, 6.9: BFCL-v3 68.5 converted, from NVIDIA's BF16 table
4. Multi-turn tasks, 5.5: Tau2-Bench 55.1 converted, from NVIDIA's BF16 table
5. Long context, 8.7: RULER 64K 87.0 converted, from NVIDIA's BF16 table
6. Total, 7.3: Mean of the five converted dimension scores (6.7+8.5+6.9+5.5+8.7)/5
7. Independent leaderboards, listed separately: Scale Labs MultiChallenge small-model tab, 1st place, 56.03±6.20; Artificial Analysis IFBench reasoning version, 61.1%; neither counts toward the total

On function and price, the following teams can already obtain the weights through public channels and run them on local hardware: teams that need dialogue plus tool calls in voice assistants, game characters or robots with a memory budget around 4GB; developers who already run GeForce RTX, Jetson or RTX/DGX Spark hardware and want conversation data to stay on the local device; and teams that need commercial use under the OpenMDW-1.1 licence. In the other direction, scenarios built on multi-turn customer-service tasks or retrieval beyond 64K context should first re-test on their target task, given the shortfalls in the benchmark table.