NVIDIA tunes a 4 GB open model for voice assistants and game characters
NVIDIA has retrained Google's small Gemma 4 model to follow instructions and use tools better, while keeping replies short. It fits in about 4 GB of memory and comes with a helper that speeds up replies.


What the study shows
The model is called Nemotron 3.5 Bolt E4B. It starts from Gemma-4-E4B-it, a small open model from Google. NVIDIA trained it further using reinforcement learning, where a model improves by being rewarded for good answers. The training targeted multi-turn tool use, following instructions, and output formatting. A compressed 4-bit version fits in about 4 GB of memory.
The team tested it on 21 benchmarks against Gemma-4-E4B-it, Qwen3.5-4B and NVIDIA's earlier Nemotron-3 Nano-4B. The authors report it scored best on 15 of the 21 tests with reasoning switched on, and 14 of 21 with it off. Reasoning here means the model writes out working steps before answering.
The largest gains over Gemma were in following instructions. On a test of resolving conflicts between system and user instructions, the score rose from about 67 to about 85. On a test that mixes tool use with dialogue, it rose from about 39 to about 55.
Averaged across all 21 tests, the authors report about 64% accuracy in reasoning mode against about 58% for Gemma and Qwen. It also used fewer output words. Qwen needed roughly five times as many output tokens, the word pieces a model generates.
The team also trained a small helper model called a drafter. It guesses several words ahead and the main model checks them, a method called speculative decoding. On one NVIDIA workstation card this made replies for a single user about 2.7 times faster than the model alone.
The paper is a technical report from NVIDIA posted as a preprint. It has not been peer reviewed. NVIDIA ran every evaluation, including those for the rival models.
Our reading
In our reading, the most useful point is the balance between accuracy and brevity. A voice assistant that talks too much is slow and annoying. This model scored higher than its parent while producing shorter answers in both modes. That pairing matters more for spoken use than a raw score.
The win is not across the board. Qwen3.5-4B still leads on several tool-use tests and on long documents. On the dialogue-plus-tools test, Qwen scored about 75 against Bolt's 55. Teams whose assistant must handle long files or complex customer-service flows should look closely at those rows.
Readiness: level III, open source available. Four model files are listed as released on Hugging Face, with a downloadable NVIDIA container and a free test endpoint. There is no documented use by a named organisation, so it does not reach level IV.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | The method is described: RL-only post-training from Gemma-4-E4B-it, verbosity control, chat template changes and a speculative decoding drafter. |
| II Benchmarked | Met | Results on 21 public benchmarks, including BFCL-v3, Tau2-Bench, IFBench and RULER, plus speed tests on SPEED-Bench. Caveat: NVIDIA ran all evaluations itself, and two benchmarks are internal. |
| III Open source | Met | Four checkpoints are listed as released on Hugging Face (4-bit GGUF, BF16, and two drafters). A downloadable NVIDIA NIM container and a free API endpoint exist. One dataset is released. Training libraries NeMo-RL and NeMo-Gym are on GitHub. The training data blend is linked in Table 1. Not confirmed: the licence terms and whether the full training recipe can be reproduced. |
| IV In production | Not yet | No named organisation is reported using it in production. The live tests (a Reachy Mini robot, a game NPC) are NVIDIA demos and qualitative. |
What would move it up
Documented use by a named third party in a shipped product (a voice assistant, device or game) would move it to level 4. Independent replication of the benchmark results, and public release of the brevity and When2Call-MT benchmarks, would strengthen level 2.
The study
NVIDIA, :, Adi Margolin et al.. Nemotron 3.5 Bolt E4B 4GB: An Open and Efficient Small Language Model for Fast Conversational Agents. arXiv, 16 November 2026. Read the paper
Notes
- NVIDIA, Nemotron 3.5 Bolt E4B 4GB: An Open and Efficient Small Language Model for Fast Conversational Agents. arXiv preprint 2611.20771, November 2026. CC BY 4.0. Source
- Released checkpoints named in the paper: NVIDIA-Nemotron-3.5-Bolt-E4B-4GB (4-bit GGUF), NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash and NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4, on Hugging Face. Direct links appear only as hyperlinks in the PDF.
- Free API endpoint listed in the paper. Source
- NeMo-RL, the training library used. Source
- NeMo-Gym, the training and evaluation environment library used. Source
- Exact figures: reasoning-mode average about 63.9% with 672 output tokens, against Gemma-4-E4B-it 58.1% (782 tokens) and Qwen3.5-4B 57.8% (3,604 tokens), read from Figure 2. IHEval 84.79 vs 67.04, Tau2-Bench 55.10 vs 38.50 (Gemma) and 75.06 (Qwen), Table 3. Drafter speed-up 2.74x on RTX PRO 6000 and 3.56x on DGX Spark at concurrency 1, Table 11. Against Gemma with MTP: 272.6 vs 225.9 tokens/s on RTX 5070, 124.4 vs 118.4 on RTX Spark N1X, Table 12.
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections