Baidu Releases ERNIE 5.5 Flash Base Weights: 280B Parameters, 14B Active per Token

What happened

The weights went live in the Hugging Face repository baidu/ERNIE-5.5-Flash-Base and on ModelScope at the same time. The model card describes it as the checkpoint at the end of pretraining, before any mid-training or post-training, positioned as a common starting point for developers to pursue domain adaptation, instruction tuning and reinforcement learning. Baidu previously opened up ERNIE 4.5; this is the first public weight release in the ERNIE 5.5 series.

The architecture is built around efficiency: of 43 layers, one is a dense feed-forward layer and 42 are Multi-Router Latent MoE layers, with 512 routed experts per layer, 10 selected per token plus one shared expert; 37 layers use Heavily Compressed Attention, caching a single entry per 128 tokens, and six use multi-head latent attention. Pretraining ran from scratch on 33.5 trillion tokens with the Muon optimizer, at a training context of 128K tokens, with fine-tuning and inference supporting up to 1 million tokens.

The deployment and license terms were published together. The weights ship in FP8 and BF16 versions, with SGLang and vLLM support, and Baidu released a vLLM-ERNIE plugin for vLLM 0.32.0. The license is Baidu's own ERNIE 5.5 Flash Base Model License Agreement v1.0, whose territory excludes the European Union, the United Kingdom and South Korea, with commercial and non-commercial use distinguished by actual purpose. Baidu says the full instruction-tuned ERNIE 5.5 Flash will follow in a later update.

Key facts

1. Publisher and date: Baidu, Nov 5, 2026; Hugging Face repository baidu/ERNIE-5.5-Flash-Base, also on ModelScope
2. Scale: 280B total parameters, 14B activated per token (about 5%); 43 layers, 512 routed experts, 10 selected per token plus one shared expert
3. Training: pretrained from scratch on 33.5 trillion tokens with the Muon optimizer; 128K-token training context, up to 1M tokens for fine-tuning and inference
4. Attention: 37 HCA layers (one cached entry per 128 tokens) plus 6 MLA-DSA layers; FlashMask-4 kernel
5. Positioning: a text-only base model without instruction tuning, for continued pretraining, fine-tuning and reinforcement learning
6. Benchmarks (Baidu-reported, one evaluation pipeline): 93.3 on MMLU, 78.4 on MMLU-Pro, 82.3 on HumanEval+ and 40.1 on LiveCodeBench, against 89.4, 69.2, 69.5 and 31.9 for DeepSeek-V4-Flash-Base
7. Deployment: FP8 and BF16 weights; SGLang and vLLM support; the vLLM-ERNIE plugin targets vLLM 0.32.0
8. License: ERNIE 5.5 Flash Base Model License Agreement v1.0; its territory excludes the European Union, the United Kingdom and South Korea
9. Next: the full instruction-tuned ERNIE 5.5 Flash will follow in a later update