Baidu reports inference gains from ERNIE Evolve

Baidu says ERNIE Evolve, its automated research system, increased deterministic inference throughput for the in-development ERNIE-5.5-Flash by 33.7%. The company announced the result on 9 November, after roughly three days of experiments on a setup its engineers had already optimised.

The workload required identical generated tokens and log probabilities regardless of which requests shared a batch. Baidu says Evolve replaced a costly sampling operation with a parallel Triton kernel, then adjusted attention, matrix multiplication and other parts of the inference stack.

Evolve chooses research directions, edits code and runs experiments. A fixed evaluator sits outside the repository being modified, while a separate reviewer checks whether changes exploit the tests. The inference gains remain company-reported.

What Baidu has published

The public results repository contains benchmark code, run logs and reproduction instructions. It covers separate training and GPU-kernel experiments, rather than the internal inference tests behind the throughput claim. The complete research system is absent, and the repository currently has no licence file.

One released result reaches NanoGPT’s target training loss in about 38.9 seconds on eight H100 GPUs. Baidu supplies logs from 18 runs, all below the loss threshold. Its instructions also disclose about seven minutes of first-run compilation and setup, excluded from the reported training time.

A second NanoGPT result reduces the number of training steps, but each step takes about 2.1 times as long as the baseline. That trade-off fits the optimisation track’s rules, which allow unlimited wall-clock time. A lower step count alone does not establish faster training.

For SOL-ExecBench, the release contains seven GPU kernels and their submission scores. Those files let researchers examine individual solutions, but cannot reproduce the blog’s aggregate result across the benchmark. The benchmark’s own task definitions and workload data must be obtained separately.

The work targets the forthcoming ERNIE-5.5-Flash. Baidu’s earlier Flash Base release provides a checkpoint for further training under a custom licence, with geographic and commercial conditions covered in Sulat’s examination of the model’s terms.