tokenizers BpeBuilder Buffer Overflow via merge token
cve-program · CVE-2026-85670
Publishedtokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.
Read the publisher's advisoryAffected versions, risk and source history
Exploitation and risk
Source statusCISA KEV lists vulnerabilities known to be exploited. FIRST EPSS estimates the likelihood of exploitation in the next 30 days. An unavailable score does not mean zero risk.
Which CVEs have scores here?
This view includes predictions for CVE identifiers explicitly listed in public NuGet advisories. Microsoft release pages have their own prediction coverage. Check source status for the scope and latest checks.
No exploitation or prediction data is available here for these CVEs. This does not mean they are safe or unlikely to be exploited.
cve-program · CVE-2026-85670
Publishedtokenizers BpeBuilder Buffer Overflow via merge token
tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.
- Source published
- 4 Sep 2026 14:32:25 UTC
- Source modified
- 4 Sep 2026 17:49:42 UTC
- Saved here
- 23 Oct 2026 15:48:27 UTC
- Source terms
- CVE Program Terms of Use
Original source record · Cite this exact observation · Complete original publisher JSON
Source: cve-program · CVE Program Terms of Use. The publisher's wording is preserved. ChangeIntel groups the information and labels changes. The JSON record includes additional source fields.
CVSS 4.0 · 7.1 CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N
CVSS 3.1 · 6.5 CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H