NLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
Historical advisory recorded 23 Oct 2026 15:48:27 UTC. Read the latest advisory.
cve-program · CVE-2026-72818
PublishedThe URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
Read the publisher's advisoryAffected versions, risk and source history
cve-program · CVE-2026-72818
PublishedNLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
- Source published
- 20 Aug 2026 21:57:34 UTC
- Source modified
- 24 Sep 2026 14:18:57 UTC
- Saved here
- 23 Oct 2026 15:48:27 UTC
- Source terms
- CVE Program Terms of Use
Original source record · Cite this exact observation · Complete original publisher JSON
Source: cve-program · CVE Program Terms of Use. The publisher's wording is preserved. ChangeIntel groups the information and labels changes. The JSON record includes additional source fields.
CVSS 4.0 · 8.7 CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N
CVSS 3.1 · 7.5 CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H