NLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
cve-program · CVE-2026-72818 · Published
No access to your tenant or devices. Every item links to its source. What this means
Historical advisory recorded 23 Oct 2026 15:48:27 UTC. Read the latest advisory.
The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
Read the publisher's advisorySee how the publisher's advisory changed over time. Each link opens the version saved at that time. The publisher's update date can differ from the date shown here.
Each link identifies one complete publisher record. History can include advisories that previously shared an identifier; an identifier removed by a publisher no longer establishes a current relationship. A historical link shows only that record and earlier saved versions. Failed checks and unchanged reads do not create new versions.
cve-program · CVE-2026-72818 · Published
This historical link shows only this version and versions saved before it. Browse all advisory history.