COMMITS
/ Lib/test/test_tokenize.py April 26, 2026
H
gh-148991: Add colour to `tokenize` CLI output (#148992)
Hugo van Kemenade committed
October 29, 2025
M
gh-140576: Fixed crash produced by lexer in case of dedented zero byte (#140583)
Mikhail Efimov committed
October 20, 2025
S
gh-63161: Fix tokenize.detect_encoding() (GH-139446)
Serhiy Storchaka committed
October 7, 2025
T
gh-139516: Fix lambda colon start format spec in f-string in tokenizer (#139657)
Tomasz Pytel committed
September 30, 2025
S
gh-63161: Add more tests for source encoding (#139440)
Serhiy Storchaka committed
May 27, 2025
E
gh-134752: Improve speed of test_tokenize.StringPrefixTest.test_prefixes. (#134766)
Eric V. Smith committed
May 26, 2025
May 25, 2025
L
April 18, 2025
P
gh-129958: Properly disallow newlines in format specs in single-quoted f-strings (GH-130063)
Pablo Galindo Salgado committed
March 27, 2025
February 25, 2025
M
GH-130396: Use computed stack limits on linux (GH-130398)
Mark Shannon committed
February 24, 2025
February 19, 2025
M
GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)
Mark Shannon committed
February 4, 2025
S
gh-127933: Add option to run regression tests in parallel (gh-128003)
Sam Gross committed
January 21, 2025
T
gh-125553: Fix backslash continuation in `untokenize` (#126010)
Tomas R. committed
October 6, 2024
T
gh-125008: Fix `tokenize.untokenize` roundtrip for `\n{{` (#125013)
Tomas R. committed
July 19, 2024
S
gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907)
Serhiy Storchaka committed
June 12, 2024
L
gh-120343: Fix column offsets of multiline tokens in tokenize (#120391)
Lysandros Nikolaou committed
June 11, 2024
L
gh-120343: Do not reset byte_col_offset_diff after multiline tokens (#120352)
Lysandros Nikolaou committed
February 19, 2024
P
gh-115154: Fix untokenize handling of unicode named literals (#115171)
Pablo Galindo Salgado committed
December 11, 2023
P
gh-112943: Correctly compute end offsets for multiline tokens in the tokenize module (#112949)
Pablo Galindo Salgado committed
October 19, 2023
N
gh-111031: Check more files in `test_tokenize` (#111032)
Nikita Sobolev committed
October 12, 2023
L
gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)
Lysandros Nikolaou committed
October 10, 2023
N
gh-108303: Move all inspect test files to `test_inspect/` (#109607)
Nikita Sobolev committed
October 5, 2023
P
gh-110259: Fix f-strings with multiline expressions and format specs (#110271)
Pablo Galindo Salgado committed
September 12, 2023
N
gh-108303: Move tokenize-related data to Lib/test/tokenizedata (GH-109265)
Nikita Sobolev committed
September 5, 2023
A
gh-108469: Update ast.unparse for unescaped quote support from PEP701 [3.12] (#108553)
Anthony Shaw committed
July 26, 2023
P
gh-107015: Remove async_hacks from the tokenizer (#107018)
Pablo Galindo Salgado committed
June 28, 2023
N
gh-106200: Remove unused imports (#106201)
Nikita Sobolev committed
June 19, 2023
L
gh-71299: Fix __all__ in tokenize (#105907)
Lysandros Nikolaou committed
June 15, 2023
L
gh-105820: Fix tok_mode expression buffer in file & readline tokenizer (#105828)
Lysandros Nikolaou committed
June 13, 2023
L
gh-105718: Fix buffer allocation in tokenizer with readline (#105728)
Lysandros Nikolaou committed
June 9, 2023
P
gh-105549: Tokenize separately NUMBER and NAME tokens and allow 0-prefixed literals (#105555)
Pablo Galindo Salgado committed
P
gh-105564: Don't include artificial newlines in the line attribute of tokens (#105565)
Pablo Galindo Salgado committed
June 7, 2023
P
gh-105435: Fix spurious NEWLINE token if file ends with comment without a newline (#105442)
Pablo Galindo Salgado committed
P
gh-105390: Correctly raise TokenError instead of SyntaxError for tokenize errors (#105399)
Pablo Galindo Salgado committed
June 6, 2023
P
gh-105259: Ensure we don't show newline characters for trailing NEWLINE tokens (#105364)
Pablo Galindo Salgado committed
May 30, 2023
L
gh-105042: Disable unmatched parens syntax error in python tokenize (#105061)
Lysandros Nikolaou committed
P
gh-105069: Add a readline-like callable to the tokenizer to consume input iteratively (#105070)
Pablo Galindo Salgado committed
May 28, 2023
M
gh-105017: Include CRLF lines in strings and column numbers (#105030)
Marta Gómez Macías committed
May 27, 2023
M
gh-105017: Fix including additional NL token when using CRLF (#105022)
Marta Gómez Macías committed
May 26, 2023
P
gh-104976: Ensure trailing dedent tokens are emitted as the previous tokenizer (#104980)
Pablo Galindo Salgado committed
P
gh-104972: Ensure that line attributes in tokens in the tokenize module are correct (#104975)
Pablo Galindo Salgado committed
May 24, 2023
L
gh-104866: Tokenize should emit NEWLINE after exiting block with comment (#104870)
Lysandros Nikolaou committed
P
gh-104825: Remove implicit newline in the line attribute in tokens emitted in the tokenize module (#104846)
Pablo Galindo Salgado committed
May 22, 2023
M
gh-104741: Add line number attribute to indentation error exception (#104743)
Marta Gómez Macías committed
May 21, 2023
M
gh-102856: Python tokenizer implementation for PEP 701 (#104323)
Marta Gómez Macías committed
May 1, 2023
C
gh-103824: fix use-after-free error in Parser/tokenizer.c (#103993)
chgnrdv committed
April 19, 2023
P
gh-102856: Initial implementation of PEP 701 (#102855)
Pablo Galindo Salgado committed