SIGN IN SIGN UP

[3.14] gh-135676: Simplify docs on lexing names (GH-140464) (GH-142015)

This simplifies the Lexical Analysis section on Names (but keeps it technically correct) by putting all the info about non-ASCII characters in a separate (and very technical) section.

It uses a mental model where the parser doesn't handle Unicode complexity “immediately”, but:

- parses any non-ASCII character (outside strings/comments) as part of a name, since these can't (yet) be e.g. operators
- normalizes the name
- validates the name, using the xid_start/xid_continue sets

(cherry picked from commit 2ff8608b4da33f667960e5099a1a442197acaea4)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Micha Albert <info@micha.zone>
Co-authored-by: KeithTheEE <kmurrayis@gmail.com>
S
Stan Ulbrych committed
79245a47ed127c14f63e1450a1b554d86c266a5d
Parent: 206f196
Committed by GitHub <noreply@github.com> on 12/3/2025, 12:43:06 PM