Skip to content

Add support for the Python 3.15+ lazy keyword#2292

Open
sirosen wants to merge 4 commits into
rouge-ruby:mainfrom
sirosen:support-python-lazy
Open

Add support for the Python 3.15+ lazy keyword#2292
sirosen wants to merge 4 commits into
rouge-ruby:mainfrom
sirosen:support-python-lazy

Conversation

@sirosen

@sirosen sirosen commented May 24, 2026

Copy link
Copy Markdown

New in Python 3.15, there is a lazy keyword which allows for deferred
imports.

Also update the demo to make use of lazy, type annotations, and None,
all of which make the demo a bit more complete.

New in Python 3.15, there is a `lazy` keyword which allows for deferred
imports.

Also update the demo to make use of `lazy`, type annotations, and `None`,
all of which make the demo a bit more complete.
@jneen

jneen commented May 29, 2026

Copy link
Copy Markdown
Member

Hi, thanks for this contribution! I think a thing that complicates this implementation is that lazy is not a true keyword, but is rather specified as a "soft" keyword, meaning that variables and classes named lazy are still allowed.

Ref: https://peps.python.org/pep-0810/#grammar

This may require some special handling similar to case and match. It may be a bit easier since we can lookahead for import.

@sirosen

sirosen commented May 29, 2026

Copy link
Copy Markdown
Author

Ah! Sorry, I clearly under-thought this a little. (I got excited when it looks "so easy" 😉 )

I'm pretty out of practice in Ruby, but let me see if I can make the requisite changes to support this as a soft keyword.

@jneen

jneen commented May 29, 2026

Copy link
Copy Markdown
Member

No worries at all, that's how it is :]. I think something like a lookahead with /lazy(?=[ \t]+(from|import)/ would be fine.

Account for the fact that `lazy` is also a valid identifier. Capture that
you know it's a keyword *iff* the next token is `from` or `import`.

Since there's no valid context in Python for putting two identifiers
adjacent without an operator, we don't need to worry about what comes
after `from` or `import`.
@sirosen

sirosen commented May 29, 2026

Copy link
Copy Markdown
Author

Okay, that was a fun little adventure! The python parser already had a nice expression for handling trailing backslashes, so I picked that up.

Here's how my addition to the visual spec renders:
image

I didn't find that in my first search over the repo for tests, which is why I modified the demo. Should I revert the demo addition? (I like it, but it might be slightly too fancy for that purpose.)

@sirosen

sirosen commented May 29, 2026

Copy link
Copy Markdown
Author

Of course, the moment I share that, I stare at it and don't... love it. lazy import with from colorized differently on the same line looks sort of odd. I see now that the styling picked up for a from <namespace> import <name> is that from and import become Keyword::Namespace.

I'd like to make lazy do the same. I'm about to start looking/tinkering but I'm not as confident that I know how to make this change.

@sirosen

sirosen commented May 29, 2026

Copy link
Copy Markdown
Author

Okay, that's fixed. I now grasp that the lazy was actually sort of breaking from-import handling. Result renders much more correctly:
image

My question about reverting the demo change stands.

Also, let me know if I should rebase and clean up.
(Sorry, I know this is a quick barrage of comments + commits.)

The `from_import` state lexing now accounts for `lazy import` being the
next tokens.
@sirosen sirosen force-pushed the support-python-lazy branch from 67a0d35 to 05f0e86 Compare May 29, 2026 22:53
Comment thread lib/rouge/lexers/python.rb Outdated
# handle `lazy import` soft keyword usage
# since `import` is a hard keyword, we know that this usage has to be
# the soft-keyword case
rule %r/lazy(?=#{inline_ws}import)/, Keyword

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small nit, but import should probably have a \b at the end so it doesn't match things like important.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I believe this will match lazyimport with no space, which should be an identifier. Probably the easiest to do is %r/lazy\b(?=#{inline_ws}import\b)/.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Yep, good catch! I didn't realize that inline_ws matches on "".
I've applied (co-authored commit! 😁 ).

Co-authored-by: Jeanine Adkisson <225017+jneen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants