tests: add whitespace tests for vertical tab behavior by Brace1000 · Pull Request #155028 · rust-lang/rust

Brace1000 · 2026-04-09T05:28:53Z

This PR adds two small tests to highlight how vertical tab (\x0B)
is handled differently across Rust's whitespace definitions.

The Rust lexer treats vertical tab as whitespace (Unicode
Pattern_White_Space), while split_ascii_whitespace follows the
WhatWG Infra Standard and does not include vertical tab.

These tests make that difference visible and easier to understand.

See: rust-lang/rust-project-goals#53

rustbot · 2026-04-09T05:29:01Z

r? @dingxiangfei2009

rustbot has assigned @dingxiangfei2009.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: compiler
compiler expanded to 69 candidates
Random selection from 11 candidates

Brace1000

fix tidy for whitespace tests spaces, trailing newline

View changes since this review

teor2345

Thanks, just needs a few tweaks

View changes since this review

teor2345 · 2026-04-10T02:28:13Z

+    let x = 5;
+    let y = 10;
+    let z = x + y;
+


Since vertical tab doesn't show up in GitHub's PR review rendering, please put a comment above each line containing the whitespace.

You might want to add lines with each of the 11 permitted whitespace characters:
https://doc.rust-lang.org/reference/whitespace.html

And then some lines with the other 14 disallowed whitespace characters (the ones from this list marked White_Space, that aren't in the first list):
https://www.unicode.org/Public/UCD/latest/ucd/PropList.txt

Done! Added a comment above each line with an invisible whitespace character so they are visible in the diff. Also expanded the test to cover all 11 permitted Pattern_White_Space characters inline, and listed the 14 disallowed Unicode White_Space characters in comments since placing them between tokens would cause a compile error.

You can create a separate test that fails, and put //@ check-fail at the top of the file:
https://rustc-dev-guide.rust-lang.org/tests/directives.html#controlling-outcome-expectations

This is how to test whitespace that is not allowed in Rust source code.

Thanks for the feedback! I’ve added the failing UI test and updated it to match the full stderr output, including the help message. I’ll go through it again and make the remaining tweaks.

teor2345 · 2026-04-10T02:28:39Z

@@ -0,0 +1,22 @@
+// This test checks that split_ascii_whitespace does NOT split on


I'm not sure if this test is relevant to the compiler?

Fair point. The test documents the gap between what the lexer accepts and what the stdlib gives you. Happy to remove it if you think it doesn't belong here.

teor2345 · 2026-04-10T02:29:39Z


 Tests on `where` clauses. See [Where clauses | Reference](https://doc.rust-lang.org/reference/items/generics.html#where-clauses).

+## `whitespace`


This will need an explanation of why the whitespace tests are needed. It's a good place to mention that is_ascii_whitespace and is_whitespace in the standard library don't match the Rust language's definition of whitespace.

Added! The README now explains that the Rust lexer uses Unicode Pattern_White_Space, which differs from both is_ascii_whitespace (WhatWG, skips vertical tab) and is_whitespace (Unicode White_Space, broader set). That context makes it clearer why these tests exist

teor2345 · 2026-04-10T23:37:46Z

+// the standard library's is_ascii_whitespace does NOT include vertical
+// tab, following the WhatWG Infra Standard instead.
+//
+// See: https://github.com/rust-lang/rust-project-goals/issues/53


View changes since the review

Where did you get this link? It's not the Outreachy tracking issue.

Got it, Fixed it to point to the correct Outreachy tracking issue.

Add two small tests to highlight how vertical tab is handled differently. - vertical_tab_lexer.rs checks that the lexer treats vertical tab as whitespace - ascii_whitespace_excludes_vertical_tab.rs shows that split_ascii_whitespace does not split on it This helps document the difference between the Rust parser (which accepts vertical tab) and the standard library’s ASCII whitespace handling. See: rust-lang/rust-project-goals#53

…e chars

make sure tabs and spaces are well checked

Updated error message to clarify invisible characters.

Updated error message for invisible characters in code.

Add error message for invalid whitespace in code

Updated the test to check for invalid whitespace handling.

Fix formatting issue by adding space around '=' in variable declaration.

Update the error messages for invalid whitespace in the test.

teor2345

Looks like you're struggling a bit, here are some notes about what to update in the tests

View changes since this review

teor2345 · 2026-04-15T00:29:11Z

+   |
+   = help: invisible characters like '\u{200b}' are not usually visible in text editors
+
+error: aborting due to 1 previous error


You need to update this file so it matches the test output exactly.

The easiest way to do this is ./x test ui/whitespace --bless

Got it, thanks! I ran "./x test ui/whitespace --bless" and updated the file to match the expected output. Everything is passing on my end now

Removed unnecessary blank lines from the test file.

Brace1000 · 2026-04-15T08:18:56Z

Hi @teor2345 , Since today is the deadline, I just wanted to check , is it okay if I keep contributing to this PR while it’s still under review? It’s almost there, so I’d be happy to address any remaining feedback if that works.

teor2345 · 2026-04-15T09:10:59Z

You can put links to incomplete PRs in your final application, and you can continue to work on those PRs.

What happens to your PR is up to the maintainers of the tool you're modifying or testing. Sometimes PRs (or parts of PRs) don't get accepted for good reasons, and that's ok. Sometimes there are review delays because reviewers are busy, and that's also ok.

Brace1000 · 2026-04-16T17:51:28Z

Hi @teor2345 ,
I still have an unmerged PR I’ve been working on. Could you please guide me on how I should continue improving it or how best to proceed?
Thank you for your help

teor2345 · 2026-04-16T23:00:52Z

The next step is a review by someone with Rust merge rights.

Please be patient, it is normal for reviewers to take 2 weeks or more to review your PR. If it has been more than 2 weeks, you can tag the assigned reviewer, or re-assign another review using the instructions at the top of this PR.

Brace1000 · 2026-04-17T02:32:31Z

Got it, thank you for the clarification. I’ll be patient and follow up if needed after some time.
I really appreciate your guidance throughout.it has been an exciting experience journey with you

rustbot assigned dingxiangfei2009 Apr 9, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 9, 2026

This comment has been minimized.

Sign in to view

This was referenced Apr 9, 2026

Goal for resolving cargo-semver-checks blockers for merging into cargo. rust-lang/rust-project-goals#53

Merged

Outreachy tracking issue for Rust whitespace check bugs rustfoundation/interop-initiative#53

Closed

This comment has been minimized.

Sign in to view

Brace1000 commented Apr 9, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

teor2345 reviewed Apr 10, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

rustbot added has-merge-commits PR has merge commits, merge with caution. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 11, 2026

This comment has been minimized.

Sign in to view

Brace1000 added 8 commits April 11, 2026 15:50

tests: add ignore-tidy-tab directive to whitespace tests

1609baf

tests: expand vertical tab lexer test to cover all Pattern_White_Spac…

7c00dc2

…e chars

tests: add whitespace/ README entry explaining lexer vs stdlib mismatch

1546b1d

Update ascii_whitespace_excludes_vertical_tab.rs

5b487fc

Update ascii_whitespace_excludes_vertical_tab.rs

91fd464

make sure tabs and spaces are well checked

Update ascii_whitespace_excludes_vertical_tab.rs

874a618

fix tidy: add whitespace README entry

b727680

Improve error message for invalid whitespace

2506ce4

Updated error message to clarify invisible characters.

This comment has been minimized.

Sign in to view

Improve error handling for invisible characters

f1eb5e7

Updated error message for invisible characters in code.

This comment has been minimized.

Sign in to view

Document error for unknown token due to whitespace

16b2655

Add error message for invalid whitespace in code

This comment has been minimized.

Sign in to view

Update error message for invalid whitespace handling

701bc97

This comment has been minimized.

Sign in to view

Modify invalid_whitespace.rs for whitespace checks

ece7316

Updated the test to check for invalid whitespace handling.

This comment has been minimized.

Sign in to view

Brace1000 added 2 commits April 14, 2026 23:01

Correct whitespace in variable declaration

a1eb231

Fix formatting issue by adding space around '=' in variable declaration.

Update error message for invalid whitespace

6e459b9

This comment has been minimized.

Sign in to view

Brace1000 added 2 commits April 14, 2026 23:49

Update invalid_whitespace.stderr

dc0d44a

Refine error handling for invalid whitespace test

5661524

Update the error messages for invalid whitespace in the test.

This comment has been minimized.

Sign in to view

Update invalid_whitespace.rs

523f70a

This comment has been minimized.

Sign in to view

Fix whitespace issues in invalid_whitespace.rs

1db9763

This comment has been minimized.

Sign in to view

teor2345 reviewed Apr 15, 2026

View reviewed changes

Brace1000 added 2 commits April 15, 2026 09:01

Update invalid_whitespace.stderr file

52225e6

Clean up whitespace in invalid_whitespace.rs

185a582

Removed unnecessary blank lines from the test file.

This comment has been minimized.

Sign in to view

Update invalid_whitespace.stderr

43f045c

		@@ -0,0 +1,22 @@
		// This test checks that split_ascii_whitespace does NOT split on


		Tests on `where` clauses. See [Where clauses \| Reference](https://doc.rust-lang.org/reference/items/generics.html#where-clauses).

		## `whitespace`

Uh oh!

Conversation

Brace1000 commented Apr 9, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Apr 9, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Brace1000 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

teor2345 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Brace1000 Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teor2345 Apr 10, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

teor2345 left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Brace1000 commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

teor2345 commented Apr 15, 2026

Uh oh!

Brace1000 commented Apr 16, 2026

Uh oh!

teor2345 commented Apr 16, 2026

Uh oh!

Brace1000 commented Apr 17, 2026

Uh oh!

Brace1000 commented Apr 9, 2026 •

edited by rustbot

Loading

Brace1000 left a comment •

edited by rustbot

Loading

teor2345 left a comment •

edited by rustbot

Loading

Brace1000 Apr 11, 2026 •

edited

Loading

teor2345 Apr 10, 2026 •

edited by rustbot

Loading

teor2345 left a comment •

edited by rustbot

Loading

Brace1000 commented Apr 15, 2026 •

edited

Loading