Improved Input Help Mode to Report Characters Typed by Key Combinations in Normal Mode

[cary-rowen]

Link to issue number:

Fixes #6621

Summary of the issue:

In input help mode, NVDA only reports the key combination name but does not report what character would be typed by that key combination in normal mode.

Description of user facing changes:

Input help mode now reports the key combination first, and then, when appropriate, the character that key combination would type in normal input mode.

• For example, pressing Shift+1 on a US keyboard will report shift+1, then !
• For key combinations that produce a printable character, input help can now additionally report that character when it provides extra information
• If the key combination is bound to a command or script, the existing behavior is preserved, i.e. the command description is reported rather than an extra typed character
• Differences that are only in case are now reported correctly as well, for example uppercase letters

Description of developer facing changes:

• Added an inputHelpCharacter auto property to the InputGesture base class in inputCore.py, provided by _get_inputHelpCharacter
• KeyboardInputGesture in keyboardHandler.py uses ToUnicodeEx to infer the character the gesture would produce in normal input mode
• KeyboardInputGesture overrides _get_inputHelpCharacter to expose that extra character to input help when appropriate
• _handleInputHelp now handles the gesture display name, gesture.inputHelpCharacter, and script description separately

Description of development approach:

Based on the abandoned PR #15907(by @Emil-18) and #17629. The implementation uses ToUnicodeEx with the 0x04 flag to get the character without modifying keyboard state. Dead key results are preserved for input help handling. Input help reports the gesture name first, and only adds the character when it provides additional information.

Testing strategy:

Manual testing in input help mode with different keyboard layouts and key combinations, including printable character combinations, dead keys, command-bound gestures, and case-only differences.

Known issues with pull request:

as yet unknown

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[cary-rowen]

cc @LeonarddeR @CyrilleB79
Thank you for your guidance and valuable feedback in #17629. I would like to continue working on this now.
Regarding the French layout, I have only performed simple testing so far, so more testing might be needed. Most importantly, I want to confirm what the expected behavior for dead key characters is after this change.

[Emil-18]

@cary-rowen Thank you for continuing this. I don't think ToUnicodeEx modifies the keyboard state at all when the 0x04 flag is used, so if the user types a dead key in input help mode, it shouldn't affect what is typed if the user exits input help

[LeonarddeR]

I have tested this from source but it doesn't seem to work here:
keyboardHandler.KeyboardInputGesture.fromName("shift+2")._nameForInputHelp
This just gives me shift+2. The character property returns an empty string.

Furthermore, how will the current code behave when symbol level is set to None?

[CyrilleB79]

I have done first tests, to be continued. I can already write the following:

1. The order is not consistent, what makes the output confusing. Sometimes you have the key combination first and then what it does, and other times you have what is typed first and then the key combination. E.g.:

   • Pressing NVDA+T, you get ['NVDA+t'] and then ['Reports the title of the current application or foreground window. If pressed twice, spells the title. If pressed three times, copies the title to the clipboard']
   • Pressing shift+$ (French keyboard), you get ['£ majuscule+$']. Note: on fr-fr keyboard, $ has its own key and if it is shifted, you get £.

   It would be better if you had something like "majuscule+$ Type £"

2. With standard input help reporting, e.g. NVDA+T, you have two speech utterances (2 lines in the log). This is better to separate the key name from what it types / produces. Else, it is sometimes confusing, e.g.: "+ majuscule+=" (or switched if you take into account point 1.)

So still better: "majuscule+$", "Type £".

3. With dead keys, I cannot reproduce the issue mentioned in "Known isssues" section. Can you still reproduce it?

---

I understand that inverting the key combination and its result and adding a pause between them, you get the result later, what may be undesirable, but that's needed for clarity IMO.

Alternatively, if someone really needs to explore many keys and is bothered by the time the produced character is reported, maybe a separate dry-typing feature would be more suitable.

[LeonarddeR]

@CyrilleB79 wrote:

1. The order is not consistent, what makes the output confusing. 

Wholeheartedly agree here. I think we should stick to the key sequence, a short pause, and then the character it would produce.

2. With standard input help reporting, e.g. `NVDA+T`, you have two speech utterances (2 lines in the log). This is better to separate the key name from what it types / produces.

This suggests a restructure of the current code is necessary:

1. _get__nameForInputHelp should become a separate property that returns the additional message to be reported when in input help. I think it makes sense to make this return a speech sequence. This ensures maximum flexibility in the future, i.e. you can spell something but also report an additional message apart from the scripts description.
2. InputManager._handleInputHelp can then speak this sequence.

Since the current implementation doesn't work here anyway, we might need some logging here and there. Also avoid to return empty strings, rather use None to communicate that nothing is returned.

[Emil-18]

@LeonarddeR Why wouldn't you use an empty string? I think always using the same datatype is a good practice. What if someone uses a function that only works with strings, and doesn't know that the function Occasionally returns None? Plus, returning an empty string is basically the same as returning nothing anyway

[seanbudd]

Please note #15891 is already closed and #6621 was untriaged. We've triaged #6621 now, so this PR's conceptually approved

[LeonarddeR]

@LeonarddeR Why wouldn't you use an empty string? I think always using the same datatype is a good practice.

It depends on the programming language and on what you're trying to communicate. For example in C#, a language I use very often, a string is a reference type and therefore can be null.

What if someone uses a function that only works with strings, and doesn't know that the function Occasionally returns None?

That's not applicable here. We're talking about a new property that will have a proper return annotation.

Plus, returning an empty string is basically the same as returning nothing anyway

That's not true IMO. When getting a character, None tells me that there is no character to report.
There is actually a pretty good explainer here.

[Copilot]

Pull request overview

This pull request enhances NVDA's input help mode to report the actual character that would be typed by a key combination, in addition to the key combination name itself. This addresses a long-standing user experience issue where pressing Shift+1 would only report "shift+1" rather than the exclamation mark "!" that would actually be typed.

Changes:

• Added character resolution capability using Windows ToUnicodeEx API with 0x04 flag to avoid modifying keyboard state
• Dead keys are now reported immediately with their character representation
• NVDA commands (containing NVDA modifier) retain their original behavior

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
user_docs/en/changes.md	Documents the new input help mode behavior for users
source/winUser.py	Adds MAPVK constants needed for virtual key to scan code conversion
source/keyboardHandler.py	Implements character property using ToUnicodeEx API and updates _nameForInputHelp to include characters; updates fromName to calculate scanCode
source/inputCore.py	Adds base _nameForInputHelp property and updates _handleInputHelp to handle multiple name items with proper speech output

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[CyrilleB79]

This point in #19422 (comment) has not been addressed:

1. The order is not consistent, what makes the output confusing. Sometimes you have the key combination first and then what it does, and other times you have what is typed first and then the key combination.

IMO addressing it is mandatory.

Point 2. has not been addressed either. You may choose not to address it, but it would be nice to hear your position on it, i.e. why do you choose to address it or not. Thanks.

[seanbudd]

@cary-rowen - do you intend to continue working on this?

[cary-rowen]

Hi @seanbudd and @CyrilleB79
I checked the diff and it looks like I missed a few commits, but I'll take a look at it this weekend.

Sorry for the delay!

[cary-rowen]

@CyrilleB79, could you please take a quick look? I might not have reached the level you're fully satisfied with yet, but it would be great if you could just give it a simple try. Capital letters will now also be reported in a configurable way.

[CyrilleB79]

@cary-rowen I have tested and almost all seems OK. I have tested the following cases:

• order is consistent, regardless if the gesture is an NVDA command or not
• dead keys are reported and there is no side effect
• number keys row on French keyboard is well reported (with and without shift)

With the numpad, there is an issue left, e.g. numpadPlus:

IO - inputCore.InputManager.executeGesture (09:53:22.906) - winInputHook (2344):
Input: kb(desktop):numpadPlus
INFO - inputCore.InputManager._handleInputHelp (09:53:22.908) - MainThread (5508):
Input help: gesture kb(desktop):numpadPlus, bound to script review_sayAll on globalCommands.GlobalCommands
IO - speech.speech.speak (09:53:22.908) - MainThread (5508):
Speaking ['pavNum plus']
IO - speech.speech.speak (09:53:22.909) - MainThread (5508):
Speaking ['pluss', EndUtteranceCommand()]
IO - speech.speech.speak (09:53:22.910) - MainThread (5508):
Speaking ["Lire à partir du curseur de revue jusqu'à la fin du texte en faisant suivre le curseur de revue"]

It reports 3 things instead of 2.

At last, in general, there are still 2 speech logged, not only one merged. Maybe this is acceptable though.

[cary-rowen]

Hi @CyrilleB79, thanks for your testing!

"With the numpad, there is an issue left, e.g. numpadPlus :"
This has been fixed.

At last, in general, there are still 2 speech logged, not only one merged. Maybe this is acceptable though.
I think this is probably acceptable.

To correctly report uppercase letters, I've added some additional logic. It might not look very elegant at the moment, so please test it again. If the behavior is correct, we can then refine the code itself.

Also, I’ve installed a French keyboard layout, but since I have no knowledge of French, it's difficult for me to test it thoroughly.

[seanbudd]

@cary-rowen - can you update the PR description? @CyrilleB79 - can you confirm testing on the latest changes?

[cary-rowen]

I updated the PR description.