UI Automation in Microsoft Word: Use move by sentence custom pattern when available instead of the legacy model #19367
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds support for UI Automation remote operations to enable sentence navigation in Microsoft Word when the legacy object model is unavailable. The implementation uses Word's custom UIA extension patterns (MoveBySentence and ExpandToEnclosingSentence) when available on Windows 11 with remote operations support, falling back to the existing legacy object model approach otherwise.
- Adds a new function
msWord_moveTextRangeBySentenceinUIAHandler/remote.pyto handle sentence navigation via UIA remote operations - Refactors
_caretMoveBySentenceHelperin Word document handling to prefer UIA remote operations, with legacy fallback - Extracts common pattern retrieval logic into a reusable helper function
_msWord_remote_getExtendedTextRangePattern
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| user_docs/en/changes.md | Adds changelog entry documenting the new sentence navigation capability in Word with UIA on Windows 11 |
| source/UIAHandler/remote.py | Implements msWord_moveTextRangeBySentence function using Word UIA custom patterns and extracts _msWord_remote_getExtendedTextRangePattern helper function |
| source/NVDAObjects/UIA/wordDocument.py | Updates _caretMoveBySentenceHelper to attempt UIA remote sentence navigation first before falling back to legacy object model |
Comments suppressed due to low confidence (2)
source/NVDAObjects/UIA/wordDocument.py:675
- The
exc_info=Trueparameter is redundant when usinglog.exception(). Thelog.exception()method automatically includes exception information, so explicitly passingexc_info=Trueis unnecessary. Consider removing this parameter.
TextInfo = WordDocumentTextInfo
def _get_mathMl(self):
try:
source/NVDAObjects/UIA/wordDocument.py:653
- Typo in comment: "overide" should be "override".
pos,
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
codeofdusk
force-pushed
the
t13517
branch
7 times, most recently
from
f82061b to
150d7ff
Compare
SaschaCowley
assigned
seanbudd and
michaelDCurran and unassigned
seanbudd and
michaelDCurran
seanbudd
added
the
conceptApproved
|
|
||
| ### Changes | ||
|
|
||
| * In Microsoft Word with UI Automation on Windows 11, NVDA can navigate by sentence in environments where the legacy object model is unavailable. (#13517, @codeofdusk) |
There was a problem hiding this comment.
It seems like there is no real user impact currently, I don't think this needs a changelog entry. Perhaps we could say there are performance improvements in newer versions of Word, if this is noticeable?
There was a problem hiding this comment.
There are environments where we can't use the object model:
- AppX builds, if they ever come back
- WDAG/RAIL (I'd think)
- Maybe Windows Sandbox
- etc.
There was a problem hiding this comment.
can you confirm the issue with any of these environments, so we can list them explicitly?
There was a problem hiding this comment.
I'm not sure how (and I think it might require licences I don't have).
There was a problem hiding this comment.
since we don't have anything confirmed here, lets remove it
| * In Microsoft Word with UI Automation on Windows 11, NVDA can navigate by sentence in environments where the legacy object model is unavailable. (#13517, @codeofdusk) |
There was a problem hiding this comment.
I do think this is worth documenting somewhere, at very least if behaviour changes (for the better or for the worse) in some corner case users might have an idea as to why even if they don't read the commit/PR log.
There was a problem hiding this comment.
we only document changes that affect end users or developers. If the behaviour changes unexpectedly/negatively an issue can be opened and we can investigate. I don't think the changelog would be useful for a non-technical user debugging here, and a technical user should be checking commit logs.
|
@codeofdusk, it may be interesting to test #9002, in case this PR changes something or even fixes it. |
| # Using the legacy object model, | ||
| # Move the caret to the next sentence in the requested direction. | ||
| legacyInfo = LegacyWordDocumentTextInfo(self, textInfos.POSITION_CARET) | ||
| legacyInfo.move(textInfos.UNIT_SENTENCE, direction) | ||
| # Save the start of the sentence for future use | ||
| legacyStart = legacyInfo.copy() | ||
| legacyInfo.move(textInfos.UNIT_SENTENCE, 1) | ||
| # Fetch the caret position (end of the next sentence) with UI automation. | ||
| endInfo = self.makeTextInfo(textInfos.POSITION_CARET) | ||
| # Move the caret back to the start of the next sentence, | ||
| # where it should be left for the user. | ||
| legacyStart.updateCaret() | ||
| # Fetch the new caret position (start of the next sentence) with UI Automation. | ||
| startInfo = self.makeTextInfo(textInfos.POSITION_CARET) | ||
| # Make a UI automation text range spanning the entire next sentence. | ||
| info = startInfo.copy() | ||
| info.end = endInfo.end |
There was a problem hiding this comment.
can we move this to a helper function? e.g. getLegacyTextInfo(self, direction: int) -> LegacyWordDocumentTextInfo
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
5 participants
Link to issue number:
Closes #13517.
Summary of the issue:
NVDA exclusively uses Word's legacy object model for sentence navigation, even when UIA is enabled.
Description of how this pull request fixes the issue:
Added support for the UIA custom extension to move and expand by sentence in supported scenarios. The legacy implementation remains as a fallback on systems without remote ops support (Windows below 11/Cobalt platform) or older Office versions.
Testing strategy:
Moved by sentence in a large document and verified functionality.
Known issues with pull request:
None known
Code Review Checklist: