[Fixes #13641] Multilang: index - fix

[etj]

Fixes index search with non-latin chars

Checklist

Reviewing is a process done by project maintainers, mostly on a volunteer basis. We try to keep the overhead as small as possible and appreciate if you help us to do so by completing the following items. Feel free to ask in a comment if you have troubles with any of them.

For all pull requests:

• Confirm you have read the contribution guidelines
• You have sent a Contribution Licence Agreement (CLA) as necessary (not required for small changes, e.g., fixing typos in the documentation)
• Make sure the first PR targets the master branch, eventual backports will be managed later. This can be ignored if the PR is fixing an issue that only happens in a specific branch, but not in newer ones.

The following are required only for core and extension modules (they are welcomed, but not required, for contrib modules):

• There is a ticket in https://github.com/GeoNode/geonode/issues describing the issue/improvement/feature (a notable exemption is, changes not visible to end-users)
• The issue connected to the PR must have Labels and Milestone assigned
• PR for bug fixes and small new features are presented as a single commit
• PR title must be in the form "[Fixes #<issue_number>] Title of the PR"
• New unit tests have been added covering the changes, unless there is an explanation on why the tests are not necessary/implemented

Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.

[gemini-code-assist]

Summary of Changes

Hello @etj, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical bug affecting the index search functionality, specifically when dealing with non-Latin characters. By refining the search string sanitization logic to properly recognize and process a broader spectrum of Unicode characters, the change significantly improves the system's ability to perform accurate and comprehensive searches across various languages. This ensures that users can effectively search for content regardless of the character set used in their queries.

Highlights

• Improved Multilingual Search: The pull request addresses an issue where the index search failed to process queries containing non-Latin characters, enhancing the system's multilingual capabilities.
• Regular Expression Update: The regular expression used for sanitizing search strings has been updated to correctly handle a wider range of Unicode characters, including non-Latin alphabets.
• Unicode Flag Implementation: The re.UNICODE flag has been added to the re.sub function call, ensuring that word (\w) and whitespace (\s) character classes in the regex are interpreted according to the Unicode standard.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses an issue with searching using non-latin characters by improving the input sanitization regex. The change is a good step forward, correctly using \w with the re.UNICODE flag to support a wider range of characters. My review includes a suggestion to further enhance the robustness of the search functionality by handling whitespace-only or padded search strings.

[gemini-code-assist]

After sanitizing the search string, it's possible for it to contain only whitespace, or have leading/trailing whitespace. This could lead to queries like ' ':* which are valid but likely not what the user intends. It's good practice to strip() the value to handle these cases gracefully. An all-whitespace search would then correctly fall back to '*:*', which is consistent with an empty search. You can chain .strip() to the re.sub() call for a concise solution.

Suggested change

	value = re.sub(r"[^\w\s./\-]+", "", value, flags=re.UNICODE)
	value = re.sub(r"[^\w\s./\-]+", "", value, flags=re.UNICODE).strip()

[codecov]

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 74.13%. Comparing base (7c402fd) to head (d6df4e9).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #13786   +/-   ##
=======================================
  Coverage   74.13%   74.13%           
=======================================
  Files         944      944           
  Lines       56291    56307   +16     
  Branches     7617     7624    +7     
=======================================
+ Hits        41730    41742   +12     
  Misses      12897    12897           
- Partials     1664     1668    +4     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.