Skip to content

fix: stabilize test_text_query_word_weights by using component assertions#524

Open
thakoreh wants to merge 1 commit intoredis:mainfrom
thakoreh:fix/stabilize-text-query-word-weights
Open

fix: stabilize test_text_query_word_weights by using component assertions#524
thakoreh wants to merge 1 commit intoredis:mainfrom
thakoreh:fix/stabilize-text-query-word-weights

Conversation

@thakoreh
Copy link

@thakoreh thakoreh commented Mar 3, 2026

Summary

Problem

The test was flaky because it relied on exact string matching with specific token ordering. Dict iteration order could cause tokens to appear in different positions across Python versions.

Solution

Check for expected query components individually rather than matching the entire string exactly:

# Before (flaky)
assert str(query) == "@description:(query | string | alpha=>{$weight:2} | ...) LIMIT 0 10"

# After (stable)
query_str = str(query)
assert "@description:(" in query_str
assert "alpha=>{$weight:2}" in query_str
assert "SCORER BM25STD" in query_str
# ... etc

Testing

  • Test now passes consistently regardless of dict ordering
  • All existing test assertions preserved (value checks, error cases)

Note

Low Risk
Low risk: only modifies unit test assertions to be less order-dependent, with no production code changes.

Overview
Re-enables test_text_query_word_weights by removing the skip marker and replacing an exact full query-string equality check with component-based in assertions.

This makes the test resilient to non-deterministic token ordering (e.g., dict iteration) while still verifying weighted tokens and key query options (scorer, dialect, limit) are present.

Written by Cursor Bugbot for commit c9f9a0d. This will update automatically on new commits. Configure here.

…ions

The test was flaky because it relied on exact string matching with
specific token ordering. Dict iteration order could cause tokens to
appear in different positions across Python versions.

Fix: Replace exact string match with component-based assertions that
check for expected query parts without relying on ordering.

Closes redis#497
Copilot AI review requested due to automatic review settings March 3, 2026 05:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes the previously flaky test_text_query_word_weights unit test by removing an exact full-string assertion and re-enabling the test in CI (fix for #497).

Changes:

  • Removed the @pytest.mark.skip decorator from test_text_query_word_weights.
  • Replaced an exact str(query) == ... assertion with component-based substring assertions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +346 to +354
# Check query components without relying on exact token ordering
query_str = str(query)
assert "@description:(" in query_str
assert "alpha=>{$weight:2}" in query_str
assert "delta=>{$weight:0.555}" in query_str
assert "query" in query_str
assert "string" in query_str
assert "bravo" in query_str
assert "tango" in query_str
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These substring assertions are much less strict than the previous exact-match and could pass even if key semantics regress (e.g., only one of the two alpha occurrences is weighted, or the @description:(...) clause is malformed). Consider adding stable-but-strong checks like counting weighted occurrences (e.g., alpha appears twice), and asserting structure (e.g., startswith @description:( and that the text clause closes before SCORER).

Suggested change
# Check query components without relying on exact token ordering
query_str = str(query)
assert "@description:(" in query_str
assert "alpha=>{$weight:2}" in query_str
assert "delta=>{$weight:0.555}" in query_str
assert "query" in query_str
assert "string" in query_str
assert "bravo" in query_str
assert "tango" in query_str
# Check query components without relying on exact token ordering,
# but with stronger structural guarantees.
query_str = str(query)
# The description clause should be present and properly delimited
assert "@description:(" in query_str
description_start = query_str.index("@description:(")
description_close = query_str.index(")", description_start)
scorer_pos = query_str.index("SCORER")
# Ensure the text clause closes before the SCORER section
assert description_close < scorer_pos
# The weighted tokens should appear the expected number of times
assert query_str.count("alpha=>{$weight:2}") == 2
assert query_str.count("delta=>{$weight:0.555}") == 1
# Ensure the main text tokens appear inside the description clause
description_clause = query_str[description_start:description_close]
assert "query" in description_clause
assert "string" in description_clause
assert "bravo" in description_clause
assert "tango" in description_clause
# Other expected query modifiers

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix flaky test_text_query_word_weights and re-enable test

2 participants