Skip to content

fix: add max_length support to Gzip/Brotli decoders for urllib3 2.6+#495

Merged
chalmerlowe merged 5 commits intogoogleapis:mainfrom
CalNightingale:fix/urllib3-2.6-decoder-compat
Feb 26, 2026
Merged

fix: add max_length support to Gzip/Brotli decoders for urllib3 2.6+#495
chalmerlowe merged 5 commits intogoogleapis:mainfrom
CalNightingale:fix/urllib3-2.6-decoder-compat

Conversation

@CalNightingale
Copy link
Contributor

Summary

Fixes #491. Supersedes #492 (stale).

urllib3 >= 2.6.0 now passes max_length to decoder.decompress() and accesses decoder.has_unconsumed_tail. Our custom _GzipDecoder and _BrotliDecoder wrappers didn't accept these, causing TypeError at runtime.

This PR:

  • Updates _GzipDecoder.decompress and _BrotliDecoder.decompress to accept and forward max_length, with a TypeError fallback for older urllib3
  • Adds has_unconsumed_tail property to _BrotliDecoder (the proxy class that needs it explicitly)
  • Applies the same fix to the async _GzipDecoder in google/_async_resumable_media/
  • Adds test coverage for the new max_length forwarding, the fallback path, and has_unconsumed_tail

Feedback from #492 addressed

  • max_length=-1 default (chandra-siri) — uses -1 to match urllib3's default, not None
  • has_unconsumed_tail only on _BrotliDecoder (chandra-siri) — _GzipDecoder inherits this from the parent urllib3.response.GzipDecoder, so no override is needed
  • Lint failure (BrennaEpp) — no trailing whitespace
  • Missing test coverage for fallback paths (BrennaEpp) — added tests for max_length forwarding, TypeError fallback, and has_unconsumed_tail (with AttributeError fallback for older urllib3)

Test plan

  • pytest tests/unit/requests/test_download.py — all pass
  • pytest tests_async/unit/requests/test_download.py — all pass
  • Full unit suite (514 tests) — all pass
  • No lint issues (flake8 clean)

🤖 Generated with Claude Code

@CalNightingale CalNightingale requested a review from a team as a code owner January 28, 2026 23:41
@CalNightingale CalNightingale requested a review from a team January 28, 2026 23:41
@google-cla
Copy link

google-cla bot commented Jan 28, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/google-resumable-media-python API. labels Jan 28, 2026
…compatibility

urllib3 2.6+ passes max_length to decoder.decompress(). Update
_GzipDecoder and _BrotliDecoder to accept and forward max_length,
with a TypeError fallback for older urllib3. Add has_unconsumed_tail
property to _BrotliDecoder (the proxy class that needs it; _GzipDecoder
inherits it from the parent).

Fixes googleapis#491
@CalNightingale CalNightingale force-pushed the fix/urllib3-2.6-decoder-compat branch from 11c51a5 to 2aa67b2 Compare January 28, 2026 23:44
@chalmerlowe chalmerlowe requested a review from a team as a code owner February 25, 2026 14:06
chalmerlowe added a commit that referenced this pull request Feb 25, 2026
Co-authored-by: chalmerlowe <7291104+chalmerlowe@users.noreply.github.com>
chalmerlowe added a commit that referenced this pull request Feb 25, 2026
Co-authored-by: chalmerlowe <7291104+chalmerlowe@users.noreply.github.com>
@chalmerlowe chalmerlowe added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@chalmerlowe chalmerlowe self-assigned this Feb 25, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@chalmerlowe chalmerlowe added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@chalmerlowe chalmerlowe added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
Copy link
Contributor

@chalmerlowe chalmerlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chalmerlowe chalmerlowe requested a review from a team as a code owner February 25, 2026 18:10
@chalmerlowe chalmerlowe added kokoro:force-run Add this label to force Kokoro to re-run the tests. automerge Merge the pull request once unit tests and other checks pass. labels Feb 25, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 25, 2026
@chalmerlowe chalmerlowe merged commit c74b95f into googleapis:main Feb 26, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the googleapis/google-resumable-media-python API. automerge Merge the pull request once unit tests and other checks pass. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_GzipDecoder.decompress() TypeError: unexpected keyword argument 'max_length'

5 participants