Don't decode YAML responses before parsing them in get_artifact #700
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Taskcluster sends those without a content-type which means that requests assumes they're latin1 text, which they might not be as taskgraph uploads its YAMLs encoded as UTF-8. While decoding UTF-8 as latin1 works most of the time, if the commit message passed in a parameters.yml contains a special character, then it will return a garbled mess which the yaml library will reject. By passing the raw bytes instead of decoding them, we sidestep the problem entirely, delegating the decoding to the YAML library which supports both utf-8 and utf-16s. From the documentation for `yaml.load` > A byte string or a file must be encoded with utf-8, utf-16-be or > utf-16-le encoding. yaml.load detects the encoding by checking the BOM > (byte order mark) sequence at the beginning of the string/file. If no > BOM is present, the utf-8 encoding is assumed.
Eijebong
force-pushed
the
dont-decode-yaml-content-ourselves
branch
from
0f72e01 to
75ff3b7
Compare
jcristau
approved these changes
Jun 2, 2025
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Taskcluster sends those without a content-type which means that requests assumes they're latin1 text, which they might not be as taskgraph uploads its YAMLs encoded as UTF-8. While decoding UTF-8 as latin1 works most of the time, if the commit message passed in a parameters.yml contains a special character, then it will return a garbled mess which the yaml library will reject.
By passing the raw bytes instead of decoding them, we sidestep the problem entirely, delegating the decoding to the YAML library which supports both utf-8 and utf-16s.
From the documentation for
yaml.load