You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Add troubleshooting guide on new S3 checksums (#684)
* docs: Add troubleshooting guide on new S3 checksums
* Apply suggestions from code review
Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com>
---------
Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com>
== S3 Logging: An error occurred (411) when calling the PutObject operation: Length Required
4
+
5
+
If Airflow is trying to access S3 (e.g. for remote task logging) and throws the following error
6
+
7
+
[source,text]
8
+
----
9
+
{s3_task_handler.py:190} ERROR - Could not write logs to s3://***/***-task-logs/dag_id=test/run_id=manual__2025-05-22T08:14:17.917519+00:00/task_id=extract/attempt=1.log
10
+
Traceback (most recent call last):
11
+
[..]
12
+
botocore.exceptions.ClientError: An error occurred (411) when calling the PutObject operation: Length Required
13
+
----
14
+
15
+
then this is an indication that your S3 store doesn't support the new AWS S3 checksums.
16
+
Ideally you should upgrade your S3 store to a more recent version that supports the new checksums (if there is any).
17
+
If this is not possible, you can work around this by setting the following env vars
18
+
19
+
[source,yaml]
20
+
----
21
+
webservers: # Also add to other roles!
22
+
envOverrides:
23
+
AWS_REQUEST_CHECKSUM_CALCULATION: when_required
24
+
AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
25
+
----
26
+
27
+
See e.g. https://github.com/minio/minio/issues/20845[this MinIO issue] for details.
0 commit comments