Skip to content

Commit bfe02aa

Browse files
sbernaueradwk67
andauthored
docs: Add troubleshooting guide on new S3 checksums (#684)
* docs: Add troubleshooting guide on new S3 checksums * Apply suggestions from code review Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com> --------- Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com>
1 parent 1133eb3 commit bfe02aa

File tree

2 files changed

+28
-0
lines changed

2 files changed

+28
-0
lines changed
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
= Troubleshooting
2+
3+
== S3 Logging: An error occurred (411) when calling the PutObject operation: Length Required
4+
5+
If Airflow is trying to access S3 (e.g. for remote task logging) and throws the following error
6+
7+
[source,text]
8+
----
9+
{s3_task_handler.py:190} ERROR - Could not write logs to s3://***/***-task-logs/dag_id=test/run_id=manual__2025-05-22T08:14:17.917519+00:00/task_id=extract/attempt=1.log
10+
Traceback (most recent call last):
11+
[..]
12+
botocore.exceptions.ClientError: An error occurred (411) when calling the PutObject operation: Length Required
13+
----
14+
15+
then this is an indication that your S3 store doesn't support the new AWS S3 checksums.
16+
Ideally you should upgrade your S3 store to a more recent version that supports the new checksums (if there is any).
17+
If this is not possible, you can work around this by setting the following env vars
18+
19+
[source,yaml]
20+
----
21+
webservers: # Also add to other roles!
22+
envOverrides:
23+
AWS_REQUEST_CHECKSUM_CALCULATION: when_required
24+
AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
25+
----
26+
27+
See e.g. https://github.com/minio/minio/issues/20845[this MinIO issue] for details.

docs/modules/airflow/partials/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
*** xref:airflow:usage-guide/operations/pod-placement.adoc[]
1919
*** xref:airflow:usage-guide/operations/pod-disruptions.adoc[]
2020
*** xref:airflow:usage-guide/operations/graceful-shutdown.adoc[]
21+
* xref:nifi:troubleshooting/index.adoc[]
2122
* xref:airflow:reference/index.adoc[]
2223
** xref:airflow:reference/crds.adoc[]
2324
*** {crd-docs}/airflow.stackable.tech/airflowcluster/v1alpha1/[AirflowCluster {external-link-icon}^]

0 commit comments

Comments
 (0)