From 162c09ec41bc23dc8fac2ed6adfa832bb12b93f6 Mon Sep 17 00:00:00 2001 From: Maxi Wittich Date: Fri, 13 Feb 2026 11:00:58 +0100 Subject: [PATCH 1/5] Add azure adls to troubleshooting guide --- .../airflow/pages/troubleshooting/index.adoc | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/modules/airflow/pages/troubleshooting/index.adoc b/docs/modules/airflow/pages/troubleshooting/index.adoc index 8403872f..47c9f97a 100644 --- a/docs/modules/airflow/pages/troubleshooting/index.adoc +++ b/docs/modules/airflow/pages/troubleshooting/index.adoc @@ -1,5 +1,53 @@ = Troubleshooting +== Azure Blob Storage Logging: + +Azure's `ADLS` can be used to store task logs Airflow. + +Assume a regular storage container in Azures ADLS backend, this can be accessed with either the `adls[s]` or `wasb` connector using the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/adls_v2.html[Azure Data Lake Storage Gen2 Connection] or the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html[Microsoft Azure Blob Storage Connection] respectively. + +Using `ADLS` as a task log backend, requires to access it via `wasb` and thus the configuration in the environment should look like: +[source,yaml] +---- + webservers: + envOverrides: &logging_overrides + AIRFLOW__AZURE_REMOTE_LOGGING__REMOTE_WASB_LOG_CONTAINER: "" #<1> + AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER: "wasb-" #<2> + AIRFLOW__LOGGING__REMOTE_LOGGING: "True" + AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID: "" #<3> + triggerers: + envOverrides: *logging_overrides + kubernetesExecutors: + envOverrides: *logging_overrides + schedulers: + envOverrides: *logging_overrides +---- +<1> This env var is only used for wasb connections. +<2> Note that the is *not* referenced. +<3> This connection can be defined in the AirflowUI or constructed as env variable. + +Due to this open https://github.com/apache/airflow/issues/58946[issue] with Airflow, it's recommended to use `wasb-` rather then `wasb://` as it would cause your backend looking like: +[source,text] +---- + + └── wasb:/ + └── tasklogs/ + └── dag_id=... +---- +However the workaround will result in +[source,text] +---- + + └── wasb-tasklogs/ + └── dag_id=... +---- + +The `Azure Blob Storage Connection` will offer the optional field `Host` which should have the a value looking like +[source,text] +---- +https://..blob.core.windows.net +---- + == S3 Logging: An error occurred (411) when calling the PutObject operation: Length Required If Airflow is trying to access S3 (e.g. for remote task logging) and throws the following error From 5b8109b8c534426dd51b7ba8defd539992207457 Mon Sep 17 00:00:00 2001 From: Maxi Wittich Date: Fri, 13 Feb 2026 11:10:01 +0100 Subject: [PATCH 2/5] Removing trailing whitespace --- docs/modules/airflow/pages/troubleshooting/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/airflow/pages/troubleshooting/index.adoc b/docs/modules/airflow/pages/troubleshooting/index.adoc index 47c9f97a..2ef2c76e 100644 --- a/docs/modules/airflow/pages/troubleshooting/index.adoc +++ b/docs/modules/airflow/pages/troubleshooting/index.adoc @@ -1,6 +1,6 @@ = Troubleshooting -== Azure Blob Storage Logging: +== Azure Blob Storage Logging: Azure's `ADLS` can be used to store task logs Airflow. From 948f46d556b4b2005edf54d6ffd0e9f2a063c9d1 Mon Sep 17 00:00:00 2001 From: Maxi Wittich Date: Fri, 13 Feb 2026 12:28:43 +0100 Subject: [PATCH 3/5] Fixing typos --- docs/modules/airflow/pages/troubleshooting/index.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/airflow/pages/troubleshooting/index.adoc b/docs/modules/airflow/pages/troubleshooting/index.adoc index 2ef2c76e..79cf7cb8 100644 --- a/docs/modules/airflow/pages/troubleshooting/index.adoc +++ b/docs/modules/airflow/pages/troubleshooting/index.adoc @@ -2,7 +2,7 @@ == Azure Blob Storage Logging: -Azure's `ADLS` can be used to store task logs Airflow. +Azure's `ADLS` can be used to store Airflow task logs. Assume a regular storage container in Azures ADLS backend, this can be accessed with either the `adls[s]` or `wasb` connector using the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/adls_v2.html[Azure Data Lake Storage Gen2 Connection] or the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html[Microsoft Azure Blob Storage Connection] respectively. @@ -45,7 +45,7 @@ However the workaround will result in The `Azure Blob Storage Connection` will offer the optional field `Host` which should have the a value looking like [source,text] ---- -https://..blob.core.windows.net +https://.blob.core.windows.net ---- == S3 Logging: An error occurred (411) when calling the PutObject operation: Length Required From 8bc44d7bf31c6a11f2b021b727d59d1164d16a67 Mon Sep 17 00:00:00 2001 From: Maximilian Wittich <56642549+Maleware@users.noreply.github.com> Date: Fri, 13 Feb 2026 14:07:07 +0100 Subject: [PATCH 4/5] Applying suggestions Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com> --- docs/modules/airflow/pages/troubleshooting/index.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/modules/airflow/pages/troubleshooting/index.adoc b/docs/modules/airflow/pages/troubleshooting/index.adoc index 79cf7cb8..172fb99f 100644 --- a/docs/modules/airflow/pages/troubleshooting/index.adoc +++ b/docs/modules/airflow/pages/troubleshooting/index.adoc @@ -1,12 +1,12 @@ = Troubleshooting -== Azure Blob Storage Logging: +== Azure Blob Storage Logging Azure's `ADLS` can be used to store Airflow task logs. -Assume a regular storage container in Azures ADLS backend, this can be accessed with either the `adls[s]` or `wasb` connector using the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/adls_v2.html[Azure Data Lake Storage Gen2 Connection] or the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html[Microsoft Azure Blob Storage Connection] respectively. +Assume a regular storage container in Azures ADLS backend: this can be accessed with either the `adls[s]` or `wasb` connector using the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/adls_v2.html[Azure Data Lake Storage Gen2 Connection] or the https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html[Microsoft Azure Blob Storage Connection] respectively. -Using `ADLS` as a task log backend, requires to access it via `wasb` and thus the configuration in the environment should look like: +If `ADLS` is used as a task log backend it must be accessed via `wasb` and thus the configuration in the environment should look like: [source,yaml] ---- webservers: @@ -24,9 +24,9 @@ Using `ADLS` as a task log backend, requires to access it via `wasb` and thus th ---- <1> This env var is only used for wasb connections. <2> Note that the is *not* referenced. -<3> This connection can be defined in the AirflowUI or constructed as env variable. +<3> This connection can be defined in the AirflowUI or declared as an environment variable. -Due to this open https://github.com/apache/airflow/issues/58946[issue] with Airflow, it's recommended to use `wasb-` rather then `wasb://` as it would cause your backend looking like: +Due to this open https://github.com/apache/airflow/issues/58946[issue] with Airflow, it's recommended to use `wasb-` rather then `wasb://` as using the latter option would assume the target location looks like this: [source,text] ---- From ccbcee02efdf92318cfa69a6795ff123e0c0c236 Mon Sep 17 00:00:00 2001 From: Maxi Wittich Date: Fri, 13 Feb 2026 14:09:28 +0100 Subject: [PATCH 5/5] fixing wording --- docs/modules/airflow/pages/troubleshooting/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/airflow/pages/troubleshooting/index.adoc b/docs/modules/airflow/pages/troubleshooting/index.adoc index 172fb99f..afd54f1c 100644 --- a/docs/modules/airflow/pages/troubleshooting/index.adoc +++ b/docs/modules/airflow/pages/troubleshooting/index.adoc @@ -42,7 +42,7 @@ However the workaround will result in └── dag_id=... ---- -The `Azure Blob Storage Connection` will offer the optional field `Host` which should have the a value looking like +The `Azure Blob Storage Connection` will offer the optional field `Host` which should have a value looking like this: [source,text] ---- https://.blob.core.windows.net