Skip to content

fix: Truncate large table samples in Teams alerts to prevent payload size issues #2104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
RamiNoodle733 wants to merge 1 commit into from
+39 −4
Closed

fix: Truncate large table samples in Teams alerts to prevent payload size issues #2104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions elementary/config/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ def __init__(
report_url: Optional[str] = None,
teams_webhook: Optional[str] = None,
maximum_columns_in_alert_samples: Optional[int] = None,
maximum_rows_in_alert_samples_table: Optional[int] = None,
env: str = DEFAULT_ENV,
run_dbt_deps_if_needed: Optional[bool] = None,
project_name: Optional[str] = None,
Expand Down Expand Up @@ -112,6 +113,12 @@ def __init__(
4,
)

self.maximum_rows_in_alert_samples_table = self._first_not_none(
maximum_rows_in_alert_samples_table,
config.get("maximum_rows_in_alert_samples_table"),
25,
)

self.timezone = self._first_not_none(
timezone,
config.get("timezone"),
Expand Down
24 changes: 21 additions & 3 deletions elementary/monitor/alerts/alert_messages/builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
class MessageBuilderConfig(BaseModel):
alert_groups_subscribers: bool = False
maximum_columns_in_alert_samples: int = 4
maximum_rows_in_alert_samples_table: int = 25


class AlertMessageBuilder:
Expand Down Expand Up @@ -394,9 +395,26 @@ def _get_result_blocks(
and len(result_sample[0].keys())
<= self.config.maximum_columns_in_alert_samples
):
result_blocks.append(
TableBlock.from_dicts(result_sample),
)
# Truncate table if too many rows to prevent Teams payload size issues
if len(result_sample) > self.config.maximum_rows_in_alert_samples_table:
truncated_sample = result_sample[:self.config.maximum_rows_in_alert_samples_table]
omitted_rows_count = len(result_sample) - self.config.maximum_rows_in_alert_samples_table
result_blocks.append(
TableBlock.from_dicts(truncated_sample),
)
result_blocks.append(
LinesBlock(
lines=[
ItalicTextLineBlock(
text=f"... and {omitted_rows_count} more rows (truncated to prevent payload size issues)"
),
]
)
)
else:
result_blocks.append(
TableBlock.from_dicts(result_sample),
)
Comment on lines +398 to +417
Copy link

@coderabbitai coderabbitai bot Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Consider guarding against non-positive maximum_rows_in_alert_samples_table values.

The truncation logic works correctly for positive values. However, if a user passes 0 or a negative value via CLI (e.g., --maximum-rows-in-alert-samples-table 0), the behavior degrades:

  • 0: result_sample[:0] → empty table, with a message saying "and N more rows" for all rows.
  • Negative: result_sample[-k:] silently drops rows from the start instead of limiting from the end, and omitted_rows_count becomes negative, producing a nonsensical message like "... and -5 more rows".

A simple guard (either in the config or here) would prevent this.

Proposed fix (in config or CLI layer)

Option A — Validate at the config level in config.py:

         self.maximum_rows_in_alert_samples_table = self._first_not_none(
             maximum_rows_in_alert_samples_table,
             config.get("maximum_rows_in_alert_samples_table"),
             25,
         )
+        if self.maximum_rows_in_alert_samples_table is not None and self.maximum_rows_in_alert_samples_table < 1:
+            raise InvalidArgumentsError(
+                "maximum_rows_in_alert_samples_table must be a positive integer."
+            )

Option B — Add a min constraint in the CLI option:

 `@click.option`(
     "--maximum-rows-in-alert-samples-table",
     "-mr",
     type=int,
     default=25,
+    callback=lambda ctx, param, value: value if value >= 1 else ctx.fail("must be >= 1"),
     help="Maximum number of rows to display in a table in alert samples.",
 )
🤖 Prompt for AI Agents 
In `@elementary/monitor/alerts/alert_messages/builder.py` around lines 398 - 417,
Guard against non-positive maximum_rows_in_alert_samples_table by normalizing it
before truncation: compute a safe_max = max(1,
self.config.maximum_rows_in_alert_samples_table) (or treat <=0 as "no
truncation" by using safe_max = None and skipping truncation), then use safe_max
when slicing result_sample and when computing omitted_rows_count = max(0,
len(result_sample) - safe_max); update the truncation branch that uses
result_sample[:self.config.maximum_rows_in_alert_samples_table],
omitted_rows_count, TableBlock.from_dicts, LinesBlock and ItalicTextLineBlock to
reference safe_max so you never produce an empty table or negative
omitted_rows_count for 0/negative config values.

else:
result_blocks.append(
JsonCodeBlock(content=result_sample),
Expand Down
9 changes: 9 additions & 0 deletions elementary/monitor/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,13 @@ def get_cli_properties() -> dict:
default=4,
help="Maximum number of columns to display as a table in alert samples. Above this, the output is shown as raw JSON.",
)
@click.option(
"--maximum-rows-in-alert-samples-table",
"-mr",
type=int,
default=25,
help="Maximum number of rows to display in a table in alert samples. Above this, the table will be truncated to prevent Teams payload size issues.",
)
@click.pass_context
def monitor(
ctx,
Expand Down Expand Up @@ -330,6 +337,7 @@ def monitor(
excludes,
teams_webhook,
maximum_columns_in_alert_samples,
maximum_rows_in_alert_samples_table,
quiet_logs,
):
"""
Expand Down Expand Up @@ -364,6 +372,7 @@ def monitor(
report_url=report_url,
teams_webhook=teams_webhook,
maximum_columns_in_alert_samples=maximum_columns_in_alert_samples,
maximum_rows_in_alert_samples_table=maximum_rows_in_alert_samples_table,
quiet_logs=quiet_logs,
)
anonymous_tracking = AnonymousCommandLineTracking(config)
Expand Down
3 changes: 2 additions & 1 deletion elementary/monitor/data_monitoring/alerts/data_monitoring_alerts.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,8 @@ def __init__(
self.alerts_integration = self._get_integration_client()
self.alert_message_builder = AlertMessageBuilder(
MessageBuilderConfig(
maximum_columns_in_alert_samples=self.config.maximum_columns_in_alert_samples
maximum_columns_in_alert_samples=self.config.maximum_columns_in_alert_samples,
maximum_rows_in_alert_samples_table=self.config.maximum_rows_in_alert_samples_table,
)
)

Expand Down