Skip to content

Conversation

@devin-ai-integration
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Jan 22, 2026

Summary

When using dbt-databricks with use_materialization_v2 and view_update_via_alter enabled, skipped views return rows_affected as a string "-1" instead of an integer. This causes Spark's VALUES clause to fail with INVALID_INLINE_TABLE.INCOMPATIBLE_TYPES_IN_INLINE_TABLE when building bulk INSERTs for dbt_run_results, since other rows have integer values.

This PR adds a normalize_rows_affected macro that:

  • Converts string "-1" to null (skipped operations have no meaningful row count)
  • Converts other string numbers to integers (defensive handling for adapter quirks)
  • Passes through normal integers unchanged

Fixes: elementary-data/elementary#2089
Linear: CORE-267

Review & Testing Checklist for Human

  • Verify the | int Jinja filter behavior: If a non-numeric string (other than "-1") is passed, the filter may fail or return 0. Consider whether additional validation is needed.
  • Confirm null is the correct semantic for "-1": The fix converts string "-1" to null rather than integer -1. Verify this aligns with how other skipped operations are handled.
  • Test on Databricks: Run a dbt project with view_update_via_alter enabled and verify the on-run-end hook completes successfully with mixed view/model results.

Notes

Summary by CodeRabbit

  • Bug Fixes

    • Standardized handling of "rows affected" in run artifacts so missing values, sentinel values (e.g., "-1"), numeric-like strings, and other edge cases are normalized consistently during artifact processing.
  • Tests

    • Added a test helper and integration tests to validate normalization behavior across representative input cases, ensuring outputs match expected normalized values.

✏️ Tip: You can customize this high-level summary in your review settings.

When using dbt-databricks with view_update_via_alter enabled, skipped views
return rows_affected as a string '-1' instead of an integer. This causes
Spark's VALUES clause to fail with INVALID_INLINE_TABLE.INCOMPATIBLE_TYPES_IN_INLINE_TABLE
when building bulk INSERTs for dbt_run_results.

The fix adds a normalize_rows_affected macro that:
- Converts string '-1' to null (skipped operations have no meaningful row count)
- Converts other string numbers to integers (defensive handling)
- Passes through normal integers unchanged

Fixes: elementary-data/elementary#2089
Linear: CORE-267
Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@linear
Copy link

linear bot commented Jan 22, 2026

@github-actions
Copy link
Contributor

👋 @devin-ai-integration[bot]
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

@coderabbitai
Copy link

coderabbitai bot commented Jan 22, 2026

📝 Walkthrough

Walkthrough

Adds a normalization macro for rows_affected and uses it when flattening adapter responses so string values like "-1" become null (or numeric types) before being included in run-results.

Changes

Cohort / File(s) Summary
Normalization macro & flattening change
macros/edr/dbt_artifacts/upload_run_results.sql
Added normalize_rows_affected(rows_affected); introduced local raw_rows_affected in flatten_run_result; flattened rows_affected now uses elementary.normalize_rows_affected(raw_rows_affected) instead of raw adapter value.
Integration test macro
integration_tests/dbt_project/macros/test_normalize_rows_affected.sql
New macro test_normalize_rows_affected(rows_affected) that returns elementary.normalize_rows_affected(rows_affected) for test invocation.
Unit tests
integration_tests/tests/test_dbt_artifacts/test_normalize_rows_affected.py
Added parameterized test test_normalize_rows_affected covering None, integers, numeric strings, and "-1", asserting expected normalized outputs via dbt_runner.run_operation.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I found a "-1" beside the root,
I wiggled it into a proper boot,
Strings turned numbers, or quietly nil,
Now inserts behave and the logs sit still,
Hooray — the run results are astute!

🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Fix: normalize rows_affected to handle string values from dbt-databricks' accurately describes the main change: adding normalization logic for rows_affected to handle string values returned by the dbt-databricks adapter.
Linked Issues check ✅ Passed The pull request fully addresses all requirements from linked issues #2089 and CORE-267: adds normalize_rows_affected macro, converts string '-1' to null, converts numeric strings to integers, preserves integers unchanged, and includes comprehensive tests.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the stated objectives: the normalize_rows_affected macro, its integration into flatten_run_result, and comprehensive unit/integration tests. No unrelated modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

devin-ai-integration bot and others added 3 commits January 22, 2026 10:07
Tests the following cases:
- None input returns None
- Integer inputs pass through unchanged (123, 0, -1)
- String '-1' returns None (skipped operations)
- Other string numbers are converted to integers ('123', '0', '456')

Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
The log_macro_results helper already applies tojson() to the result,
so the test macro should return the raw value instead of tojson(result).

Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
When the macro returns None, log_macro_results doesn't log anything,
so run_operation returns an empty list. The test now handles this case
by checking if the result list is empty before parsing.

Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
@haritamar haritamar merged commit b8ebc80 into master Jan 27, 2026
50 of 61 checks passed
@haritamar haritamar deleted the devin/CORE-267-1769076042-fix-rows-affected-string branch January 27, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rows_affected string value from dbt-databricks causes INSERT failure on Databricks Runtime 17.3+

1 participant