Skip to content

feat(sql): Bundle Databricks SQLAlchemy dialect #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
m1so merged 5 commits into from
Jan 27, 2026
+315 −59
Merged

feat(sql): Bundle Databricks SQLAlchemy dialect #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8a0cf9d
feat(sql): Bundle Databricks SQLAlchemy dialect
m1so Jan 27, 2026
4f30ccd
fix: Ignore protobuf DoS CVE
m1so Jan 27, 2026
f8b8dfa
fix: Ignore false positive Arrow finding
m1so Jan 27, 2026
9264447
fix: Flaky test by not patching `time.sleep`
m1so Jan 27, 2026
2b03f7c
fix: Suppress warnings from Databricks packages
m1so Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,8 +465,17 @@ jobs:
# PYSEC-2023-121 (CVE-2022-4899 / GHSA-5c9c-6x87-f9vm)
# * incorrectly flags zstd versions >= 1.5.4.0 as vulnerable to CVE-2022-4899
# * see https://github.com/pypa/advisory-database/blob/main/vulns/zstd/PYSEC-2023-121.yaml (our version ranges are outside of reported versions)
# CVE-2026-0994 (protobuf DoS via nested Any messages in ParseDict)
# * no fix version available yet
# * low risk: only affects json_format.ParseDict() with untrusted input
# PYSEC-2024-161 (pyarrow deserialization vulnerability)
# * false positive: only affects Arrow R package, not PyArrow
# * see CVE description: "This vulnerability only affects the arrow R package, not other Apache Arrow implementations"
# * databricks-sqlalchemy 1.x caps pyarrow<17, but upgrading requires SQLAlchemy 2.x (which is not possible for some Python versions)
ignore-vulns: &ignore-vulns |
PYSEC-2023-121
CVE-2026-0994
PYSEC-2024-161

audit-all:
name: Audit - All
Expand Down
26 changes: 25 additions & 1 deletion deepnote_toolkit/sql/sql_execution.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,27 @@ def _execute_sql_with_caching(
)


@contextlib.contextmanager
def suppress_third_party_deprecation_warnings():
"""Suppress known deprecation warnings from third-party SQL packages.

These warnings are caused by internal implementation details of upstream packages
and cannot be fixed in deepnote-toolkit. We suppress them to avoid cluttering
user output with warnings they cannot act upon.

Suppressed warnings:
- databricks-sqlalchemy: '_user_agent_entry' parameter deprecated
https://github.com/databricks/databricks-sqlalchemy/issues/36
"""
with warnings.catch_warnings():
# databricks-sqlalchemy uses deprecated '_user_agent_entry' parameter
warnings.filterwarnings(
"ignore",
message=r"Parameter '_user_agent_entry' is deprecated",
)
yield


def _query_data_source(
query,
bind_params,
Expand All @@ -454,7 +475,10 @@ def _query_data_source(
if url is None:
url = sql_alchemy_dict["url"]

engine = create_engine(url, **sql_alchemy_dict["params"], pool_pre_ping=True)
with suppress_third_party_deprecation_warnings():
engine = create_engine(
url, **sql_alchemy_dict["params"], pool_pre_ping=True
)

try:
dataframe = _execute_sql_on_engine(engine, query, bind_params)
Expand Down
Loading