SQLite plugin: respect config.SQL.MaxConns instead of hardcoding SetMaxOpenConns(1)#9730
Open
brucearctor wants to merge 1 commit intotemporalio:mainfrom
Open
SQLite plugin: respect config.SQL.MaxConns instead of hardcoding SetMaxOpenConns(1)#9730brucearctor wants to merge 1 commit intotemporalio:mainfrom
brucearctor wants to merge 1 commit intotemporalio:mainfrom
Conversation
…axOpenConns(1) Fixes temporalio#9686 The SQLite plugin previously hardcoded db.SetMaxOpenConns(1) and db.SetMaxIdleConns(1), ignoring the config.SQL.MaxConns and config.SQL.MaxIdleConns fields. This caused all Temporal services to serialize through a single database connection, leading to ShardIOTimeout failures at ~1000 concurrent workflows. This change makes the SQLite plugin respect the user's connection pool configuration, consistent with the MySQL and PostgreSQL plugins: - cfg.MaxConns controls SetMaxOpenConns (default: 1) - cfg.MaxIdleConns controls SetMaxIdleConns (default: 1) - cfg.MaxConnLifetime controls SetConnMaxLifetime A warning is logged if MaxConns > 1 without WAL mode (journal_mode=wal) enabled, since SQLite without WAL will encounter 'database is locked' errors under concurrent access. For in-memory databases, ConnMaxIdleTime remains 0 (infinite) to prevent the database from being garbage collected when idle connections close.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #9686
The SQLite plugin previously hardcoded
db.SetMaxOpenConns(1)anddb.SetMaxIdleConns(1), ignoring theconfig.SQL.MaxConnsandconfig.SQL.MaxIdleConnsfields. This caused all Temporal services (frontend, history, matching, worker) to serialize through a single database connection, leading toShardIOTimeoutfailures at ~1000 concurrent workflows.Changes
This change makes the SQLite plugin respect the user's connection pool configuration, consistent with the MySQL and PostgreSQL plugins:
cfg.MaxConnsSetMaxOpenConnscfg.MaxIdleConnsSetMaxIdleConnscfg.MaxConnLifetimeSetConnMaxLifetimeSafety guardrails
MaxConns > 1without WAL mode (journal_mode=wal) enabled, since SQLite without WAL will encounterdatabase is lockederrors under concurrent access.ConnMaxIdleTimeremains0(infinite) to prevent the database from being garbage collected when idle connections close.MaxConns/MaxIdleConnsare unset or 0, the plugin defaults to 1 — identical to current behavior.Testing
go test ./common/persistence/sql/sqlplugin/sqlite/...)go build ./common/persistence/sql/sqlplugin/sqlite/...)