feat: Route solver output through Python logging

[FBumann]

Summary

• Routes solver output (HiGHS, Gurobi, etc.) through a flixopt.solver child logger via CONFIG.Solving.capture_solver_log
• Solver writes to a log file (log_fn), a background thread tails the file and forwards each line to logging.getLogger('flixopt.solver') at INFO level
• No double logging: solver's native console output is disabled when capturing

Motivation

Previously, solver output was C-level stdout that bypassed Python's logging system entirely. This made it impossible to capture solver logs in file handlers, filter them independently, or integrate them with structured logging pipelines.

Usage

import flixopt as fx

# Via presets (debug, exploring, production, notebook all enable it)
fx.CONFIG.debug()

# Or manually
fx.CONFIG.Solving.capture_solver_log = True
fx.CONFIG.Logging.enable_console('INFO')

# Filter solver output independently
import logging
logging.getLogger('flixopt.solver').setLevel(logging.WARNING)

# Route solver output to a separate file
solver_logger = logging.getLogger('flixopt.solver')
solver_logger.propagate = False
solver_logger.addHandler(logging.FileHandler('solver.log'))

Changes

• flixopt/io.py: Added stream_solver_log() context manager (file tailing via background thread)
• flixopt/config.py: Added capture_solver_log setting (default False), updated presets
• flixopt/flow_system.py: FlowSystem.solve() uses log capture when enabled
• flixopt/optimization.py: Optimization.solve() uses log capture when enabled

Type of Change

• New feature

Testing

• Tested with HiGHS and Gurobi
• Existing tests still pass (137 passed)

Checklist

• My code follows the project style
• I have updated documentation if needed
• I have added tests for new functionality (if applicable)

Summary by CodeRabbit

• New Features

  • Added solver log capture: route solver output through the Python logger at INFO level.
  • New optional progress control and support for passing a persistent solver log file when running solves.

• Configuration / Presets

  • Presets (debug, exploring, production, notebook) enable solver log capture by default; silent keeps it off.

• Documentation

  • Added "Logging & Solver Output" guidance, examples, and double-logging caveats.

• Tests

  • Updated tests to reflect new preset logging expectations.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a public Solving.capture_solver_log flag and wires it into config presets and serialization; introduces stream_solver_log to tail solver output and forward it to the flixopt.solver logger; solve codepaths in FlowSystem/Optimization/OptimizeAccessor use this context when the flag is enabled and a new progress parameter controls progress display.

Changes

Cohort / File(s)	Summary
Configuration flixopt/config.py	Adds capture_solver_log: bool to CONFIG.Solving (default False), includes it in _DEFAULTS and to_dict(), and updates presets: debug, exploring, production, notebook set it True; silent sets it False.
I/O / Log streaming flixopt/io.py	Adds `stream_solver_log(log_fn: pathlib.Path
Solve integration flixopt/flow_system.py, flixopt/optimization.py	FlowSystem.solve (signature now accepts log_fn and progress) and Optimization.solve gain logic: when CONFIG.Solving.capture_solver_log is True, wrap the solve call with fx_io.stream_solver_log() and pass the captured path to the solver (progress disabled for the solver call); otherwise preserve prior log_fn/log_to_console behavior.
Caller API flixopt/optimize_accessor.py	OptimizeAccessor.__call__ gains progress: bool = True and propagates it to FlowSystem.solve; rolling_horizon passes progress=False for per-segment solves.
Docs / Changelog docs/user-guide/optimization/index.md, CHANGELOG.md	Adds "Logging & Solver Output" docs and changelog entry describing capture_solver_log, usage examples, and interactions with console logging.
Tests tests/utilities/test_config.py, tests/deprecated/test_config.py	Adjusts exploring preset expectations: log_to_console now expected False and capture_solver_log expected True.

Sequence Diagram(s)

sequenceDiagram
    participant App as Application
    participant Solver as Solver (linopy / HiGHS)
    participant LogFile as Log File
    participant Tail as Tail Thread
    participant Logger as flixopt.solver Logger

    rect rgba(100,150,200,0.5)
    Note over App,Logger: CONFIG.Solving.capture_solver_log = True
    App->>App: Check CONFIG.Solving.capture_solver_log
    App->>LogFile: Create/open log file (temp if needed)
    App->>Tail: Start tailing thread
    App->>Solver: Call model.solve(..., log_fn=LogFile.path, progress=False)
    Solver->>LogFile: Write solver output lines
    Tail->>LogFile: Read new lines
    Tail->>Logger: Forward lines at INFO
    end

    rect rgba(150,150,150,0.5)
    Note over App,Tail: Context exit
    App->>Tail: Signal tail thread to stop
    Tail->>LogFile: Drain remaining lines & stop
    App->>LogFile: Remove temp file if created
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

• optimize.rolling_horizon() #528 — Changes touching OptimizeAccessor/rolling_horizon behavior; relevant because this PR adds the progress parameter and passes it in rolling horizon flows.
• Feature/402 feature silent framework #407 — Earlier work around solver-output handling and CONFIG.Solving/preset logic that this PR extends with log capture and streaming.

Poem

🐰 I dug a tiny log and found a stream,

Solver whispers hop in lines that gleam,
A tailing thread that listens, neat and small,
I forward every whisper to the logger’s hall,
Quiet console, cosy file — I watch them all.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title concisely summarizes the main feature: routing solver output through Python logging, which is the primary change across multiple files.
Description check	✅ Passed	The PR description covers all required template sections: Summary, Type of Change (New feature), Testing, and Checklist. It provides clear motivation, detailed usage examples, and documents changes across affected files.
Docstring Coverage	✅ Passed	Docstring coverage is 94.44% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/log_fn

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@flixopt/io.py`:
- Around line 1513-1621: stream_solver_log currently starts the tail thread
which reads from log_path from the file start, causing stale content to be
replayed when a user-provided log_fn already exists; before starting the tail
(after resolving log_path and creating parent dirs) ensure existing files are
truncated so only new solver output is streamed (e.g., if not cleanup and
log_path.exists(): open/truncate the file or unlink+recreate it), keeping the
temporary-file behavior unchanged when cleanup is True; update logic near
stream_solver_log/_tail startup where log_path is prepared and before
thread.start() to perform the truncation.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@flixopt/optimization.py`:
- Around line 247-249: The code mutates the caller's solver by setting
solver.log_to_console = False inside the capture block; preserve and restore the
original solver.log_to_console value (e.g., save orig = solver.log_to_console
before changing, set solver.log_to_console = False for the duration of the with
fx_io.stream_solver_log(log_fn=log_fn) context, and restore
solver.log_to_console = orig in a finally or after the context) so the solver
object isn't left mutated after solve() returns; apply the same pattern to the
analogous mutation in flow_system.py where solver.log_to_console is changed.

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@flixopt/config.py`:
- Around line 744-748: The debug() and exploring() presets enable both a console
logger via Logging.enable_console(...) and direct solver console output via
Solving.log_to_console = True while also setting Solving.capture_solver_log =
True, causing duplicate console output; update the debug() and exploring()
functions to set Solving.log_to_console = False (matching notebook() and
production()) so solver output is only routed through the Python logger when
capture_solver_log is enabled, keep Solving.capture_solver_log = True and
preserve the call to Logging.enable_console(...) unchanged.

In `@flixopt/flow_system.py`:
- Around line 1442-1455: FlowSystem.solve currently differs from
Optimization.solve: when CONFIG.Solving.capture_solver_log is True it uses
fx_io.stream_solver_log() which creates a temporary (deleted) file, and when
False it omits log_fn entirely; make behavior consistent by always passing a
log_fn to self.model.solve (like Optimization.solve). Update FlowSystem.solve
signature to accept an optional log_file parameter (or derive a persistent path
if none provided), then: if CONFIG.Solving.capture_solver_log use
fx_io.stream_solver_log() to obtain a path and pass it as log_fn to
self.model.solve; otherwise ensure a persistent log path is obtained/created (or
use the provided log_file) and pass that as log_fn; alternatively, if this
difference is intentional, add a short comment in FlowSystem.solve referencing
Optimization.solve to document the divergence.

🧹 Nitpick comments (1)

tests/utilities/test_config.py (1)

167-167: Consider adding capture_solver_log assertions to the other preset tests.

Only test_preset_exploring verifies the new flag. The silent, debug, production, and notebook presets also set capture_solver_log but lack corresponding assertions, which would catch regressions if preset logic changes.