refactor: stateful Solver instances and two-step solve API

[FabianHofmann]

closes #628 #583

Changes proposed in this Pull Request

Refactor of the solver layer to put solver state on a stateful Solver instance and expose a clean construct-then-solve workflow.

• Stateful Solver on Model.solver. Solver state (native model, results) now lives on a Solver instance attached to Model.solver. Model.solver_model and Model.solver_name become read-only properties delegating to model.solver (assigning anything but None raises; setting None closes the solver). Model.solver_name may be None before a solve. These two properties are candidates for future deprecation. The solver exposes an explicit close() (also auto-invoked from __del__) that releases the native handle and any held license.
• Construct-then-solve API. Build a solver via Solver.from_name(name, model, io_api=..., options=...) (or SolverClass.from_model(model, ...)), then call solver.solve() to run and obtain a Result, and model.assign_result(result) to write the solution back. Solver is a dataclass; subclasses no longer need __init__ overrides.
• Clear lifecycle hooks per solver. Each subclass overrides at most three methods: _build_direct(**kwargs) (build the native model from self.model), _run_direct(**kwargs) (run the prebuilt native model), and _run_file(**kwargs) (invoke the solver on self._problem_fn). File-only solvers (CBC, GLPK, Cplex, SCIP, Xpress, Knitro, COPT, MindOpt) override only _run_file. Direct-API solvers (Highs, Gurobi, Mosek, cuPDLPx) override all three.
• Legacy entry points deprecated. Solver.solve_problem, Solver.solve_problem_from_model, and Solver.solve_problem_from_file are kept on the base class as thin shims that route through the new pipeline and emit DeprecationWarning. To be removed in a future release.
• Two ways to get the native solver model. Either via the module-level / Model-bound helpers (model.to_gurobipy(), model.to_highspy(), to_cupdlpx(model)), or directly via Solver.from_model(model, io_api=\"direct\").solver_model. The previous public Solver.to_solver_model method is removed and folded into the internal _build_direct hook to avoid exposing a third redundant path.
• Declarative solver capabilities. Capabilities are declared as features: frozenset[SolverFeature] ClassVars on each Solver subclass (with an optional runtime_features() classmethod for version-dependent ones); query with Solver.supports(feature). SolverFeature is exported from linopy (and linopy.solvers); linopy.solver_capabilities remains as a back-compat shim with a lazy SOLVER_REGISTRY mapping and solver_supports() helper.
• License-aware solver discovery. linopy.available_solvers is now lazy and no longer probes license-managed solvers (Gurobi, Mosek, Knitro, MindOpt, COPT, cuPDLPx) at import time — membership now means "package/binary installed", not "working license". Same lazy behaviour for quadratic_solvers. New linopy.licensed_solvers returns the installed solvers that currently pass a license probe. New linopy.solvers.check_solver_licenses(*names), plus per-class SolverClass.is_available() and SolverClass.license_status() returning a LicenseStatus dataclass (name, ok, message). All caches are refreshable via .refresh().
• Result carries solver report. Result gains solver_name and report: SolverReport | None (runtime, MIP gap, dual bound, iteration counts) and prints them in __repr__. CBC, HiGHS, Gurobi, Knitro, and cuPDLPx populate report; when possible also populate the MIP dual_bound. SolverReport is exported from linopy.constants.
• Positional-indexed solution mapping. Solution.primal and Solution.dual are now dense np.ndarrays indexed by linopy label (length = max_label + 1); masked or solver-dropped slots are NaN. Previously pd.Series keyed by name. Each solver emits arrays in this label-indexed form — direct-API solvers via cached _vlabels/_clabels populated at _build_direct time, file-based solvers via the shared _solution_from_names helper. Robust to solver iteration order and to solvers dropping unused variables (e.g. CPLEX on LP read). The dense numpy arrays are assigned to Model.solution by simple reshaping.
• Cached constraint labels. Constraints exposes a cached label_index (mirroring Variables.label_index), so assign_result and IIS extraction no longer trigger a full model.matrices rebuild. Invalidated on add/remove. ConstraintBase gains active_labels() and a range property, and CSRConstraint exposes coords to support this without forcing a CSR build.
• Common helpers. linopy.common gains values_to_lookup_array and ConstraintLabelIndex; the legacy pandas-keyed helpers series_to_lookup_array and lookup_vals are removed.

Reviewer requests

• @FBumann: align SolverReport with SolverMetrics (feat: add unified SolverMetrics #583); expose dual_bound (best bound)
• @coroa: make Solver a dataclass; drop the solver_class(**solver_options) pattern
• @coroa: add Solver.from_name(name, model, io_api=..., options=...) static constructor + SolverClass.from_model(...) classmethod
• @coroa: drop prepare_solver / run_solver two-step API
• @coroa: cache vlabels / clabels on the solver instead of going through model.matrices, see vlabels-clabels-flow.html for more details
• @coroa: adopt the Model.solve = self.solver = Solver.from_name(...); assign_result(...) pattern in Model.solve
• @coroa: add is_available() classmethod per solver + lazy SOLVER_REGISTRY so available_solvers discovery doesn't grab licenses prematurely
• @coroa: refactor OETC as a Solver subclass (Refactor OETC as a Solver subclass #683)
• @coroa: keep the interface extensible for asynchronous solving (Gurobi batch optimization, OETC) — return early with a job handle and retrieve later (Refactor OETC as a Solver subclass #683)
• @coroa: incremental update path — solver holds a shallow copy of the linopy model, solver.update(model) diffs and pushes only changed bounds/rhs/coefficients (CoW on variable/constraint data) (defer to follow-up pr)

Checklist

• Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in `doc`.
• Unit tests for new features were added (if applicable).
• A note for the release notes `doc/release_notes.rst` of the upcoming release is included.
• I consent to the release of this PR's code under the MIT license.

[FBumann]

@FabianHofmann I suggest checking the new class SolverReport against my SolverMetrics in #583, the purpose is pretty much the same. This will also probably close #583.
One thing we definitely need is SolverReport.dual_bound (also known as best bound).
Peak memory was also added in #583, but might be less commonly exposed by the solver. I also dont really need it...

[coroa]

Can we make the solver class into a data class, too? And get rid of this strange instantiate solver_class(**solver_options) pattern.

I am not sure whether there is a benefit to holding a solver class instance without a model attached, so what about constructors like:

Solver.from_name("gurobi", linopy_model, io_api=..., options=options)
# a staticmethod

which dispatches to

Gurobi.from_model(linopy_model, io_api=..., options=options)
# the specific classmethod, even though that is also implemented on the main Solver class

which dispatches according to io_api.

I'd say this then gets rid of any case for prepare_solver or some such.

[coroa]

Benchmark: OETC should be able to become a solver class.

class Model:
    ...
    def solve(solver_name, io_api, **solver_options):
        self.solver = None
        self.solver = solver = Solver.from_name(solver_name, model=self, io_api=io_api, options=solver_options)
        result = solver.solve()
        self.apply_result(result)

[coroa]

Solvers should get a class method:

class Gurobi:
    @classmethod
    def is_available():
        try:
            from gurobipy import Model
            with Model():
                  return True
         except ImportError, LicenseError?:
             return False

We then use a derived Collection pattern similar to the following for available_solvers so not to grab licenses prematurely.

https://github.com/snakemake/snakemake/pull/3900/changes#diff-857b2ff1aff916efdddad6bcd645a90f388276460cf75e6c9a0362745a70371fR13-R58

[coroa]

ah yes, and OETC and some Gurobi compute like instances can have an asynchronous solving option. Where you do not want to block the process, but return early and only give back some sort of job identifier in hand with which to retrieve the solution later.

Gurobi docs: https://docs.gurobi.com/projects/optimizer/en/current/features/batchoptimization.html

And i don't mean implement this now, here, but the interface should be extensible to allow for that.

[coroa]

i am unsure how to effectively keep track of the state that was communicated to the solver. ie. when you then do modifications on the linopy model data after it was communicated to the solver. when do you send those updates (and which updates)

the most promising way in my mind would be the following:

the solver object holds a shallow copy of the linopy model (up until the individual constraint and variable objects). EDIT: probably only of the constraints and variable objects.

when you make an update to a variable bound or constraint you use some sort of cow to create a copy in the linopy model (this mostly means something like a v.data = v.data.assign pattern and maybe on mutable constraints and rhs an explicit cow numpy flag).

this means you share variable data/constraint data until you make the first modification.

then you do solver.update(linopy_model) which does diffing and sends changes to solver.

several nice characteristics that way:

1. you don't send changes prematurely (ie. immediately communicate any change on rhs assignment)
2. sending changes is minimal/ incremental
3. its compatible with putting solver connections into a separate thread
4. update does not need to care whether it is effectively the same model or a completely newly constructed one
5. memory use is efficient

[coroa]

@FabianHofmann i reread the pr description. don't use vlabels, clabels through m.matrices. m.matrices is expensive when constraints are not frozen. but I think you know and that's why they are stored on solver, isn't it

[FabianHofmann]

@FabianHofmann I suggest checking the new class SolverReport against my SolverMetrics in #583, the purpose is pretty much the same. This will also probably close #583. One thing we definitely need is SolverReport.dual_bound (also known as best bound). Peak memory was also added in #583, but might be less commonly exposed by the solver. I also dont really need it...

thanks @FBumann for raising this. I pulled over the dual_bound feature from your pr. should cover all if it then

[FBumann]

Thats a great and important note for commercial usage, as the license is blocked otherwise. We might want to add a notebook about the new solver stuff, as an example with highs or gurobi...? If you want i can craf one, exploring the new feature on the way and you can add what i missed?

[FabianHofmann]

cool, sounds good!

[FabianHofmann]

@FBumann I am done with my features for this pr. another will follow for the persistent solve. since jonas is on holiday he won't be able to make another review here. do you want to take another look? focus could be breaking changes and whether flexopt tests pass with it (pypsa does). feel free to pull in the notebook here or to raise another pr in case you are able to quickly approve (no pressure anyway)

[FBumann]

@FabianHofmann No, i think a followup for the notebook is better anyway.
I'll let the flixopt test suite run on this PR now

[FBumann]

@FabianHofmann All tests pass.

[FabianHofmann]

wonderful! thanks @FBumann