Enforce strict mypy type checking and additional ruff rules by FabianHofmann · Pull Request #492 · PyPSA/atlite

FabianHofmann · 2026-03-16T10:16:09Z

Changes proposed in this Pull Request

This PR enforces strict static type checking with mypy and enables additional ruff lint rules across the entire codebase (including test/ and doc/).

Docstrings

Add docstrings to all functions

Mypy

Configured strict mypy settings in pyproject.toml: disallow_incomplete_defs = true, check_untyped_defs = true
Removed all directory exclusions from mypy — it now checks atlite/, test/, and doc/
Updated .pre-commit-config.yaml to match
Leveraged existing atlite/_types.py type aliases and added TypeAlias annotations where needed (e.g. csp.py, utils.py)
Added missing type annotations to function signatures (e.g. conversion functions in convert.py)
Fixed __all__ in __init__.py to use strings instead of object references

Ruff

Enabled 9 new rule sets and fixed all violations (~260 fixes):

B (flake8-bugbear): Added from err/from None to re-raised exceptions, fixed mutable defaults
SIM (flake8-simplify): Collapsed nested if statements, simplified dict operations
RET (flake8-return): Removed superfluous else after return/raise
C4 (flake8-comprehensions): Simplified unnecessary dict()/list() constructors
TC (flake8-type-checking): Moved type-only imports into TYPE_CHECKING blocks
NPY (numpy): Replaced legacy np.random.seed/np.random.rand with np.random.default_rng
G (flake8-logging-format): Replaced f-strings in logging with lazy % formatting
PTH (flake8-use-pathlib): Replaced os.path.* and open() with pathlib equivalents
RUF100: Cleaned up unused # noqa comments

TODO

replace cast() where possible, need bump python version for mypy
add ruff check for docstring consistency

Checklist

Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
Newly introduced dependencies are added to environment.yaml, environment_docs.yaml and setup.py (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

…cross codebase

…l violations

…covered by ruff)

…t.py

…nd removing unnecessary type: ignore comments

…tions, remove stray code

…all 61 violations

…te_time helper, consistent keep_attrs, use fixtures in tests

for more information, see https://pre-commit.ci

…ypes across codebase - Add NumPy-style docstrings to all functions in datasets/era5.py and improve convert.py docstrings - Define shared Literal type aliases (TrackingType, ClearskyModel, TrigonModel, etc.) in _types.py - Update ~30 function signatures across convert.py, pv/, and data.py to use Literal types - Fix CSPConfig TypedDict missing 'technology' key and widen orientation/panel param types

Per-file ignores for datasets modules still lacking docstrings (cordex, ncep, sarah, gebco).

…d remove per-file D103 ignores

for more information, see https://pre-commit.ci

…inition

lkstrp

Tons of good cleanup changes, thank youuu ! Just a bit too much and mixed which makes it a bit hard to review, since this now includes changed docstrings, ruff, types and tests.

I think some things need another look. I only went through parts for it now:

There are many comments removed, and sometimes it looks like Claude chucked them, but some might be removed intentionally? I am not always sure and I would wanna keep hand crafted code explaining comments in general (if useful and up to date). The same goes for docstrings sometimes. They are rewritten, often improved, but sometimes useful old information is removed
In _types.py, many types are defined which are then used everywhere. This makes sense for new literals and more complicated types, but some also just define an alias for existing types from the library. I suppose this is also a style question, but since I believe that simplicity is becoming increasingly important, I would try to simplify it. If an argument requires many different types, this is also an indication that the signature should be restructured. Abstracting all those types away just hides that

lkstrp · 2026-04-08T11:32:44Z

atlite/datasets/__init__.py

+if TYPE_CHECKING:
+    from types import ModuleType
+
+modules: dict[str, ModuleType] = {"era5": era5, "sarah": sarah, "gebco": gebco}


Why is ModuleType needed here?

lkstrp · 2026-04-08T11:32:57Z

atlite/datasets/cordex.py

        ds = ds.sel(x=xs, y=ys)

        if newname in {"influx", "outflux"}:
-            # shift averaged data to beginning of bin


Why remove the comments?

I just don't like comments which are describing the code

lkstrp · 2026-04-08T11:33:00Z

atlite/datasets/cordex.py

                )
            )
        elif newname in {"runoff"}:
-            # shift and fill 6hr average data to beginning of 3hr bins


Why remove the comments?

lkstrp · 2026-04-08T11:33:08Z

atlite/datasets/cordex.py

-            config.cordex_dir,  # noqa: F821
+weather_data_config: dict[str, dict[str, Any]] = {}
+try:
+    from atlite import config  # type: ignore[attr-defined]


That should never be an import error is it?

lkstrp · 2026-04-08T11:34:01Z

atlite/datasets/cordex.py

+            "prepare_func": prepare_weather_types_cordex,
+            "oldname": "CWT",
+            "newname": "CWT",
+            "template": os.path.join(  # noqa: PTH118


I would rather ignore PTH118 globally for now, so we can add it easily later again. Using pathlib instead of os is very recommended

lkstrp · 2026-04-08T11:40:53Z

atlite/pv/irradiation.py

        )
    elif clearsky_model == "enhanced":
-        # Enhanced Reindl model with ambient air temperature and relative
-        # humidity


Why remove the comment?

lkstrp · 2026-04-08T11:40:59Z

atlite/pv/irradiation.py

    influx = direct + diffuse

    with np.errstate(divide="ignore", invalid="ignore"):
-        # brightening factor


Why remove comments?

lkstrp · 2026-04-08T11:41:33Z

atlite/_types.py

+NDArrayInt: TypeAlias = np.ndarray[Any, np.dtype[np.signedinteger[Any]]]
+NDArrayBool: TypeAlias = np.ndarray[Any, np.dtype[np.bool_]]
+DataArray: TypeAlias = xr.DataArray
+Dataset: TypeAlias = xr.Dataset


Do we need all those? Some are just aliases. I think it's much cleaner to just keep for example use xr.Dataset than redefining it?

lkstrp · 2026-04-08T11:42:35Z

atlite/_types.py

+SparseMatrix: TypeAlias = sp.lil_matrix | sp.csr_matrix
+
+TrackingType: TypeAlias = (
+    Literal["horizontal", "tilted_horizontal", "vertical", "dual"] | None


I like to define those literals! But I would not put | None in there in general. That information if something is optional should be in the signature. And again, I am not so sure if this needs to be defined in an extra Type? How often is this actually used in the signature? If I wanna see this in the function I need to click once more now. Guess for some it makes sense, but you think for all of them?

lkstrp · 2026-04-08T11:42:57Z

atlite/_types.py

+DataFormat: TypeAlias = Literal["grib", "netcdf"]
+
+
+class ERA5RetrievalParams(TypedDict, total=False):


Is it wise to hardcode that here? Since it's a dict it doesn't give you any benefits, is it?

FabianHofmann and others added 22 commits March 13, 2026 14:43

Add aggregate_time support to convert_and_aggregate

bf12877

add missing docstrings

5543e1a

Add mypy type checking with strict settings and fix all type errors a…

e0c511c

…cross codebase

Enable ruff rules B, SIM, RET, C4, TC, NPY, G, PTH, RUF100 and fix al…

9228ca7

…l violations

Include test directory in mypy checking and fix all test type errors

820e7e2

Remove examples/ from mypy exclude (no .py files to check, notebooks …

b1edf3f

…covered by ruff)

Include doc/ in mypy checking and fix type errors in conf.py and char…

c5a7a8d

…t.py

Bump mypy python_version to 3.11, enabling proper xarray Self types a…

8153e27

…nd removing unnecessary type: ignore comments

Fix mypy errors: remove unused type: ignore comments, fix type annota…

16cd01c

…tions, remove stray code

Enable ruff DOC/D417 rules for docstring-signature alignment and fix …

6591a14

…all 61 violations

Address PR review: add 'legacy' default, drop False, extract _aggrega…

80d6cdb

…te_time helper, consistent keep_attrs, use fixtures in tests

update release notes

5a2fba9

Merge branch 'feat/aggregate_time' into refac/type-annotation

7fe656b

[pre-commit.ci] auto fixes from pre-commit.com hooks

0f1d464

for more information, see https://pre-commit.ci

Enforce D103 (missing docstring in public function) via ruff

fd68cd3

Per-file ignores for datasets modules still lacking docstrings (cordex, ncep, sarah, gebco).

Add docstrings to all dataset modules (sarah, ncep, cordex, gebco) an…

2d0497c

…d remove per-file D103 ignores

Merge branch 'master' into refac/type-annotation

2486e59

[pre-commit.ci] auto fixes from pre-commit.com hooks

67f7366

for more information, see https://pre-commit.ci

Ignore D103 in test/ and examples/, fix duplicate _aggregate_time def…

aef7b64

…inition

Remove commented-out D103 ignore line

182b93e

Fix docstring lint errors: D205, D401, D100, D101, D105, DOC201

0c807ce

lkstrp self-requested a review April 1, 2026 13:20

FabianHofmann added 2 commits April 7, 2026 11:32

Merge branch 'master' into refac/type-annotation

80f744a

Merge branch 'master' into refac/type-annotation

55d5345

lkstrp requested changes Apr 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enforce strict mypy type checking and additional ruff rules#492

Enforce strict mypy type checking and additional ruff rules#492
FabianHofmann wants to merge 24 commits intomasterfrom
refac/type-annotation

FabianHofmann commented Mar 16, 2026 •

edited

Loading

Uh oh!

lkstrp left a comment

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

FabianHofmann Apr 9, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

lkstrp Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		DataFormat: TypeAlias = Literal["grib", "netcdf"]


		class ERA5RetrievalParams(TypedDict, total=False):

Conversation

FabianHofmann commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Docstrings

Mypy

Ruff

Checklist

Uh oh!

lkstrp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FabianHofmann commented Mar 16, 2026 •

edited

Loading