Generic-based typing for Validator classes

Makefile

@@ -39,6 +39,11 @@ venv-tox:
 test:
 	tox run -e clean,py,report
 
+# Only run pytest, but limited to the type checker tests in `tests/mypy`.
+.PHONY: test
+test-typing:
+	tox run -e pytest-mypy
+
 # Only run flake8 linter
 .PHONY: flake8
 flake8:
@@ -71,7 +76,7 @@ docker-tox: TOX_ARGS='-e clean,py313,py312,py311,py310,report,flake8,py313-mypy'
 docker-tox: _docker-tox
 
 # Run partial tox test suites in Docker
-.PHONY: docker-tox-py313 docker-tox-py312 docker-tox-py311 docker-tox-py310
+.PHONY: docker-test-py313 docker-test-py312 docker-test-py311 docker-test-py310
 docker-test-py313: TOX_ARGS="-e clean,py313,py313-report"
 docker-test-py313: _docker-tox
 docker-test-py312: TOX_ARGS="-e clean,py312,py312-report"

docs/03-basic-validators.md

@@ -12,14 +12,7 @@ of the data types that we cover here.
 
 ## Quick overview
 
-There are at least two ways to categorize the existing validator classes.
-
-One distinction would be "base types" vs. "extended types": Base types are all validators that do **not** extend an existing validator,
-i.e. they directly implement the `Validator` base class. Examples for base type validators would be `StringValidator`, `IntegerValidator`
-and `DictValidator`. Extended types are all validators that **do** extend existing validators, e.g. `DecimalValidator` is based on
-`StringValidator`.
-
-A much more useful distinction is to categorize the validators according to their function:
+The validators provided by the library can be roughly categorized based on their functionality and purpose:
 
 - Boolean types:
   - `BooleanValidator`: Validates boolean values (`True` / `False`, optionally allowing strings `"true"` / `"false"`)
@@ -60,7 +53,6 @@ A much more useful distinction is to categorize the validators according to thei
   - `DiscardValidator`: Discards any input and returns a predefined value
   - `AllowEmptyString`: Wraps another validator but allows the input to be empty string `('')`
 
-
 These are a lot of different validators (and there will be even more in future versions) and many of them have a lot of parameters, so we
 will not cover all of them here in detail.
 

docs/05-dataclasses.md

@@ -167,12 +167,12 @@ _Let me tell you about the `DataclassValidator`._
 
 ## The DataclassValidator
 
-The `DataclassValidator` basically is just a very specialized `DictValidator`. It validates dictionaries using **field validators**,
-and then converts the validated dictionaries to objects of a specified **dataclass**.
+The `DataclassValidator` validates dictionaries using **field validators** similar to the `DictValidator` and then
+converts the validated dictionaries to objects of a specified **dataclass**.
 
-But instead of specifying the field validators in the validator, you can now define the validators directly inside the dataclass.
-The `DataclassValidator` will read these field validators from the dataclass and pass them to the underlying `DictValidator`. In the
-same way it also determines which fields are required and which are optional.
+But instead of specifying the field validators in the validator, you can now define the validators directly inside the
+dataclass. The `DataclassValidator` will read these field validators from the dataclass and use them to validate the
+input dictionary. In the same way it also determines which fields are required and which are optional.
 
 The usage of the `DataclassValidator` then is pretty trivial, assuming we have already defined the dataclass:
 

pytest.ini

@@ -4,6 +4,8 @@ addopts =
     --import-mode=importlib
     --cov-context=test
     --cov-report=
+    --mypy-ini-file=tests/mypy/pytest_mypy.ini
+    --mypy-only-local-stub
 
 testpaths = tests
 python_files = *_test.py *Test.py

setup.cfg

@@ -40,6 +40,7 @@ where = src
 testing =
     pytest ~= 8.4
     pytest-cov ~= 6.2
+    pytest-mypy-plugins ~= 3.2
     coverage ~= 7.9
     flake8 ~= 7.3
     mypy ~= 1.17

src/validataclass/dataclasses/validataclass.py

@@ -193,7 +193,7 @@ def _get_existing_validator_fields(cls: type[_T]) -> dict[str, _ValidatorField]:
     return validator_fields
 
 
-def _parse_validator_tuple(args: tuple[Any, ...] | Validator | Default | None) -> _ValidatorField:
+def _parse_validator_tuple(args: tuple[Any, ...] | Validator[Any] | Default | None) -> _ValidatorField:
     """
     Parses field arguments (the value of a field in a dataclass that has not been parsed by `@dataclass` yet) to a
     tuple of a Validator and a Default object.

src/validataclass/dataclasses/validataclass_field.py

@@ -16,7 +16,7 @@
 
 
 def validataclass_field(
-    validator: Validator,
+    validator: Validator[Any],
     default: Any = NoDefault,
     *,
     metadata: dict[str, Any] | None = None,

src/validataclass/validators/__init__.py

@@ -13,18 +13,18 @@
 from .anything_validator import AnythingValidator
 from .big_integer_validator import BigIntegerValidator
 from .boolean_validator import BooleanValidator
-from .dataclass_validator import DataclassValidator, T_Dataclass
+from .dataclass_validator import DataclassValidator
 from .date_validator import DateValidator
 from .datetime_validator import DateTimeFormat, DateTimeValidator
 from .decimal_validator import DecimalValidator
 from .dict_validator import DictValidator
 from .discard_validator import DiscardValidator
 from .email_validator import EmailValidator
-from .enum_validator import EnumValidator, T_Enum
+from .enum_validator import EnumValidator
 from .float_to_decimal_validator import FloatToDecimalValidator
 from .float_validator import FloatValidator
 from .integer_validator import IntegerValidator
-from .list_validator import ListValidator, T_ListItem
+from .list_validator import ListValidator
 from .none_to_unset_value import NoneToUnsetValue
 from .noneable import Noneable
 from .numeric_validator import NumericValidator
@@ -34,6 +34,14 @@
 from .time_validator import TimeFormat, TimeValidator
 from .url_validator import UrlValidator
 
+# Using the following TypeVars outside of the modules they were defined in is deprecated. You should define your own
+# TypeVars if needed. They are still imported here for compatibility, but they won't be exported in __all__, so that
+# linting tools will complain about it.
+# TODO: Deprecated. Remove imports in future version.
+from .dataclass_validator import T_Dataclass  # noqa  # isort:skip
+from .enum_validator import T_Enum  # noqa  # isort:skip
+from .list_validator import T_ListItem  # noqa  # isort:skip
+
 __all__ = [
     'AllowEmptyString',
     'AnyOfValidator',
@@ -59,9 +67,6 @@
     'RegexValidator',
     'RejectValidator',
     'StringValidator',
-    'T_Dataclass',
-    'T_Enum',
-    'T_ListItem',
     'TimeFormat',
     'TimeValidator',
     'UrlValidator',

src/validataclass/validators/allow_empty_string.py

@@ -5,7 +5,9 @@
 """
 
 from copy import deepcopy
-from typing import Any
+from typing import Any, overload
+
+from typing_extensions import Generic, TypeVar
 
 from validataclass.exceptions import InvalidTypeError
 from .validator import Validator
@@ -14,8 +16,15 @@
     'AllowEmptyString',
 ]
 
+# Type parameters for the validation result of the wrapped validator and for the default value (empty string by default)
+T_WrappedValidated = TypeVar('T_WrappedValidated')
+T_EmptyStringDefault = TypeVar('T_EmptyStringDefault', default=str)
+
 
-class AllowEmptyString(Validator):
+class AllowEmptyString(
+    Validator[T_WrappedValidated | T_EmptyStringDefault],
+    Generic[T_WrappedValidated, T_EmptyStringDefault],
+):
     """
     Special validator that wraps another validator, but allows empty strings as the input value.
 
@@ -42,12 +51,20 @@ class AllowEmptyString(Validator):
     """
 
     # Default value returned in case the input is empty string
-    default_value: Any
+    default_value: T_EmptyStringDefault
 
     # Validator used in case the input is not empty string
-    wrapped_validator: Validator
+    wrapped_validator: Validator[T_WrappedValidated]
+
+    @overload
+    def __init__(self, validator: Validator[T_WrappedValidated], *, default: str = ''):
+        ...
+
+    @overload
+    def __init__(self, validator: Validator[T_WrappedValidated], *, default: T_EmptyStringDefault):
+        ...
 
-    def __init__(self, validator: Validator, *, default: Any = ''):
+    def __init__(self, validator: Validator[T_WrappedValidated], *, default: Any = ''):
         """
         Creates a `AllowEmptyString` wrapper validator.
 
@@ -62,18 +79,18 @@ def __init__(self, validator: Validator, *, default: Any = ''):
         self.wrapped_validator = validator
         self.default_value = default
 
-    def validate(self, input_data: Any, **kwargs: Any) -> Any | None:
+    def validate(self, input_data: Any, **kwargs: Any) -> T_WrappedValidated | T_EmptyStringDefault:
         """
         Validates input data.
 
         If the input is an empty string, returns an empty string (or the value specified in the `default` parameter).
         Otherwise, pass the input to the wrapped validator and return its result.
         """
-        if input_data == "":
+        if input_data == '':
             return deepcopy(self.default_value)
 
         try:
-            # Call wrapped validator for all values other than empty string ('')
+            # Call wrapped validator for all values other than empty string
             return self.wrapped_validator.validate_with_context(input_data, **kwargs)
         except InvalidTypeError as error:
             # If wrapped validator raises an InvalidTypeError, add str to its 'expected_types' list and reraise it

src/validataclass/validators/any_of_validator.py

@@ -6,7 +6,7 @@
 
 import warnings
 from collections.abc import Iterable
-from typing import Any
+from typing import Any, TypeVar
 
 from validataclass.exceptions import ValueNotAllowedError, InvalidValidatorOptionException
 from .validator import Validator
@@ -15,8 +15,11 @@
     'AnyOfValidator',
 ]
 
+# Type parameter for the allowed values of an AnyOfValidator
+T_AnyOfValues = TypeVar('T_AnyOfValues')
 
-class AnyOfValidator(Validator):
+
+class AnyOfValidator(Validator[T_AnyOfValues]):
     """
     Validator that checks an input value against a specified list of allowed values. If the value is contained in the
     list, the value is returned.
@@ -59,17 +62,19 @@ class AnyOfValidator(Validator):
     max_allowed_values_in_validation_error: int = 20
 
     # Values allowed as input
-    allowed_values: list[Any]
+    allowed_values: list[T_AnyOfValues]
 
     # Types allowed for input data (set by parameter or autodetermined from allowed_values)
     allowed_types: list[type]
 
     # If set, strings will be matched case-sensitively
     case_sensitive: bool = False
 
+    # TODO: Improve typing: If allowed_types is set, T_AnyOfValues could be narrowed down. This is difficult without an
+    #   intersection type though, but maybe it can be done in the validataclass mypy plugin.
     def __init__(
         self,
-        allowed_values: Iterable[Any],
+        allowed_values: Iterable[T_AnyOfValues],
         *,
         allowed_types: Iterable[type] | type | None = None,
         case_sensitive: bool | None = None,
@@ -120,14 +125,10 @@ def __init__(
         # Set case_sensitive parameter, defaulting to False.
         self.case_sensitive = case_sensitive if case_sensitive is not None else False
 
-    def validate(self, input_data: Any, **kwargs: Any) -> Any:
+    def validate(self, input_data: Any, **kwargs: Any) -> T_AnyOfValues:
         """
         Validate that input is in the list of allowed values. Returns the value (as defined in the list).
         """
-        # Special case to allow None as value if None is in the allowed_values list (bypasses _ensure_type())
-        if None in self.allowed_values and input_data is None:
-            return None
-
         # Ensure type is one of the allowed types (set by parameter or autodetermined from allowed_values)
         self._ensure_type(input_data, self.allowed_types)
 

src/validataclass/validators/anything_validator.py

@@ -5,7 +5,9 @@
 """
 
 from collections.abc import Iterable
-from typing import Any
+from typing import Any, cast, overload
+
+from typing_extensions import TypeVar
 
 from validataclass.exceptions import InvalidValidatorOptionException
 from .validator import Validator
@@ -14,8 +16,15 @@
     'AnythingValidator',
 ]
 
+# Type parameter for the allowed values of an AnyOfValidator
+T_AllowedTypes = TypeVar('T_AllowedTypes', default=object)
+
 
-class AnythingValidator(Validator):
+# TODO: This validator allows a lot of redundant/unnecessary configurations that overcomplicate typing, like setting
+#   `allowed_types` and `allow_none` with the latter influencing the first. Typing-wise, these cases will be ignored,
+#   resulting in a less specific type. In the long term, some of these configurations should be deprecated to simplify
+#   the validator.
+class AnythingValidator(Validator[T_AllowedTypes]):
     """
     Special validator that accepts any input without validation.
 
@@ -65,13 +74,29 @@ class AnythingValidator(Validator):
     allow_none: bool
 
     # Which input types to allow (None for anything)
-    allowed_types: list[type] | None
+    allowed_types: list[type[T_AllowedTypes]] | None
+
+    @overload
+    def __init__(self, *, allow_none: bool | None = None, allowed_types: None = None):
+        ...
+
+    @overload
+    def __init__(self, *, allow_none: bool | None = None, allowed_types: type[T_AllowedTypes]):
+        ...
+
+    @overload
+    def __init__(self, *, allow_none: bool | None = None, allowed_types: Iterable[type[T_AllowedTypes]]):
+        ...
+
+    @overload
+    def __init__(self, *, allow_none: bool | None = None, allowed_types: Iterable[type[T_AllowedTypes] | None]):
+        ...
 
     def __init__(
         self,
         *,
         allow_none: bool | None = None,
-        allowed_types: Iterable[type | None] | type | None = None,
+        allowed_types: Iterable[type[T_AllowedTypes] | None] | type[T_AllowedTypes] | None = None,
     ):
         """
         Creates an `AnythingValidator` that accepts any input.
@@ -140,13 +165,13 @@ def _normalize_allowed_types(
 
         return list(allowed_types_set)
 
-    def validate(self, input_data: Any, **kwargs: Any) -> Any:
+    def validate(self, input_data: Any, **kwargs: Any) -> T_AllowedTypes:
         """
         Validates input data. Accepts anything (or only specific types) and returns data unmodified.
         """
         # Accept None (if allowed explicitly with allow_none=True or implicitly with NoneType in allowed_types)
         if self.allow_none and input_data is None:
-            return None
+            return None  # type: ignore[return-value]
 
         # If allowed_types is set, do a type check (this also raises a RequiredValueError if the input is None)
         if self.allowed_types is not None:
@@ -156,4 +181,4 @@ def validate(self, input_data: Any, **kwargs: Any) -> Any:
             self._ensure_not_none(input_data)
 
         # Return input unmodified
-        return input_data
+        return cast(T_AllowedTypes, input_data)

src/validataclass/validators/boolean_validator.py

@@ -14,7 +14,7 @@
 ]
 
 
-class BooleanValidator(Validator):
+class BooleanValidator(Validator[bool]):
     """
     Validator for boolean values (`True` and `False`).