AttributeError: working_set when using include_subclasses inside struct_hook_factory

[mm-andritz]

I want to de/serialize a complex hierarchy of objects holding the configuration of the system I am working on. Some parts of this hierarchy has subclasses. I want them to have the include_subclasses strategy. That is why I have a fully featured register_structure_hook_factory that handles it.

Below is a minimum reproducible example for the problem. In reality the factory contains even more features (overrides, metadata, etc.). Note: this has been tested on cattrs 24.1.2 but I think the problematic code in include_subclasses is still there.

from functools import partial
from typing import Callable, Type

from attrs import frozen, has
from cattr.gen import make_dict_structure_fn
from cattrs import Converter
from cattrs.gen._consts import already_generating
from cattrs.preconf.json import make_converter
from cattrs.strategies import configure_tagged_union, include_subclasses


@frozen
class A:
    """Base class"""


@frozen
class A1(A):
    a1: int


@frozen
class B:
    id: int
    b: str


@frozen
class Container1:
    id: int
    a: A
    b: B


@frozen
class Container2:
    id: int
    c: Container1
    foo: str


def show_working_set(action: str, cl: Type):
    """Utility to help debuging cattrs global already_generating.working_set..."""
    working_set = getattr(already_generating, "working_set", None)
    if working_set is not None:
        working_set_classes = ",".join([c.__name__ for c in working_set])
    else:
        working_set_classes = "None"

    print(f"{action} {cl.__name__}: working_set={working_set_classes}")


def struct_hook_factory(cl: Type, converter: Converter) -> Callable:
    show_working_set("Before make_dict_structure_fn for", cl)
    struct_hook = make_dict_structure_fn(cl, converter)
    show_working_set("After calling make_dict_structure_fn for", cl)
    if not cl.__subclasses__():
        converter.register_structure_hook(cl, struct_hook)

    else:

        def cls_is_cl(cls, _cl=cl):
            return cls is _cl

        converter.register_structure_hook_func(cls_is_cl, struct_hook)
        union_strategy = partial(configure_tagged_union, tag_name="type")
        show_working_set(f"Before include_subclasses for", cl)
        include_subclasses(cl, converter, union_strategy=union_strategy)
        show_working_set(f"After include_subclasses for", cl)

    return converter.get_structure_hook(cl)


converter = make_converter()
converter.register_structure_hook_factory(has, struct_hook_factory)

unstructured = {
    "id": 0,
    "c": {"id": 1, "a": {"type": "A1", "a1": 42}, "b": {"id": 2, "b": "hello"}},
    "foo": "world",
}
res = converter.structure(unstructured, Container2)
print(res)

Here is the commented output:

$ python cattrs_bug.py
Before make_dict_structure_fn for Container2: working_set=None
Before make_dict_structure_fn for Container1: working_set=Container2
Before make_dict_structure_fn for A: working_set=Container1,Container2
After calling make_dict_structure_fn for A: working_set=Container1,Container2
Before include_subclasses for A: working_set=Container1,Container2  <----- Before entering include_subclasses, 2 classes in set
Before make_dict_structure_fn for A1: working_set=A
After calling make_dict_structure_fn for A1: working_set=A
After include_subclasses for A: working_set=                        <----- After exiting include_subclasses, no more classes!
Before make_dict_structure_fn for B: working_set=
After calling make_dict_structure_fn for B: working_set=None
After calling make_dict_structure_fn for Container1: working_set=None   <----- Logic in make_dict_structure_fn eventually fails
Traceback (most recent call last):
  File "cattrs_bug.py", line 82, in <module>
    res = converter.structure(unstructured, Container2)
  File "...removed.../site-packages/cattrs/converters.py", line 558, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "...removed.../site-packages/cattrs/dispatch.py", line 134, in dispatch_without_caching
    res = self._function_dispatch.dispatch(typ)
  File "...removed.../site-packages/cattrs/dispatch.py", line 75, in dispatch
    return handler(typ, self._converter)
  File "cattrs_bug.py", line 55, in struct_hook_factory
    struct_hook = make_dict_structure_fn(cl, converter)
  File "...removed.../site-packages/cattrs/gen/__init__.py", line 788, in make_dict_structure_fn
    del already_generating.working_set
AttributeError: working_set

The problem seems to be in include_sublcasses that does not hesitate to take ownership of the already_generating.working_set without checking first if it is not empty here and there

[Tinche]

Interesting. Maybe this can be fixed easily by being more careful with the working set? Care to give it a try? :)