Skip to content

add serializable protocol#3799

Open
d-v-b wants to merge 2 commits intozarr-developers:mainfrom
d-v-b:feat/serializable-protocol
Open

add serializable protocol#3799
d-v-b wants to merge 2 commits intozarr-developers:mainfrom
d-v-b:feat/serializable-protocol

Conversation

@d-v-b
Copy link
Contributor

@d-v-b d-v-b commented Mar 20, 2026

This PR defines a JSONSerializable protocol in zarr.abc.serializable that our metadata classes can use. That protocol is generic with 2 type parameters. 1 type parameter is used to declare the type that the class can be deserialized from, and the other type parameter declares the type the class serializes to. This allows our metadata classes to have generous input types, and narrow output types.

The JSONSerializable protocol has 3 methods:

  • to_json (serialization)
  • from_json (deserialization from typed input)
  • try_from_json (deserialization from arbitrary input; performs type checking before calling from_json)

I implement these methods on the ArrayV3Metadata class, which involves the following changes:

  • Defining the ArrayMetadataJSONLike_V3 type, which is effectively the signature of ArrayV3Metadata.__init__
  • Widening the type of the data_type parameter that the ArrayV3Metadata will accept. Now it accepts string or object declarations of ZDType instances, instead of requiring ZDType instances.
  • Widening the type of the shape parameter that ArrayV3Metadata will accept. It now accepts a ShapeLike input.
  • Defined new ChunkGridLike and CodecLike aliases for the inputs ArrayV3Metadata.__init__ accepts
  • Adding DimensionNamesLike as an alias for DimensionNames

The goal is to make our array metadata class more useful by more clearly defining the types it accepts and creates. If we accept these changes, we can remove metadata parsing at higher levels in the codebase, and move towards making the ArrayV3Metadata class part of our public API.

related:
#3786
#3795

@d-v-b d-v-b requested a review from a team March 20, 2026 14:22
@d-v-b
Copy link
Contributor Author

d-v-b commented Mar 20, 2026

for context, we have a Metadata ABC, but it has a serious flaw: if we define some class method like from_dict(obj: T) -> Self, no subclass can narrow the type T to something more specific without violating liskov substitution. So this base class approach is a dead end if we ever want separate metadata objects to clearly indicate the type the deserialize from, and IMO we do want that.

...

@classmethod
def try_from_json(cls, obj: object) -> Self:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def try_from_json(cls, obj: object) -> Self:
def from_object(cls, obj: object) -> Self:

as discussed on zulip, I'm don't find try_from_json to be intuitive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I'm a fan of from_object, because the term "object" is unfortunately overloaded in python. People might think "object" means "a class instance" as opposed to "the base type object". I'd like to get a range of opinions here. cc @zarr-developers/python-core-devs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from_json_checked could convey that there's a type checking step...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we require typed input for now and add the untyped method later if there's demand?

from_json_checked could convey that there's a type checking step...

I think it could also mean that it's from a checked json. what about from_untyped_json or from_untyped?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants