Replace voluptuous with a faster schema validator

[ahal]

Imported from: https://bugzilla.mozilla.org/show_bug.cgi?id=1652123

When profiling taskgraph in Gecko, we discovered that voluptuous is responsible for a significant performance hit to Taskgraph. So much so that we disbale schema validation entirely when taskgraph.fast is set. We also avoid using voluptuous to set default values and instead set defaults via a transform.

There are many more performant validation libraries out there. In another project, I once successfully replaced voluptuous with validx (but we should investigate all options).

[Alphare]

I have tried migrating over the task_decision_schema to validx and with careful ordering of the OneOf attributes to prevent error creation and fallback and general attention to validx best practices, validating the schema drops from ~4s to ~1s over all calls within a large decision task when run on my computer.

This was an experiment to see whether this would make any impact and it does show promise, but only if we can add native extensions (validx uses Cython) to the decision task, and migrate over more schemas.

See https://hg.mozilla.org/try/rev/fcdcde5e36caba6bd74bf7c5ee41114874fec397

[bhearsum]

pydantic seems to be the standard for this sort of thing these days. According to https://validx.readthedocs.io/en/latest/benchmarks.html, validx is ~5x/faster than pydantic 1.x. According to this issues, pydantic 2.x is massively faster than 1.x when it comes to validation.

We should do a small scale test of validx vs. pydantic to help inform which we should choose.

[ahal]

Yeah, it looks like validx isn't well maintained anymore, so I'd lean towards pydantic between the two of them (even if 2.x is slightly slower).

I was also meaning to look into https://github.com/jcrist/msgspec, which I saw someone recommend as a faster, lighter weight alternative to pydantic.

[bhearsum]

https://www.reddit.com/r/Python/comments/w68nv3/a_quick_benchmark_comparing_pydantic_v2_with/ suggests that msgspec is significantly faster than even pydantic 2.x. https://gist.github.com/jcrist/d62f450594164d284fbea957fd48b743 has similar results (albeit that's from the msgspec author; I don't know if there's any bias there.)

[bhearsum]

@abhishekmadan30 has been working on this in #844. There are some complications around getting it landed, due to it being written in rust, but those are being worked through.