Skip to content

refactor(wsdb): drop parallel domain types; handlers operate on wire types#23522

Open
charlielye wants to merge 1 commit into
cl/ipc-runtime-zigfrom
cl/ipc-codegen-wire-types
Open

refactor(wsdb): drop parallel domain types; handlers operate on wire types#23522
charlielye wants to merge 1 commit into
cl/ipc-runtime-zigfrom
cl/ipc-codegen-wire-types

Conversation

@charlielye
Copy link
Copy Markdown
Contributor

Summary

Stacked on #23499#23498#23497#23316.

Eliminates the dual type hierarchy that's been sitting under wsdb. Before: `generated/wsdb_types.hpp` (codegen wire types) AND `wsdb_commands.hpp` + `wsdb_execute.cpp` (hand-written domain types with the same fields but semantic typedefs and `execute()` methods). After: one type system. Handlers in `wsdb_handlers.cpp` take wire types in, convert to/from domain types at the boundary, return wire types out. The codegen's `--server` emission is the dispatch entry point; `wsdb_ipc_server.cpp` is now a thin make_server + install_default_signal_handlers + make_wsdb_handler + run.

Net diff: +846 / -1194. Same dispatch logic, smaller surface, single source of truth for command/response shapes (`wsdb_schema.json`).

Codegen changes (`ipc-codegen/src/cpp_codegen.ts`)

  • Emit `Fr` as a struct wrapping `std::array<uint8_t, 32>` plus an explicit `msgpack::adaptor::pack` / `convert` specialization that packs as msgpack `bin`. This matches the schema's `["fr","bin32"]` alias, matches `bb::fr`'s wire bytes, and matches what Rust's `serialize_bytes` and TS's `msgpackr` already produce. The default `std::array<T, N>` adapter packs as `array`, which would have produced wire-incompatible bytes the moment any consumer used wire types as the encoder.
  • `convert` also accepts `array` on the read side for forward compatibility.
  • Codegen-emitted `msgpack_struct_map_impl.hpp` now opts out when `IPC_CODEGEN_USE_BB_MSGPACK_ADAPTORS` is predefined (barretenberg's `serialize/msgpack.hpp` sets it). Otherwise both barretenberg's framework adaptor and the codegen's bundled one partially-specialize `msgpack::adaptor::convert` on overlapping concepts in the same TU, producing "ambiguous partial specializations" errors.

What about bbapi?

bbapi has the same dual-type structure but is much bigger:

  • 62 commands across 8 files (vs wsdb's 30 in one)
  • No C++ codegen step exists for `bb_schema.json` today; needs a CMake `add_custom_command` analogous to wsdb's
  • ~6 non-bbapi consumers reference `bb::bbapi::Command` / `Response` types directly
  • Live wire-format consumers (bb.js, barretenberg-rs)

That's a ~3000 LOC PR on its own. I'd rather land this wsdb migration as the validated pattern and do bbapi as PR-E, applying the same conventions (wire-convert helpers, handler overloads, codegen --server, deleting hand-written types). Same shape, just bigger.

Test plan

  • `ninja aztec-wsdb wsdb_ipc_client wsdb_ipc_merkle_db ipc_runtime_tests` clean
  • `./bin/ipc_runtime_tests` passes
  • ipc-codegen cross-language matrix (18/18) green
  • CI: AVM tests + wsdb integration (Stack B is what'll exercise the wire end-to-end)

…types

Eliminates the dual type hierarchy that's been sitting under wsdb:
- generated/wsdb_types.hpp (codegen wire types: POD, schema-driven)
- wsdb_commands.hpp / wsdb_execute.cpp (hand-written domain types with
  identical fields but semantic typedefs and execute() methods)

After this change there's just one type system per command. Handlers in
wsdb_handlers.cpp take wire types directly, convert fields to/from domain
types at the entry/exit boundary, call the existing world_state APIs, and
return wire response types. The codegen's --server emission becomes the
dispatch entry point; wsdb_ipc_server.cpp shrinks to construction +
make_wsdb_handler<WsdbRequest> + server->run.

Codegen changes (ipc-codegen/src/cpp_codegen.ts):
- Emit Fr as a `struct Fr { std::array<uint8_t, 32> bytes; }` plus an
  explicit `msgpack::adaptor::pack<Fr>` / `convert<Fr>` specialization
  that packs as msgpack `bin` (matching the schema's ["fr","bin32"]
  alias, matching bb::fr's wire bytes, matching what Rust's
  serialize_bytes and TS's msgpackr already produce). The default
  std::array<T,N> adapter packs as `array<uint8>` which would have
  produced wire-incompatible bytes the moment any consumer used wire
  types as the encoder.
- convert<Fr> also accepts `array<uint8>` on the read side for forward
  compatibility with msgpack producers that don't emit `bin`.

Codegen-emitted msgpack_struct_map_impl.hpp now opts out when
IPC_CODEGEN_USE_BB_MSGPACK_ADAPTORS is predefined (barretenberg's
serialize/msgpack.hpp sets it). Otherwise both barretenberg's framework
adaptor and the codegen's bundled one partially-specialize
`msgpack::adaptor::convert<HasMsgPack T>` on overlapping concepts in the
same TU, producing "ambiguous partial specializations" errors.

CMakeLists: wsdb codegen now passes --server (emitting
generated/wsdb_ipc_server.hpp) and the aztec-wsdb target compiles
wsdb_handlers.cpp instead of wsdb_execute.cpp.

The AVM's wsdb_ipc_merkle_db now uses the same shared wire-convert
helpers (wsdb_wire_convert.hpp) instead of its own private copies.

Net diff: +846 / -1194. Same dispatch logic, smaller surface, single
source of truth for command/response shapes (wsdb_schema.json).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant