refactor(bbapi): adopt codegen-emitted dispatch; wire-typed handler boundary#23527
Open
charlielye wants to merge 2 commits into
Open
refactor(bbapi): adopt codegen-emitted dispatch; wire-typed handler boundary#23527charlielye wants to merge 2 commits into
charlielye wants to merge 2 commits into
Conversation
…oundary
api_msgpack.cpp's hand-rolled msgpack dispatch lambda (~80 LOC of
length-prefix decode + NamedUnion convert + bbapi() call + error
wrapping) is replaced by make_bb_handler<BBApiRequest>(request) from
the codegen-emitted bb_ipc_server.hpp. Same wire format, same in-process
semantics — BBApiRequest is kept at server-function scope so IVC state
(loaded_circuit_constraints, ivc_in_progress, etc.) is preserved across
the call sequence (ChonkStart → ChonkLoad → ChonkAccumulate → ChonkProve),
matching the bbapi() global_request convention.
C++ codegen step added for bb_schema.json
(barretenberg/cpp/src/barretenberg/bbapi/CMakeLists.txt). Same shape as
the wsdb side: add_custom_command emits generated/bb_{types,ipc_client,
ipc_server}.hpp and the codegen's msgpack_struct_map_impl.hpp (opted out
in barretenberg context via IPC_CODEGEN_USE_BB_MSGPACK_ADAPTORS).
bbapi_objects gets ipc_runtime as a PUBLIC link dependency since the
generated headers include ipc_runtime/ipc_server.hpp.
60 non-Shutdown commands need a non-template handle_<method> overload
for BBApiRequest so make_bb_handler<BBApiRequest>'s template instantiation
resolves to a concrete definition. Each handler is one line:
return msgpack_roundtrip<wire::Resp>(
msgpack_roundtrip<DomainCmd>(cmd).execute(ctx));
The wire and domain types share a SERIALIZATION_FIELDS shape (same field
names, msgpack-compatible field types — Fr packs as bin32 matching bb::fr;
nested point types pack as {x, y} maps in both versions), so a wire value
can be turned into its domain counterpart by pack-then-unpack. Slower
than field-by-field copying (two extra pack/unpack passes) but trivial
to write and resistant to schema drift — a follow-up can replace the
roundtrips with field-by-field conversions and delete the hand-written
domain types.
Verified:
- ninja bb bb-avm aztec-wsdb ipc_runtime_tests bbapi_tests api_tests
- ./bin/bbapi_tests → 30 tests pass (msgpack default-roundtrip suite +
CBind exception-handling tests, covering CircuitProve, ChonkProve,
all the structural commands).
- ./bin/ipc_runtime_tests → 2 SHM tests pass.
- ipc-codegen ./bootstrap.sh test → 18/18 cross-language matrix green.
- Wire format unchanged: bb.js and barretenberg-rs continue to talk to
the same bb binary over the same msgpack bytes; only the C++ server-
side decoder/encoder pair changes (from hand-rolled NamedUnion to
codegen wire types + roundtrip-to-domain).
Followup (PR-F): drop the hand-written types in bbapi_*.hpp and inline
the per-field conversions into the handlers; delete bbapi_execute.cpp.
PR-E's handlers used msgpack_roundtrip (pack-then-unpack) for every
command. This works but pays an extra serialize/deserialize round-trip
per call. PR-F replaces that with explicit field-by-field conversion
for the ~30 simple cryptographic primitives (Poseidon2/Pedersen/Blake2s/
AES, Grumpkin/Bn254/Secp256{k1,r1}, Schnorr/ECDSA, SrsInit), where the
conversion is mechanical and trivially auditable.
The chonk/ultra_honk/circuit/avm/batch-verifier handlers (~20) keep
msgpack_roundtrip — their wire/domain types are intricate aggregates
(CircuitInput, ProofSystemSettings, ChonkProof) and the extra pack pair
is acceptable for these non-hot-path commands.
bbapi_wire_convert.hpp adds:
- field_to_wire/field_from_wire<F> — templated over field type (bb::fr,
bb::fq, grumpkin::fr, grumpkin::fq, secp256k1::fr/fq, secp256r1::fr/fq).
- fr_to_wire / fr_from_wire — bb::fr convenience wrappers.
- fr_wrap / fr_unwrap — std::array<uint8_t, 32> <-> wire Fr.
- {grumpkin,bn254_g1,secp256k1,secp256r1}_point_{to,from}_wire — affine
point <-> {x: Fr, y: Fr} struct.
- bn254_g2_point_{to,from}_wire — uses msgpack_roundtrip (G2 has field2
components that aren't worth a hand-written converter).
- array_from_vec<N> / vec_from_array<N> — fixed-size <-> length-prefixed
byte buffer (AES iv/key 16 bytes; Reduce512 input 64 bytes).
Codegen schema_visitor.ts: the alias resolver now maps the family of
field-element aliases (fr, fq, secp256k1_fr/fq, secp256r1_fr/fq) to the
fr primitive (wire Fr struct), not bytes (std::vector<uint8_t>). This
makes the wire types consistent — every "32-byte field element" appears
as Fr regardless of whether the schema author wrote ["alias","fr","bin32"]
or ["array","unsigned char",32]. Wire-format-neutral: both encodings
collapse to msgpack bin via Fr's adapter (PR-D).
Net effect: hot-path crypto handlers no longer pay the
pack+unpack-pair-per-call cost; wire/domain field mismatches surface
at compile time (e.g. forgotten field rename, wrong field type). The
hand-written domain types in bbapi_{crypto,ecc,ecdsa,schnorr,srs}.hpp
still exist — handlers populate them and call their execute() bodies.
Deleting those types means inlining all execute() bodies into handlers
and refactoring c_bind.cpp (CBIND/WASM entry point) and ipc.bench.cpp
to use the codegen dispatch — left as PR-G.
Verified:
- ninja bb bb-avm aztec-wsdb bbapi_tests api_tests ipc_runtime_tests — clean
- ./bin/bbapi_tests — 30/30 pass
- ./bin/ipc_runtime_tests — 2/2 SHM pass
- ipc-codegen ./bootstrap.sh test — 18/18 cross-language matrix green
- Wire bytes unchanged (every Fr-bearing field still encodes as bin32).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #23522 → #23499 → #23498 → #23497 → #23316.
api_msgpack.cpp's hand-rolled msgpack dispatch lambda (~80 LOC of length-prefix decode + NamedUnion convert + bbapi() call + error wrapping) is replaced by `make_bb_handler(request)` from the codegen-emitted bb_ipc_server.hpp. Same wire format, same in-process semantics — BBApiRequest is kept at server-function scope so IVC state is preserved across the call sequence (ChonkStart → ChonkLoad → ChonkAccumulate → ChonkProve).
Codegen wiring
Added C++ codegen step in `bbapi/CMakeLists.txt` (analogous to wsdb's). Emits `generated/bb_{types,ipc_client,ipc_server}.hpp` + `bb_ipc_client.cpp` from `bb_schema.json`. `bbapi_objects` gets ipc_runtime as a PUBLIC link dep since the generated headers include `ipc_runtime/ipc_server.hpp`.
Handler implementation
The codegen-emitted `make_bb_handler` template instantiates a dispatch table that calls `handle_(BBApiRequest&, wire::Cmd&&)` overloads. Two new files provide them:
The wire types (from codegen) and domain types (`bbapi_*.hpp`) share a SERIALIZATION_FIELDS shape — same field names, msgpack-compatible field types (`Fr` packs as bin32 matching `bb::fr`; `GrumpkinPoint`/`Bn254G1Point`/etc. pack as {x, y} maps matching `affine_element`). So pack-then-unpack converts between them with no information loss. Slower than field-by-field copying (one extra pack+unpack pair per direction) but trivial to write and resistant to schema drift.
Test plan
What's deferred to PR-F
The hand-written types in `bbapi_{crypto,ecc,ecdsa,schnorr,chonk,ultra_honk,avm,srs}.hpp` still exist alongside the codegen-emitted wire types. PR-F will:
That's a mechanical follow-up; this PR establishes the wire-typed handler boundary as the contract.