msgpack_Build_object(cleartext_buffer)

[smititelu]

Hi,

Supposing I'm having:

cleartext_buffer="{'foo': 12345, 'bar': { 'key1':['a', 'b', 'c'], 'key2':"string"}}"

Currently, is there such a C function that will build a struct msgpack_object from cleartext_buffer? It would be very useful because one can further call msgpack_pack_object(), thus packing the cleartext_buffer very easy.

Thanks,
Stefan

[redboltz]

No, the C part of msgpack-c doesn't have such function. Perhaps #393, #416 are related to this issue.

[smititelu]

Ok. I will try an implementation using Jansson library.

[smititelu]

Question: what kind of object is a MSGPACK_OBJECT_EXT ?

[redboltz]

Here are douments:
https://github.com/msgpack/msgpack/blob/master/spec.md#types-extension-type
https://github.com/msgpack/msgpack/blob/master/spec.md#formats-ext

[smititelu]

The idea that I have in order to msgpack_build_object() is to have an initial msgpack_object of type MSGPACK_OBJECT_MAP to which I can keep adding:

• key -> always a MSGPACK_OBJECT_STR
• value -> depending on what jansson lib returns(could be another MSGPACK_OBJECT_MAP)

I think of using already existing template_callback_str/map/array/int_x/true/false/nil

The questions I have now is how should I allocate memory for MSGPACK_OBJECT_STR objects? Should I use zones for this? If yes, should I use a zone per object?

[redboltz]

Sorry about my late reply.

The questions I have now is how should I allocate memory for MSGPACK_OBJECT_STR objects? Should I use zones for this? If yes, should I use a zone per object?

First, I answer about msgpack_object allocation, not MSGPACK_OBJECT_STR payload.

When msgpack-c unpacking from msgpack formatted byte stream, msgpack-c creates `msgpack_unpacked'. See https://github.com/msgpack/msgpack-c/blob/master/include/msgpack/unpack.h#L28

It contains one msgpack_zone pointer and the root msgpack_object. All children objects of root object are allocated on the same msgpack_zone. I think that creating msgpack_object from cleartext_buffer should be implemented by the same approach.

Next, how to treat MSGPACK_OBJECT_STR payload.

When msgpack-c unpacking from msgpack formatted byte stream, MSGPACK_OBJECT_STR payload doen't copy but is referenced.

See
https://github.com/msgpack/msgpack-c/wiki/v2_0_c_overview#using-unpack-function

If msgpack data contains STR, BIN, or EXT, unpacked msgpack_object refer to
msgpack data buffer. You need to keep the lifetime of the buffer during you use 
unpacked msgpack_object. Also you need to keep zone's lifetime. It's pretty 
complecated, so I recommend to use msgpack_unpack_next() or msgpack_unpacker 
instead of msgpack_unpack().

I think that when creating msgpack_object from cleartext_buffer, MSGPACK_OBJECT_STR payload should be copied on msgpack_zone. Because if the msgpack_object references cleartext_buffer, it's difficult to manage the lifetime of them. The msgpack_zone that is used as the destination of MSGPACK_OBJECT_STR payload copying should be the same as the one msgpack_object is located on. That means you need only one msgpack_zone for creating a msgpack_object from cleartext_buffer. You can use https://github.com/msgpack/msgpack-c/blob/master/include/msgpack/zone.h#L106