dorotaC6d ago

What's a good #format to serialize simple structured #data in an efficient way?

I'm thinking about some binary #XML encoding. Also, #ASN.1 because it natively uses integers, not just strings. Is the packed representation (PER) widely supported?

The data will be a bunch of interleaved time series streams, so variable field lengths could be a big win.

Is there a format for stuff like that?

#Matroska's EBML? Only I'd have to hack the schema in.

#mkv #serialization #formats

Show threadnick6d ago
@dcz Protobufs? Flat buffers? Cap'n Proto? Apache Thrift?
Show threaddorotaC6d ago

@nickzoic There are even some no-std implementations:

https://docs.rs/capnp/latest/capnp/

https://github.com/YuhanLiin/micropb

https://raw.githubusercontent.com/ahsanu123/firmoto/ae9b772c4fa6ac4675f165e728279937c8112663/nostd_playground/src/bin/nostd_firmoto_schema_usage.rs

Now I REALLY don't know what to choose.

capnp - Rust

Cap’n Proto Runtime Library

Show threaddorotaC6d ago

@nickzoic Definitely not flatbuffers:

> There is an experimental support only in C++ for a vector of unions

Show threaddorotaC

@nickzoic Protobuf is also out:

> A oneof cannot be repeated.

https://protobuf.dev/programming-guides/proto3/#oneof-features

Language Guide (proto 3)

Covers how to use the proto3 revision of the Protocol Buffers language in your project.

Mar 19 at 7:16amWeb
Show threadmaxime borges6d ago
@dcz cap'n is the less stupid one in the list, the API is not great in Rust but it forces you to know exactly what you're doing, and it's as close to zero cost as possible