protobuf

import pb from "protobuf";
import str from "str";

struct Point {
    @pb.field(1) x: i32;
    @pb.field(2) y: i32;
    @pb.field(3) label: string;
}

@tape { pb.gen_codec(Point); }

let p: Point = Point { x: 3, y: -4, label: "corner" };
var out: str.Builder = str.builder();
to_protobuf_Point(&p, &out);            // 08 03 10 fc ff ff ff ff ff ff ff ff 01 1a 06 63 6f 72 6e 65 72
let back: Point = from_protobuf_Point(out.as_view()) or {
    io.println("not a point: {err}");
    return 1;
};
out.free();

protobuf reads and writes protocol buffers’ binary wire format, proto3: what gRPC carries, and what protoc’s generated code, Go’s, Rust’s prost and every other protobuf library read and write. Your structs say which field has which number; protobuf.gen_codec writes the code that turns them into bytes and back.

It does not read .proto files. Write the structs to match the schema, by hand or with a tool of your own that reads the schema: a field’s number and type are all the wire needs. Text format and protobuf’s JSON mapping are not here either.

protobuf is a pure module.

Structs

Mark each field with its number, @protobuf.field(n), then call the generator, after the generators of the structs it holds:

import pb from "protobuf";
import collections from "collections";

enum Status : i32 { Unknown = 0; Up = 1; Down = 2; }

pub struct Endpoint {
    @pb.field(1) host: string;
    @pb.field(2) port: u32;
}

@tape { collections.gen_vec(Endpoint); }

struct Service {
    @pb.field(1) name: string;
    @pb.field(2) status: Status;
    @pb.field(3) endpoints: Vec_Endpoint;
    @pb.field(4) @pb.zigzag offset: i64;       // sint64
    @pb.field(5) started: ?pb.Timestamp;
}

@tape {
    pb.gen_codec(Endpoint);       // first: Service's codec uses it
    pb.gen_codec(Service);
}

It emits two functions into your module:

Encoding can’t fail. A nested message is written with its exact length in one pass over the value, nothing copied: the generator also emits a size function for each struct, and a few helpers whose names start with __protobuf_.

Which fields take part is up to serial: a field marked @serial.skip is neither written nor read, and may have any type; a @serial.explicit struct takes only its @serial.field fields. Every field that takes part needs a number, from 1 to 536870911 but not 19000..19999 (protobuf reserves those); two fields with one number, a number out of range or a field without one is a compile error naming the struct and the field.

The attributes:

Types

A negative int32 takes ten bytes on the wire, as protobuf defines it: use @protobuf.zigzag for fields that are often negative.

Presence. proto3 doesn’t write a field that holds its zero, and reads a missing one as zero. So the zero values are what a struct’s fields should default to; a field with another default keeps it when the field is missing. A ?S field is the way to tell “not set” from zero: it is written whenever it isn’t null, and read as null when missing. A plain struct field is always written, even empty; make it a ?S to leave it out, as prost does with every message field.

Lists. A Vec_S reads any number of elements, in order, from packed records and single ones alike. An [N]S reads up to N, more is an error, fewer leaves the rest as the struct’s defaults have them; it is always written whole.

Maps. Keys are string, bool or an integer type; values any type of the table but a list, a map or a ?S. Each entry is written with its key and value, even zero ones. Reading, a key seen twice keeps the last value. A Map is a hash map, so its entries come out in no particular order: the bytes of a message with a map of two entries or more may differ from run to run (they decode the same).

Oneofs. A oneof is a ?U field without a number, U a tagged union whose every variant holds exactly one value and carries the number:

tagged Target {
    @pb.field(4) Host { name: string; }
    @pb.field(5) Addr { ip: u32; }
    @pb.field(6) @pb.zigzag Offset { by: i64; }   // marks go on the variant
}

struct Route {
    @pb.field(1) path: string;
    target: ?Target;              // fields 4, 5, 6 of Route
}

The variant that is set is written, even when its value is zero; reading, the last of the oneof’s fields wins.

Enums. An enum is written as its variant’s value, so give a tape enum the numbers of the proto enum it stands for (: i32 for negative ones). A value no variant has reads as the variant of value 0: proto3 keeps unknown enum values, a tape enum can’t. If the enum has no variant 0 it is OutOfRange instead. Declare the field i32 to keep every value.

Strings are not checked for UTF-8, either way: a proto string and bytes are both a tape string. Check with utf8 when the data isn’t yours, since a peer may refuse a string field that isn’t UTF-8.

Unknown fields are skipped, so a newer peer’s messages read fine. They are not kept. A field that appears twice is the last one, for messages too: they are not merged.

Errors

from_protobuf_T fails with a protobuf.ProtobufError, and gives back what it had decoded so far:

Timestamp

protobuf.Timestamp is google.protobuf.Timestamp: seconds since the Unix epoch and nanoseconds. Use it as a field type like any other; protobuf.timestamp(seconds, nanos) makes one, and time converts. Timestamp.write, size and protobuf.timestamp_read write and read one on its own.

Reading by hand

For a message you’d rather walk yourself, a protobuf.Reader goes field by field over the bytes. It allocates nothing: bytes() gives views into the data.

var r: pb.Reader = pb.reader(data);
while (!r.done()) {
    let t: pb.Tag = r.tag() or return;
    match (t.field) {
        1 => { let name: []const u8 = r.bytes() or return; }
        2 => { let port: u64 = r.varint() or return; }
        _ => { r.skip(t.wire) or return; }      // a field you don't need
    }
}

Writing by hand, put_varint, put_sint, put_fixed32, put_fixed64, put_tag and put_bytes append to a str.Builder, and varint_len and tag_len say how many bytes a varint or a tag takes.

Last modified: 2026-10-27 09:25:47 UTC