json

tape
import json from "json";

let doc: json.Value = json.parse(r"""{"name": "tape", "tags": ["vm", "native"]}""") or {
    io.println("bad JSON: {err}");
    return 1;
};
let name: []const u8 = doc.get("name")!.as_str()!;     // "tape"
let first: []const u8 = doc.get("tags")!.at(0)!.as_str()!;   // "vm"
io.println(json.stringify(&doc));    // {"name":"tape","tags":["vm","native"]}

json reads and writes JSON exactly as RFC 8259 defines it: no comments, no trailing commas, no NaN, well-formed UTF-8 only. Numbers are read and printed exactly (the nearest f64, and the shortest text that reads back as the same number). Objects keep their members in the order they were written.

There are two ways in:

  • json.parse builds the whole document as a json.Value tree you can query, change and print.
  • json.Tokenizer hands you the document one token at a time and allocates nothing, for documents too large to hold or for reading straight into your own types.

json is a pure module.

Values

tape
pub tagged Value {
    Null;
    Bool { v: bool; }
    Number { v: f64; }
    Str { v: string; }
    Array { items: Vec_Value; }
    Object { fields: Map_string_Value; keys: collections.Vec_string; }
}

A json.Value is a tagged union that owns its contents: assigning one copies the whole tree, and it is freed when its variable goes out of scope. Build scalars with the variant literals and containers with json.array() and json.object():

tape
var user: json.Value = json.object();
user.set("name", json.Value.Str { v: "Ada" });
user.set("age", json.Value.Number { v: 36.0 });
var roles: json.Value = json.array();
roles.push(json.Value.Str { v: "admin" });
user.set("roles", roles);
user.set("active", json.Value.Bool { v: true });
user.set("manager", json.Value.Null);

push and set store a copy. set on a key that’s already there replaces its value and keeps its place; new keys go last. Don’t build an Object literal yourself: set keeps fields and keys in step.

match works as for any tagged union:

tape
fn describe(v: *const json.Value) -> string {
    match (*v) {
        json.Value.Number { v: n } => { return "number {n}"; }
        json.Value.Str { v: s } => { return "string {s}"; }
        _ => { return "{v.kind_name()}"; }
    }
    return "";
}

Reading values

MethodReturns
v.get(key)The object member key as a ?*const Value, or null (missing, or v isn’t an object)
v.at(i)The array element i as a ?*const Value, or null (out of range, or not an array)
v.as_str()The string as a ?[]const u8 view into v, or null
v.as_f64()The number, or null
v.as_i64()The number if it is a whole number that fits i64 (2.0 yes, 2.5 no), or null
v.as_bool()The boolean, or null
v.is_null()Whether v is null
v.len()Elements of an array, members of an object, 0 otherwise
v.kind_name()"null", "bool", "number", "string", "array" or "object"
v.items()Iterates an array’s elements (as *const Value)
v.members()Iterates an object’s members in order, as Member { key; value; }
v.equals(&w)Whether two values are equal: same kind and contents, object members in any order

Accessors never stop the program: a wrong kind or a missing key is null. Use ! when the document’s shape is guaranteed, and check for null when it isn’t:

tape
let port: i64 = doc.get("server")!.get("port")!.as_i64()!;

var seconds: f64 = 30.0;
let timeout: ?*const json.Value = doc.get("timeout");
if (timeout != null && timeout!.as_f64() != null) { seconds = timeout!.as_f64()!; }

for (m in doc.members()) {
    io.println("{m.key}: {m.value.kind_name()}");
}

The pointers and views these return point into the tree, read-only, so they work on a let document: they’re valid until you change that part of it.

Changing values

MethodDoes
v.push(item)Appends a copy of item to an array
v.set(key, item)Sets an object member to a copy of item
v.remove(key)Removes an object member; false if it wasn’t there
v.get_mut(key), v.at_mut(i)A member or element as a ?*Value, to change it in place
v.items_mut()Iterates an array’s elements as *Value, to change them

They stop the program with a trap when v is the wrong kind: push needs an array, set and remove an object. These need a var document. Nested values change through the pointers get_mut and at_mut return:

tape
doc.get_mut("owner")!.set("name", json.Value.Str { v: "Grace" });
doc.get_mut("tags")!.push(json.Value.Str { v: "json" });

Printing

FunctionReturns
json.stringify(&v)Compact JSON: {"a":[1,true],"b":"x"}
json.stringify_pretty(&v, indent)One element or member per line, indent spaces (0 to 16) per level
json.quote(s)s as a JSON string literal, quotes and escapes included

Numbers print in their shortest exact form (2, 0.1, 1e21), the same text "{x}" gives. NaN and infinity have no JSON form and print as null. Strings escape ", \ and control characters; other text, non-ASCII included, is written as is, and bytes that aren’t valid UTF-8 become U+FFFD.

tape
io.println(json.stringify_pretty(&user, 2));
text
{
  "name": "Ada",
  "age": 36,
  "roles": [
    "admin"
  ],
  "active": true,
  "manager": null
}

Printing and parsing round-trip: json.parse(json.stringify(&v)) gives a value equal to v.

Structs

json.gen_codec(T) generates the code that turns your struct into a json.Value and back, from its fields:

tape
enum Role : u8 { Admin = 1; Guest = 2; }
struct Pos { x: i64; y: i64; }
struct User {
    name: string;
    age: i32;
    role: Role;
    home: Pos;
    manager: ?string;
    scores: [3]f64;
    active: bool = true;
}

@tape {
    json.gen_codec(Pos);      // first: User's codec uses it
    json.gen_codec(User);
}

let u: User = User { name: "Ada", age: 36, role: Role.Admin, home: Pos { x: 1, y: 2 },
                     manager: null, scores: [1.5, 2.0, 3.0] };
io.println(json.stringify(&to_json_User(&u)));
// {"name":"Ada","age":36,"role":"Admin","home":{"x":1,"y":2},"manager":null,"scores":[1.5,2,3],"active":true}

let doc: json.Value = json.parse(text) or { return 1; };
let back: User = from_json_User(&doc) or {
    io.println("not a user: {err}");
    return 1;
};

Which fields are saved, and under which names, is up to serial’s markings: @serial.skip leaves a field out (it then may have any type), @serial.rename("hp") saves it under another name, @serial.explicit saves only the fields marked @serial.field.

It emits two functions into your module:

FunctionDoes
to_json_T(v: *const T) -> json.ValueAn object with one member per field, in declaration order
from_json_T(j: *const json.Value) -> T or json.JsonErrorThe struct back from such an object
from_json_T_at(j: *const json.Value, at: *string) -> T or json.JsonErrorThe same, and on an error *at says where: things[3].material

A field can be a bool, any integer, f32 or f64, a string, an enum (written as its variant’s name), a struct whose codec you generated before, or a ?S (null when it has none), [N]S (an array of N) or Vec_S (an array of any length, from collections.gen_vec) of those. Any other field type, such as a pointer, a slice or a tagged union, is a compile error naming the field, unless the field is marked @serial.skip. For a list whose length the file decides, use a Vec_S: it owns its elements.

tape
pub struct Thing { name: string; size: f64; }     // pub: gen_vec's methods are pub

@tape {
    collections.gen_vec(Thing);
    json.gen_codec(Thing);
    json.gen_codec(Scene);
}

struct Scene {
    title: string;
    things: Vec_Thing;
    @serial.optional lights: collections.Vec_i64;   // may be left out: empty
}

var at: string = "";
var scene: Scene = from_json_Scene_at(&doc, &at) or {
    io.println("scene file: {err} at {at}");       // a value of the wrong type at things[3].size
    return 1;
};
// ... and scene.things.free(), scene.lights.free() when done

Decoding is forgiving about what it doesn’t need and strict about what it does:

  • Members the struct doesn’t have are ignored.
  • A missing member gives the field its default (active above), or its zero when the field is marked @serial.optional (an empty Vec), and fails with MissingField otherwise.
  • A member of the wrong kind, a name that isn’t one of the enum’s variants, a fractional number for an integer field or an array of the wrong length fails with WrongType.
  • An integer that doesn’t fit the field’s type (300 for a u8) fails with NumberOutOfRange.

Numbers are f64 in JSON, so an integer beyond 2^53 loses precision.

A field can also be a coroutine’s frame: it is saved where the coroutine is, and loads to carry on from there (coroutines).

Frames

json.frame_writer(&obj) and json.frame_reader(&obj) are JSON’s serial Writer and Reader, for @frame_save and @frame_load by hand (a frame field in gen_codec uses them for you):

tape
var doc: json.Value = json.object();
var fw: json.FrameWriter = json.frame_writer(&doc);
var w: serial.Writer = fw.writer();
@frame_save(&walk, &w) or { return 1; };
// {"layout":"...","point":"step","point_layout":"...","args":{"n":4},"locals":{"$0":4,"total":1,"i":1}}

var fr: json.FrameReader = json.frame_reader(&doc);
var r: serial.Reader = fr.reader();
@frame_load(&walk, &r) or { return 1; };

Each record is an object and each value a member; an integer f64 can’t hold exactly (a 64-bit hash) is written as its decimal text. Keep the FrameWriter or FrameReader where it is while its writer() or reader() is in use: they point at it.

Errors

json.parse returns Value or JsonError:

JsonErrorWhen
UnexpectedEndThe text ends early: empty, an unclosed string, array or object
UnexpectedCharA character that doesn’t belong: [1,], {'a':1}, [1 2]
InvalidNumberA malformed number: 01, 1., 1e, -
InvalidStringA control character in a string, a bad escape like \x, a lone surrogate \ud800
TooDeepArrays and objects nested deeper than the limit (128 by default)
InvalidUtf8A string with bytes that aren’t valid UTF-8
NumberOutOfRangeA number too large for f64, like 1e400 (and, decoding a struct, an integer its field can’t hold)
TrailingDataSomething after the document: {} x
WrongTypeDecoding a struct (§Structs): a value of the wrong kind for its field
MissingFieldDecoding a struct: a member that the struct needs is missing

To find out where, and what, use a json.Parser, which has the position and, for an UnexpectedChar, what it found and what it wanted:

tape
var p: json.Parser = json.parser(text);
let v: json.Value = p.parse() or {
    io.println("{err} at line {p.line()}, column {p.col()}: {p.message()}");
    // unexpected character at line 5, column 1: a '}' where a member name should be: JSON allows no comma before '}'
    return 1;
};

p.offset() is the byte offset; p.line() and p.col() count from 1, columns in characters; p.message() is "" for the other errors, whose name says it. json.parse itself returns the error alone: a JsonError is a plain value with no position in it, by design. p.set_max_depth(n) raises or lowers the nesting limit (1 to 1024) before parsing.

A few choices RFC 8259 leaves open: a repeated key keeps its last value (in the first one’s place); a byte order mark at the start is an error; numbers too small for f64 become 0.

Streaming

A json.Tokenizer reads the text one token at a time. It checks the whole grammar as it goes and allocates nothing, so it can read documents of any size and fill your own structures directly:

tape
var t: json.Tokenizer = json.tokenizer(text);
while (true) {
    let tok: json.Token = t.next() or {
        io.println("{err} at {t.line()}:{t.col()}");
        return 1;
    };
    if (tok.kind == json.TokenKind.End) { break; }
    if (tok.kind == json.TokenKind.Key && bytes.eq(tok.text, "id")) {
        let value: json.Token = t.next() or { return 1; };
        let id: f64 = value.number() or { return 1; };
    }
}

Each Token has a kind (BeginObject, EndObject, BeginArray, EndArray, Key, Str, Number, True, False, Null, End), its text (a view of the input: a string’s raw contents between the quotes, a number as written) and its offset. Keys and strings may contain escapes: tok.unescaped() decodes them into a new string, tok.unescape_to(buf) into your buffer. tok.number() converts a number.

t.skip(tok) skips the rest of the value that tok starts (a whole nested array or object), t.depth() tells how deep you are. After an error, next keeps returning it; after End, it keeps returning End. The tokenizer accepts exactly the documents json.parse does, except numbers too large for f64, which only number() notices.

Last modified: