brotli
import brotli from "brotli";
let page: []const u8 = "<html>...</html>";
var br: collections.Vec_u8 = brotli.compress(page, 11) or { return 1; };
// br.slice() is a Content-Encoding: br body, or a .br file's bytes
let body: string = fs.read_file("index.html.br") or { return 1; };
var html: collections.Vec_u8 = brotli.decompress(body, 16 << 20) or {
io.println("not good brotli: {err}");
return 1;
};brotli reads brotli (RFC 7932),
the br of Content-Encoding. Browsers prefer it to gzip, and sites
often keep pre-compressed .br copies of their files. Every stream the
RFC allows decodes:
- every window size from 1 KiB to 16 MiB;
- the context modes and their trees;
- the 120 KiB static dictionary with its 121 word transforms;
- metadata and uncompressed blocks.
Use compress and decompress when the data fits in memory. Use an
Encoder or a Decoder when it arrives in pieces, such as an HTTP body:
they hold a bounded amount of memory however long the stream is.
Compressed output is deterministic: the same input at the same quality gives the same bytes, in the VM and natively, however the input is split into pushes. Any brotli decoder reads it. It is not the reference encoder’s output: at quality 11 it comes out about the size of gzip -9, where the reference encoder’s is 9–15% smaller.
Decompression is strict: anything that isn’t a valid stream is an error,
including bytes left over after the stream ends. brotli is a pure
module, separate from compress so that a program which
only needs gzip doesn’t carry the dictionary.
Errors
pub enum BrotliError : u8 {
BadData = 1: "not valid brotli data";
Truncated = 2: "brotli data ends early";
TooLarge = 3: "decompressed data over the limit";
BadQuality = 4: "brotli quality outside 0..11";
}BadData: the bytes are not a brotli stream. Causes include a broken prefix code, a distance before the start of the output that is not a dictionary word, non-zero padding, or bytes after the end.Truncated: the data ends before the stream does.TooLarge: the output would pass themax_outyou gave.BadQuality: compressing at a quality outside 0..11.
Whole buffers
compress
pub fn compress(data: []const u8, quality: i64) -> collections.Vec_u8 or BrotliErrorCompresses data at quality, from 0 (fastest) to 11 (smallest), as
brotli numbers it. Fails only with BadQuality. Data that doesn’t
compress comes out a few bytes larger than it went in.
let css: string = fs.read_file("site.css") or { return 1; };
var br: collections.Vec_u8 = brotli.compress(css, 11) or { return 1; };
fs.write_file("site.css.br", br.slice()) or { return 1; };decompress
pub fn decompress(data: []const u8, max_out: i64) -> collections.Vec_u8 or BrotliErrorDecompresses data, which must be exactly one brotli stream. max_out
is the most output you accept: a small input that would expand to
gigabytes stops with TooLarge at that size, and the buffer never grows
past it. Traps when max_out is negative.
Streams
pub fn decoder() -> Decoder
pub fn Decoder.push(*self, data: []const u8) -> i64 or BrotliError
pub fn Decoder.read(*self, dst: []u8) -> i64
pub fn Decoder.done(*const self) -> bool
pub fn Decoder.free(*self)A Decoder works like compress’s Inflater:
pushthe compressed bytes as they come. It returns how many it took: all of them, or fewer when 64 KiB of output is waiting.readthe output into your buffer. It returns 0 when none is waiting.- Repeat until
done().
An error stays: every later push returns it.
var z: brotli.Decoder = brotli.decoder();
defer z.free();
var buf: [16384]u8;
while (!z.done()) {
let piece: []const u8 = conn.read_some() or { return 1; };
var at: i64 = 0;
while (at < piece.len) {
at = at + (z.push(piece[at..piece.len]) or { return 1; });
var n: i64 = z.read(buf[0..buf.len]);
while (n > 0) {
out.write(buf[0..n]);
n = z.read(buf[0..buf.len]);
}
}
}To compress a stream:
pub fn encoder(quality: i64) -> Encoder or BrotliError
pub fn Encoder.push(*self, data: []const u8)
pub fn Encoder.finish(*self)
pub fn Encoder.output(*const self) -> []const u8
pub fn Encoder.advance_output(*self, n: i64)
pub fn Encoder.free(*self)An Encoder works like compress’s Deflater. push takes all the
bytes you give it. Compressed bytes collect in output(): send them,
then advance_output past what you sent. finish writes the end of the
stream. Calling push after finish, finish twice, or advancing past
output() traps.
var e: brotli.Encoder = brotli.encoder(5) or { return 1; };
defer e.free();
while (true) {
let piece: []const u8 = file.read_some() or { return 1; };
if (piece.len == 0) { break; }
e.push(piece);
conn.write(e.output()) or { return 1; };
e.advance_output(e.output().len);
}
e.finish();
conn.write(e.output()) or { return 1; };An encoder holds about 2 MiB: its 256 KiB window, the window’s hash chains and one meta-block’s commands.
A decoder’s memory is set by the stream’s window. It holds a ring of twice the window, up to 32 MiB for the largest; the ring grows from 128 KiB as output comes, so a small response stays small. It also holds up to 64 KiB of unread output and the codes of the current block.
Last modified: