compress

import compress from "compress";

let page: []const u8 = "<html>...</html>";
var gz: collections.Vec_u8 = compress.gzip(page, 6) or { return 1; };
// gz.slice() is a Content-Encoding: gzip body, or a .gz file's bytes

var back: collections.Vec_u8 = compress.gunzip(gz.slice(), 1 << 20) or {
    io.println("not a good gzip: {err}");
    return 1;
};

compress reads and writes deflate (RFC 1951) in its three usual framings:

Use the whole-buffer functions when the data fits in memory, and a Deflater or Inflater when it arrives in pieces (an HTTP body, a large file): they hold a bounded amount of memory however long the stream is.

Compressed output is deterministic: the same input at the same level gives the same bytes, in the VM and natively, on every platform, however the input is split into pushes. Decompression is strict: anything that isn’t a valid stream is an error, and checksums are always checked.

compress is a pure module. Brotli is its own module, brotli; zstd and archives (.zip, .tar) are not here.

Errors

pub enum CompressError : u8 {
    BadData = 1: "not valid compressed data";
    BadChecksum = 2: "compressed data failed its checksum";
    Truncated = 3: "compressed data ends early";
    TooLarge = 4: "decompressed data over the limit";
    BadLevel = 5: "compression level outside 1..9";
}

• BadData: the bytes are not a stream of the format: a wrong header, a reserved block type, a broken Huffman code, a back-reference to before the start of the output, or bytes left over after a raw or zlib stream ends.
• BadChecksum: the stream decoded but its Adler-32, CRC-32, length or gzip header CRC doesn’t match.
• Truncated: the data ends before the stream does.
• TooLarge: the output would pass the max_out you gave.
• BadLevel: compressing at a level outside 1..9.

Levels

level runs from 1 to 9 as in zlib and gzip: 1 is fastest, 9 is smallest, 6 is the usual default. Data that doesn’t compress (already compressed, or random) comes out in stored blocks, a few bytes larger than it went in.

On an x86-64 desktop, 100 KiB of English text compresses in about 2 ms at level 1, 6 ms at level 6 and 7 ms at level 9 (to 44%, 37% and 37% of its size); decompressing it takes about 1.5 ms.

Whole buffers

deflate

pub fn deflate(data: []const u8, level: i64, format: Format) -> collections.Vec_u8 or CompressError

Compresses data in format. Fails only with BadLevel.

var z: collections.Vec_u8 = compress.deflate(text, 9, compress.Format.Zlib) or { return 1; };
io.println("{text.len} -> {z.len} bytes");

inflate

pub fn inflate(data: []const u8, format: Format, max_out: i64) -> collections.Vec_u8 or CompressError

Decompresses data, which must be exactly one stream of format (for Gzip, one or more members one after another, as cat a.gz b.gz makes: they decompress as one). max_out is the most output you accept: a small input that would expand to gigabytes (a “decompression bomb”) stops with TooLarge at that size, and the buffer never grows past it. Traps when max_out is negative.

// a PNG's image data: no more than its rows can hold
var rows: collections.Vec_u8 = compress.inflate(idat, compress.Format.Zlib, height * (1 + row_bytes)) or {
    io.println("bad image data: {err}");
    return 1;
};

gzip

pub fn gzip(data: []const u8, level: i64) -> collections.Vec_u8 or CompressError

deflate(data, level, Format.Gzip). The header has no file name or comment, a modification time of 0 and “unknown” as the operating system, so the same data always gives the same file.

var gz: collections.Vec_u8 = compress.gzip(log_text, 6) or { return 1; };
fs.write_file("today.log.gz", gz.slice()) or { return 1; };

gunzip

pub fn gunzip(data: []const u8, max_out: i64) -> collections.Vec_u8 or CompressError

inflate(data, Format.Gzip, max_out). Reads every header gzip allows: file name, comment, extra field and header CRC are skipped (the CRC is checked).

let file: string = fs.read_file("today.log.gz") or { return 1; };
var text: collections.Vec_u8 = compress.gunzip(file, 64 << 20) or {
    io.println("today.log.gz: {err}");
    return 1;
};

crc32

pub fn crc32(data: []const u8) -> u32

The CRC-32 gzip’s trailer holds: the same function as hash.crc32, here so a program writing gzip framing needs one import. crc32("123456789") is 0xCBF43926.

adler32

pub fn adler32(data: []const u8) -> u32

The Adler-32 zlib’s trailer holds. adler32("Wikipedia") is 0x11E60398.

io.println("{compress.adler32("Wikipedia") == 0x11E60398}");   // true

Streams

The streams don’t read or write anything themselves: you push bytes in and take bytes out, so they work the same with a socket, a file or a buffer.

Deflater

pub fn deflater(level: i64, format: Format) -> Deflater or CompressError
pub fn Deflater.push(*self, data: []const u8) -> void or CompressError
pub fn Deflater.flush(*self)
pub fn Deflater.finish(*self)
pub fn Deflater.output(*const self) -> []const u8
pub fn Deflater.advance_output(*self, n: i64)
pub fn Deflater.free(*self)

deflater makes a compressor (BadLevel outside 1..9). push takes all of data; compressed bytes collect in output() as blocks complete. Send or copy what you can of output(), then advance_output(n) past the n bytes you used. finish writes the last block and the trailer; call it once, after the last push (pushing after it traps, and so does a second finish). flush is a sync flush (zlib’s Z_SYNC_FLUSH): everything pushed so far goes into output(), ending with an empty stored block (the bytes 00 00 FF FF), so the reader can decode all of it now, and the stream goes on with the same window; WebSocket’s permessage-deflate ends each message this way. It traps after finish. advance_output traps when n is more than output() holds. free releases the compressor’s memory: about 520 KiB, and more while a block of very repetitive input is gathered.

var d: compress.Deflater = compress.deflater(6, compress.Format.Gzip) or { return 1; };
defer d.free();
while (true) {
    let n: i64 = body.read(buf[0..buf.len]) or { return 1; };
    if (n == 0) { break; }
    d.push(buf[0..n]) or { return 1; };
    conn.write_all(d.output()) or { return 1; };
    d.advance_output(d.output().len);
}
d.finish();
conn.write_all(d.output()) or { return 1; };
d.advance_output(d.output().len);

Without flush, the output is the same bytes deflate gives for the whole input.

Inflater

pub fn inflater(format: Format) -> Inflater
pub fn Inflater.push(*self, data: []const u8) -> i64 or CompressError
pub fn Inflater.read(*self, dst: []u8) -> i64
pub fn Inflater.done(*const self) -> bool
pub fn Inflater.free(*self)

inflater makes a decompressor. push decodes from data and returns how many bytes it took: all of them, unless 64 KiB of output is waiting to be read, in which case read and push the rest. read copies decompressed bytes into dst and returns how many (0 when none are waiting); as it makes room it goes on decoding input already pushed. done is true once the stream has ended and its trailer was checked. An error is kept: every later push returns it (an error found while read decodes comes back from the next push). After done, a push is BadData for Raw and Zlib, and starts the next member for Gzip. free releases its memory (about 120 KiB).

The stream can’t say Truncated by itself (more data may come); when your input ends, check done().

var z: compress.Inflater = compress.inflater(compress.Format.Gzip);
defer z.free();
var out: [65536]u8;
while (true) {
    let n: i64 = conn.read(buf[0..buf.len]) or { return 1; };
    if (n == 0) { break; }
    var at: i64 = 0;
    while (at < n) {
        let took: i64 = z.push(buf[at..n]) or { io.println("{err}"); return 1; };
        at = at + took;
        while (true) {
            let k: i64 = z.read(out[0..out.len]);
            if (k == 0) { break; }
            sink.write(out[0..k]) or { return 1; };
        }
    }
}
if (!z.done()) { io.println("the body ended early"); return 1; }

Last modified: 2026-11-03 11:50:16 UTC