video

tape
import video from "video";

var v: video.Video = video.open_file("movie.mp4") or {
    io.println("cannot open it");
    return 1;
};
defer v.close();
let info: video.Info = v.info();
io.println("{info.width}x{info.height}, {info.duration_us / 1000000} s");

// frames in presentation order
while (true) {
    let f: ?video.Frame = v.next_frame() or {
        io.println("{v.last_error()}");      // a damaged picture: go on
        continue;
    };
    if (f == null) { break; }               // the end
    var frame: video.Frame = f!;
    frame.to_surface(&surface, 0, 0, info.width, info.height) or { };
    show(surface, frame.pts_us());
    frame.release();
}

// the sound, interleaved f32
var out: [2048]f32;
let c: video.AudioChunk = v.next_audio(out[0..]) or { return 1; };
// c.frames samples a channel, the first at c.pts_us; 0 frames at the end

video opens a video file and decodes it: frames in presentation order with their times, which to_surface converts and scales onto a gfx.Surface, and the sound as f32 samples for audio. It does not play anything itself: a player decides when to show each frame and writes the samples to an audio stream. The same API on every platform; today it decodes in software (MP4 with H.264 video and AAC sound).

FunctionDoes
open(data) -> VideoA file in memory (keep data alive while the Video is used)
open_file(path) -> VideoA file, read as it is needed
v.info() -> Infowidth, height, duration_us, frame_rate, has_video, has_audio, audio_rate, audio_channels, codec, backend
v.next_frame() -> ?FrameThe next frame; null at the end
v.next_audio(out) -> AudioChunkThe next samples, as many as fit (up to 1024 a channel); frames 0 at the end
v.seek(time_us) -> i64Go to the key frame at or before a time; where it lands
v.last_error(), v.close()What went wrong; give everything back
f.pts_us(), f.width(), f.height()A frame’s time and display size
f.to_surface(dst, x, y, w, h)The frame converted into a rectangle of a surface, scaled
f.release()The frame is no longer needed

Decoding happens in the call: next_frame takes a few milliseconds for a large frame, so a window program decodes from a task and keeps a few frames ahead. Hold at most 32 frames at once. The sound’s priming and padding are cut as the file’s edit list says.

Errors: UnknownFormat (not an MP4), Unsupported (no H.264 and no AAC, or a stream the decoders refuse: last_error says what), Corrupt (a damaged file or picture; decoding goes on after it), Unreadable, OutOfMemory.

Last modified: