Vectors

A vector holds 16 bytes of integer lanes and works on all of them at once. Nothing is vectorized for you: a loop over u32 stays a loop over u32, a loop over u32x4 handles four at a time.

TypeLanes
u8x16, i8x1616 of u8 / i8
u16x8, i16x88 of u16 / i16
u32x4, i32x44 of u32 / i32
u64x2, i64x22 of u64 / i64

Making one

tape
let a: u32x4 = [1, 2, 3, 4];          // one element per lane, lane 0 first
let b: u32x4 = @splat(u32x4, 7);      // every lane 7
var px: [8]u32;
let vp: *u32x4 = (&px[0]) as *u32x4;  // a pointer to vectors
let c: u32x4 = *vp;                   // px[0..4]
let d: u32x4 = vp[1];                 // px[4..8]

Vectors live in locals and expressions and behind pointers. They are not struct fields, array elements, parameters, results or globals: pass a *u32x4 instead.

Operations

tape
let s: u32x4 = a + b;                 // lane by lane, wrapping
let m: u32x4 = (a * b) >> 3;          // shifts take one scalar count
let x: u8x16 = ~(s as u8x16);         // `as` reinterprets the same 128 bits
var v: u32x4 = s;
v[2] = 9;                              // one lane
let lane: u32 = v[2];
*vp = v;                               // back to memory
  • + - * & | ^ ~ and unary - work lane by lane and wrap like integers; * is on 16- and 32-bit lanes.
  • << and >> shift every lane by a scalar; >> is arithmetic on signed lanes. A count at or past the lane width gives 0 (or the sign).
  • Both operands have the same vector type: use @splat for a scalar.
  • A lane index outside the vector traps, like an array index.

Builtins

BuiltinResult
@splat(T, x)every lane x
@widen_lo(v), @widen_hi(v)the low or high half of the lanes at twice the width (sign-extended for signed lanes)
@narrow(a, b)the lanes of a then b at half the width, saturated
@narrow(T, a, b)signed lanes (i16x8, i32x4) to unsigned T (u8x16, u16x8), clamped to 0 .. max
@eq(a, b), @lt(a, b), @gt(a, b)a mask: all ones in each lane where it holds, else 0
@min(a, b), @max(a, b)in each lane the smaller / larger (8-, 16- and 32-bit lanes)
@min_lanes(v), @max_lanes(v)the smallest / largest lane, as the lane type (8-, 16- and 32-bit lanes)
@any(v), @all(v)bool: some lane / every lane is not 0

== and < are not defined on vectors; compare with @eq and @lt. Unsigned lanes compare unsigned, in @lt, @min and @max_lanes alike.

tape
// The largest of n i32, four at a time (n a multiple of 4, at least 4)
fn max_of(p: *i32, n: i64) -> i32 {
    var m: i32x4 = *(p as *i32x4);
    var i: i64 = 4;
    while (i < n) {
        m = @max(m, *((&p[i]) as *i32x4));
        i = i + 4;
    }
    return @max_lanes(m);       // the four running maxima down to one
}

Example

tape
// Four pixels at a time: every 8-bit channel scaled by a / 256
fn scale(dst: *u32, n: i64, a: u16) {
    let av: u16x8 = @splat(u16x8, a);
    var i: i64 = 0;
    while (i + 4 <= n) {
        let p: *u32x4 = (&dst[i]) as *u32x4;
        let s: u8x16 = *p as u8x16;
        let lo: u16x8 = (@widen_lo(s) * av) >> 8;
        let hi: u16x8 = (@widen_hi(s) * av) >> 8;
        *p = @narrow(lo, hi) as u32x4;
        i = i + 4;
    }
}

The VM, x86-64 (SSE2) and arm64 (NEON) give the same results bit for bit.

Choosing without a branch

@select(c, a, b) is a when c is true, else b, for vectors as for integers, bools and enums: both are evaluated and the choice is mask arithmetic, never a branch (what constant-time code needs, see Attributes).

tape
let r: u32x4 = @select(swap, b, a);

Processor features

SSE2 and NEON are always there. Some instructions are not: @cpu_has(f) asks the processor running the program, with f one of aes, clmul, sha, sse41, sse42, avx2, rdrand, rdseed (x86-64), neon (arm64) and crc32 (rdrand and rdseed are for the #bytecode instructions). The answer is read once at start-up (cpuid on x86-64, getauxval on Linux arm64, sysctlbyname on macOS arm64; the VM asks the same way), so asking in a loop is cheap.

tape
if (@cpu_has(aes)) {
    encrypt_hw(keys, block);
} else {
    encrypt_portable(keys, block);
}

AES and carry-less multiply

BuiltinResult
@aes_enc(s, k)one AES round: ShiftRows, SubBytes, MixColumns, then ^ k
@aes_enc_last(s, k)the last round: ShiftRows, SubBytes, ^ k
@aes_dec(s, k)one decryption round (equivalent inverse cipher): InvShiftRows, InvSubBytes, InvMixColumns, ^ k
@aes_dec_last(s, k)InvShiftRows, InvSubBytes, ^ k
@aes_imc(k)InvMixColumns: an encryption round key made a decryption one
@aes_keygen_assist(k, rcon)the key-schedule helper: SubWord and RotWord of words 1 and 3, rcon (a literal 0..255) XORed into words 1 and 3
@clmul_lo(a, b), @clmul_hi(a, b)the 128-bit carry-less product of lane 0 (or lane 1) of two u64x2

The AES builtins take and give u8x16 (byte i is row i % 4, column i / 4, as in FIPS-197); the carry-less multiplies u64x2. They behave exactly like x86-64’s AES-NI and PCLMULQDQ on both x86-64 and arm64 (arm64’s aese adds the round key first; the compiler arranges the difference), and the VM computes the same bits in portable code.

tape
// AES-128: rk holds the 11 round keys
fn encrypt(rk: *u8x16, p: *u8x16, c: *u8x16) {
    var s: u8x16 = *p ^ rk[0];
    for (i in 1..10) { s = @aes_enc(s, rk[i]); }
    *c = @aes_enc_last(s, rk[10]);
}

On a processor without AES (or CLMUL for @clmul_*) a builtin stops the program with trap: CPU lacks feature aes at file:line:col (exit code 101), in tape run and in a native build alike. Inside if (@cpu_has(aes)) { ... } the compiler leaves that check out, since it cannot fail there.

Last modified: