tape test

Usage

bash
tape test [<file.tape> | <dir>]... [options]

Options go before or after the paths, in any order.

Examples

bash
tape test                                   # every test file under the current directory
tape test stdlib/                           # every test file under stdlib/
tape test module.tape
tape test module.tape --native-only
tape test --filter "string" module.tape     # only tests whose name contains "string"
tape test app.tape --substitute io=./mock_io.tape
tape test lib.tape -I deps/

Which files

  • A file is tested as it is; it must have a #test region.
  • A directory is searched recursively for test files: every .tape file with a #test region. That covers a module’s inline tests and standalone test files alike (by convention <module>_test.tape, a file whose #code imports what it tests). Directories whose name starts with ., and out, build and node_modules, are skipped. The files run in path order.
  • No path means the current directory.

How it works

  1. Split source into regions
  2. Parse #code region (with preamble)
  3. Parse #view region, merge components
  4. Parse #test region → generates __test_* functions
  5. Append test functions to the module’s function list
  6. Compile (comptime → typecheck → lower)
  7. Run each test in the VM, and again as a native program
  8. Run setup before and teardown after each test (if defined)

By default every test runs in both backends, and the two reports must be identical: the same results and the same output, byte for byte. If they differ, tape test prints the VM’s report, then the native one, and fails. --vm-only or --native-only runs just one.

Each test starts from a fresh program state: every module variable at its initial value. That covers the module’s own vars and lets and those of every module it imports (numbers, flags, arrays, structs, strings, optionals), each initializer run again, as a program started for that one test would see them. What one test leaves in a variable never reaches the next, so the tests don’t depend on their order, and --filter running one test alone gives the same result as the full run. A table that a module fills on first use behind a flag is filled again in each test that uses it.

The native run builds one executable holding the module, its tests and a small test runtime. Each test runs in its own process, so a trap ends only that test. For @assert_trap(expr) the program announces the expected trap before evaluating expr; when the process then traps, that expectation held, and the test runs again past it. The executable and copies of the runtime libraries it links (from the test file’s directory, the current directory or tape’s own) live in a temporary directory; the tests run in the current directory, so relative paths behave as they do under tape run. As with tape run, the environment variable TAPE holds the path of the tape binary, so a test can start the compiler with process (in both backends).

Options

FlagDescription
--filter <pattern>Run only the tests whose name contains pattern
--vm-onlyRun the tests in the VM only
--native-onlyRun the tests as native code only
--verboseList every test with its result (by default only failed tests are listed)
--substitute M=pathReplace module M with file at path (dependency injection; repeatable)
--checkedCheck indexing and slicing in t0 modules as in t1 (a trap with the source location instead of stray bytes); the code tape build makes stays unchecked
-j <n>Test up to n files at once (default 1); the reports still come in file order. See Several files at once
-I <path>Add module search path (repeatable, any number; see tape build)
--target, --runtime, --uusi-syscall, -O0As for tape build; they apply to the native runs
--tape-fuel <n>The compile-time budget of each @tape block, as for tape build; applies to both runs
--set NAME=VALUEA build setting, as for tape build; passed on to both runs of every file. Each file is its own build: a --set naming a setting a file’s build doesn’t have is an error for that file (Build settings)
--humanHuman-readable error output
--max-errors <N>Stop after N errors

--filter compares names the way the report shows them: every character other than a letter, a digit or _ reads as _, so --filter "add one" matches test "add one" (shown as add_one). A test the filter leaves out is not run and not counted.

Substituting a module

--substitute M=path builds the tests with every import of M (the import’s path, such as io or ./clock.tape, or its local name) resolved to path instead, in both backends. It can be given several times (up to 16).

The substitute must keep the pub surface of the module it replaces: it declares every pub item that module declares, with the same signature (functions and methods with the same parameter and return types; structs, unions and tagged unions with the same fields in the same order; enums with the same variants and values; constants and globals of the same type; components with the same props). A missing or different item is a compile error pointing at the substitute. It may declare more pub items, for a test to drive or inspect the mock:

tape
// mock_clock.tape: replaces clock.tape, which declares pub fn now() -> i64
var fixed: i64 = 42;
pub fn now() -> i64 { return fixed; }
pub fn set(v: i64) { fixed = v; }     // only the mock has this
bash
tape test app.tape --substitute ./clock.tape=mock_clock.tape

Several files at once

tape test dir -j 8 keeps eight files’ runs going at the same time, each in its own process, and prints the reports in file order as they finish. Every file is compiled on its own either way, so a directory of files that share their imports gains about the number of cores. Tests of different files then run at the same time, and under -j a file’s VM and native runs do too: tests that bind a fixed port, or write a fixed path, can collide with each other. Keep those files on -j 1, or give each test its own port and path.

Where the time goes: TAPE_TIMING=1 tape test file.tape prints each stage of the build on stderr (per module its parse, stub injection, compile-time code, typecheck and lowering, then compile totals: over the modules), and for a native run the link, dead-code elimination, the emit, and the tests’ own running time, with the file’s whole run last.

Output

By default the report lists the tests that failed, with what they printed, then the totals:

plaintext
  test "division_by_zero" ...
    assertion failed: result == 42
      expected: 42
           got: 0
      at math_test.tape:14
FAILED

results: 2 passed, 1 failed, 1 skipped, 4 total

A failure report ends with at <file>:<line>: where the @assert, @fail, @assert_err or @assert_trap that failed is written, in the test or in a helper fn of the #test region it called (the file as tape test was given it).

With --verbose every test is listed with its result:

plaintext
  test "addition_works" ... ok
  test "string_concat" ... ok
  test "division_by_zero" ...
    assertion failed: result == 42
      expected: 42
           got: 0
      at math_test.tape:14
FAILED
  test "skip_on_windows" ... skipped

results: 2 passed, 1 failed, 1 skipped, 4 total

When several files are tested (more than one named, or a directory), each file’s report comes under its path, then the total:

plaintext
stdlib/str.tape

results: 16 passed, 0 failed, 0 skipped, 16 total

stdlib/tests/fs_test.tape

results: 9 passed, 0 failed, 0 skipped, 9 total

total: 2 files (0 failed): 25 passed, 0 failed, 0 skipped, 25 total

Exit codes

CodeMeaning
0All tests passed (or skipped)
1One or more tests failed, a file did not compile, the VM and native reports differ, or no test file was found
2Usage error (an unknown option, an option without its value, --vm-only with --native-only)

Test result values

Each test function returns an i64:

Return valueMeaning
0Pass
-1Fail (@assert failure or @fail)
-2Skipped (@skip condition was true)

Last modified: