tape test
Usage
tape test [<file.tape> | <dir>]... [options]Options go before or after the paths, in any order.
Examples
tape test # every test file under the current directory
tape test stdlib/ # every test file under stdlib/
tape test module.tape
tape test module.tape --native-only
tape test --filter "string" module.tape # only tests whose name contains "string"
tape test app.tape --substitute io=./mock_io.tape
tape test lib.tape -I deps/Which files
- A file is tested as it is; it must have a
#testregion. - A directory is searched recursively for test files: every
.tapefile with a#testregion. That covers a module’s inline tests and standalone test files alike (by convention<module>_test.tape, a file whose#codeimports what it tests). Directories whose name starts with., andout,buildandnode_modules, are skipped. The files run in path order. - No path means the current directory.
How it works
- Split source into regions
- Parse
#coderegion (with preamble) - Parse
#viewregion, merge components - Parse
#testregion → generates__test_*functions - Append test functions to the module’s function list
- Compile (comptime → typecheck → lower)
- Run each test in the VM, and again as a native program
- Run
setupbefore andteardownafter each test (if defined)
By default every test runs in both backends, and the two reports must be
identical: the same results and the same output, byte for byte. If they
differ, tape test prints the VM’s report, then the native one, and fails.
--vm-only or --native-only runs just one.
Each test starts from a fresh program state: every module variable at its
initial value. That covers the module’s own vars and lets and those of
every module it imports (numbers, flags, arrays, structs, strings,
optionals), each initializer run again, as a program started for that one
test would see them. What one test leaves in a variable never reaches the
next, so the tests don’t depend on their order, and --filter running one
test alone gives the same result as the full run. A table that a module
fills on first use behind a flag is filled again in each test that uses it.
The native run builds one executable holding the module, its tests and a
small test runtime. Each test runs in its own process, so a trap ends only
that test. For @assert_trap(expr) the program announces the expected trap
before evaluating expr; when the process then traps, that expectation
held, and the test runs again past it. The executable and copies of the
runtime libraries it links (from the test file’s directory, the current
directory or tape’s own) live in a temporary directory; the tests run in
the current directory, so relative paths behave as they do under
tape run. As with tape run, the environment variable TAPE holds the
path of the tape binary, so a test can start the compiler with
process (in both backends).
Options
| Flag | Description |
|---|---|
--filter <pattern> | Run only the tests whose name contains pattern |
--vm-only | Run the tests in the VM only |
--native-only | Run the tests as native code only |
--verbose | List every test with its result (by default only failed tests are listed) |
--substitute M=path | Replace module M with file at path (dependency injection; repeatable) |
--checked | Check indexing and slicing in t0 modules as in t1 (a trap with the source location instead of stray bytes); the code tape build makes stays unchecked |
-j <n> | Test up to n files at once (default 1); the reports still come in file order. See Several files at once |
-I <path> | Add module search path (repeatable, any number; see tape build) |
--target, --runtime, --uusi-syscall, -O0 | As for tape build; they apply to the native runs |
--tape-fuel <n> | The compile-time budget of each @tape block, as for tape build; applies to both runs |
--set NAME=VALUE | A build setting, as for tape build; passed on to both runs of every file. Each file is its own build: a --set naming a setting a file’s build doesn’t have is an error for that file (Build settings) |
--human | Human-readable error output |
--max-errors <N> | Stop after N errors |
--filter compares names the way the report shows them: every character
other than a letter, a digit or _ reads as _, so --filter "add one"
matches test "add one" (shown as add_one). A test the filter leaves out
is not run and not counted.
Substituting a module
--substitute M=path builds the tests with every import of M (the import’s
path, such as io or ./clock.tape, or its local name) resolved to path
instead, in both backends. It can be given several times (up to 16).
The substitute must keep the pub surface of the module it replaces: it
declares every pub item that module declares, with the same signature
(functions and methods with the same parameter and return types; structs,
unions and tagged unions with the same fields in the same order; enums with
the same variants and values; constants and globals of the same type;
components with the same props). A missing or different item is a compile
error pointing at the substitute. It may declare more pub items, for a test
to drive or inspect the mock:
// mock_clock.tape: replaces clock.tape, which declares pub fn now() -> i64
var fixed: i64 = 42;
pub fn now() -> i64 { return fixed; }
pub fn set(v: i64) { fixed = v; } // only the mock has thistape test app.tape --substitute ./clock.tape=mock_clock.tapeSeveral files at once
tape test dir -j 8 keeps eight files’ runs going at the same time, each
in its own process, and prints the reports in file order as they finish.
Every file is compiled on its own either way, so a directory of files that
share their imports gains about the number of cores. Tests of different
files then run at the same time, and under -j a file’s VM and native runs
do too: tests that bind a fixed port, or write a fixed path, can collide
with each other. Keep those files on -j 1, or give each test its own
port and path.
Where the time goes: TAPE_TIMING=1 tape test file.tape prints each stage
of the build on stderr (per module its parse, stub injection, compile-time
code, typecheck and lowering, then compile totals: over the modules), and
for a native run the link, dead-code elimination, the emit, and the tests’
own running time, with the file’s whole run last.
Output
By default the report lists the tests that failed, with what they printed, then the totals:
test "division_by_zero" ...
assertion failed: result == 42
expected: 42
got: 0
at math_test.tape:14
FAILED
results: 2 passed, 1 failed, 1 skipped, 4 totalA failure report ends with at <file>:<line>: where the @assert,
@fail, @assert_err or @assert_trap that failed is written, in the test
or in a helper fn of the #test region it called (the file as tape test
was given it).
With --verbose every test is listed with its result:
test "addition_works" ... ok
test "string_concat" ... ok
test "division_by_zero" ...
assertion failed: result == 42
expected: 42
got: 0
at math_test.tape:14
FAILED
test "skip_on_windows" ... skipped
results: 2 passed, 1 failed, 1 skipped, 4 totalWhen several files are tested (more than one named, or a directory), each file’s report comes under its path, then the total:
stdlib/str.tape
results: 16 passed, 0 failed, 0 skipped, 16 total
stdlib/tests/fs_test.tape
results: 9 passed, 0 failed, 0 skipped, 9 total
total: 2 files (0 failed): 25 passed, 0 failed, 0 skipped, 25 totalExit codes
| Code | Meaning |
|---|---|
| 0 | All tests passed (or skipped) |
| 1 | One or more tests failed, a file did not compile, the VM and native reports differ, or no test file was found |
| 2 | Usage error (an unknown option, an option without its value, --vm-only with --native-only) |
Test result values
Each test function returns an i64:
| Return value | Meaning |
|---|---|
| 0 | Pass |
| -1 | Fail (@assert failure or @fail) |
| -2 | Skipped (@skip condition was true) |
Last modified: