A first test

This section walks through a minimal .pyt test: what it looks like, how to run it, and where to find results.

The test file

Consider the test file first.pyt:

# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT

import sys

import canary

canary.directives.keywords("basic")


def add(a: int, b: int) -> int:
    return a + b


def test():
    assert add(3, 2) == 5


if __name__ == "__main__":
    sys.exit(test())

This example also introduces a directive: canary.directives.keywords(). Directives are how a test file communicates metadata and configuration back to canary during test discovery and generation. The keywords directive assigns labels to a test, which can later be used for filtering (for example, selecting a subset of tests to run).

Note

Files ending in .pyt are Python scripts that use canary directives. The remainder of this tutorial uses .pyt examples unless noted otherwise.

Running the test

To run the test, navigate to examples/basic/first and execute:

$ canary run .
INFO: Initializing empty canary workspace at .
INFO: Collecting generator files from .
INFO: Instantiating generators from collected files
INFO: Generating test specs from generators
INFO: Searching for duplicated tests
INFO: Resolving test spec dependencies
INFO: Generated 1 test specs from 1 generators
INFO: Caching test specs
INFO: Created selection 'calm-warden'
INFO: Selecting test jobs based on runtime environment
INFO: Starting session 2026-06-04T20-42-21.675221
INFO: Starting process pool with max 1 workers
Job    ID        Status                                          Elapsed      Rank  
──────────────────────────────────────────────────────────────────────────────────
first  4b172fc   SUBMITTED                                                     1/1  
first  4b172fc   STARTED                                                       1/1  
first  4b172fc   PASS (SUCCESS)                                     0.6s       1/1  
INFO: 1/1 tests finished with status PASS
INFO: Finished session in 0.71 s. with returncode 0
INFO: Updating view at /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/latest/src/canary/examples/basic/first/TestResults

A test is considered successful if it exits with return code 0. See Test statuses for more details on statuses and failure modes.

Inspecting the results

Tests run inside a per-session workspace (stored under .canary). After execution finishes, canary writes a user-facing view of the session under TestResults. The session tree mirrors the source tree used to generate the session:

$ canary tree examples/basic/first/TestResults
examples/basic/first/TestResults
├── VIEW.TAG
└── first
    ├── canary-out.txt
    ├── first.pyt@
    └── testcase.lock

To see status information for the session, run canary status from within the session directory:

$ canary status
┌────────┬────────────┬──────────────────┬──────────────────────┬────────────────────┬────────────────┬────────────────┐
│ ID     │ Name       │ Session          │ Exit Code            │ Duration           │ Status         │ Details        │
├────────┼────────────┼──────────────────┼──────────────────────┼────────────────────┼────────────────┼────────────────┤
└────────┴────────────┴──────────────────┴──────────────────────┴────────────────────┴────────────────┴────────────────┘

By default, canary status prints details only for non-passing tests. To show all tests, including passing ones, use -rA:

$ canary status -rA
┌────────────┬─────────┬───────────────────────────────────┬──────────────┬────────────┬────────────────────┬──────────┐
│ ID         │ Name    │ Session                           │ Exit Code    │ Duration   │ Status             │ Details  │
├────────────┼─────────┼───────────────────────────────────┼──────────────┼────────────┼────────────────────┼──────────┤
│ 4b172fc    │ first   │ 2026-06-04T20-42-21.675221        │ 0            │ 0.31       │ PASS (SUCCESS)     │          │
└────────────┴─────────┴───────────────────────────────────┴──────────────┴────────────┴────────────────────┴──────────┘