A second test

This example introduces two common patterns:

• staging an input file into the test working directory with canary.directives.link(); and

• running an external program using Executable.

The test file

In this example, the external program add.py adds two numbers and prints the result to standard output. The test links add.py into the working directory, executes it, and checks the output for correctness:

# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT

import sys

import canary

canary.directives.keywords("basic", "second")
canary.directives.link("add.py")


def test():
    print("Verifying that 2 + 3 = 5")
    import os

    print(os.getcwd())
    add = canary.Executable(f"{sys.executable} ./add.py")
    result = add("2", "3", stdout=str)
    assert int(result.get_output()) == 5, "Bummer, test failed."
    print("Test passed!")


if __name__ == "__main__":
    sys.exit(test())

Running the test

From the examples directory, run only tests with the second keyword:

$ canary run -k second ./basic
INFO: Initializing empty canary workspace at .
INFO: Collecting generator files from basic
INFO: Instantiating generators from collected files
INFO: Generating test specs from generators
INFO: Searching for duplicated tests
INFO: Resolving test spec dependencies
INFO: Generated 2 test specs from 2 generators
INFO: Caching test specs
INFO: Selecting specs based on 1 rules
INFO: Selected 1 test specs
INFO: Excluded 1 test spec during selection
                                                     
  Reason                                      Count  
 ─────────────────────────────────────────────────── 
  keyword expression 'second' did not match       1  
                                                     
INFO: Created selection 'warm-boulder'
INFO: Selecting test jobs based on runtime environment
INFO: Starting session 2026-06-04T20-42-25.374812
INFO: Starting process pool with max 1 workers
Job     ID        Status                                          Elapsed      Rank  
───────────────────────────────────────────────────────────────────────────────────
second  b1f18b4   SUBMITTED                                                     1/1  
second  b1f18b4   STARTED                                                       1/1  
second  b1f18b4   PASS (SUCCESS)                                     0.7s       1/1  
INFO: 1/1 tests finished with status PASS
INFO: Finished session in 0.76 s. with returncode 0
INFO: Updating view at /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/latest/src/canary/examples/TestResults

Here, -k filters the session to tests matching the given keyword expression.

Inspecting test output

When a test runs, its captured console output is written to canary-out.txt in the test’s working directory. The canary log command locates and prints that file:

$ canary log second
ERROR: second: no matching test job found in /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/latest/src/canary/examples/.canary

Contents of add.py

#!/usr/bin/env python
# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT
import argparse


def add(a: int, b: int) -> int:
    return a + b


if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("a", type=int)
    p.add_argument("b", type=int)
    args = p.parse_args()
    print(add(args.a, args.b))