A second test

The test file

In this second example, the external program “add.py” adds two numbers and writes the result to the console’s stdout:

# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT

import sys

import canary

canary.directives.keywords("basic", "second")
canary.directives.link("add.py")


def test():
    print("Verifying that 2 + 3 = 5")
    import os

    print(os.getcwd())
    add = canary.Executable(f"{sys.executable} ./add.py")
    result = add("2", "3", stdout=str)
    assert int(result.get_output()) == 5, "Bummer, test failed."
    print("Test passed!")


if __name__ == "__main__":
    sys.exit(test())

In the test, add.py is linked to the execution directory, is executed, and output verified for correctness.

This test introduces two new features:

• canary.directives.link(): links add.py into the execution directory (see Test Directives for more directives); and

• Executable: creates a callable wrapper around executable scripts.

Running the test

To run the test, navigate to the examples folder and run:

$ canary run -k second ./basic
INFO: Initializing empty canary workspace at .
INFO: Collecting generator files from basic
INFO: Instantiating generators from collected files
INFO: Generating test specs from generators
INFO: Searching for duplicated tests
INFO: Resolving test spec dependencies
INFO: Generated 2 test specs from 2 generators
INFO: Caching test specs
INFO: Selecting specs based on 1 rules
INFO: Selected 1 test specs
INFO: Excluded 1 test spec during selection
                                                     
  Reason                                      Count  
 ─────────────────────────────────────────────────── 
  keyword expression 'second' did not match       1  
                                                     
INFO: Created selection 'lively-apex'
INFO: Selecting test cases based on runtime environment
INFO: Starting session 2026-04-21T15-34-06.394245
INFO: Starting process pool with max 2 workers
Job     ID        Status                                           Queued   Elapsed      Rank  
─────────────────────────────────────────────────────────────────────────────────────────────
second  57a9775   SUBMITTED                                                               1/1  
second  57a9775   STARTED                                            0.3s                 1/1  
second  57a9775   PASS (SUCCESS)                                     0.3s      0.6s       1/1  
INFO: 1/1 tests finished with status PASS
INFO: Finished session in 0.64 s. with returncode 0
INFO: Updating view at /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/TestResults

Here, the -k flag was used to select only the tests having the second keyword.

Inspecting test output

When a test is run, its output is captured to the file canary-out.txt in its execution directory. The canary log command can find and print the contents of this file to the console:

$ canary log second
/home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second/canary-out.txt:
[2026-04-21-15:34:06.658027] Creating workspace root at /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second
[2026-04-21-15:34:06.658141] Preparing test: second
[2026-04-21-15:34:06.658141] Directory: /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second
[2026-04-21-15:34:06.658141] Linking and copying working files...
[2026-04-21-15:34:06.658141] Linking /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/basic/second/second.pyt to /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second
[2026-04-21-15:34:06.658141] Linking /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/basic/second/add.py to /home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second
[2026-04-21-15:34:06.659441] Begin executing second/second
Verifying that 2 + 3 = 5
/home/docs/checkouts/readthedocs.org/user_builds/canary-wm/checkouts/release-26.4.16/src/canary/examples/.canary/sessions/2026-04-21T15-34-06.394245/second/second
Test passed!

Contents of add.py

#!/usr/bin/env python
# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT
import argparse


def add(a: int, b: int) -> int:
    return a + b


if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("a", type=int)
    p.add_argument("b", type=int)
    args = p.parse_args()
    print(add(args.a, args.b))