A first test#

The test file#

Consider the test file first.pyt that defines a function to add two numbers and verifies it for correctness:

# Copyright NTESS. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: MIT

import sys

import canary

canary.directives.keywords("basic")

def add(a: int, b: int) -> int:
    return a + b

def test():
    assert add(3, 2) == 5

if __name__ == "__main__":
    sys.exit(test())

In addition to the test body, this test contains a “directive”: canary.directives.keywords(). Directives are the method for a test file to communicate back to canary. The keywords directive applies “keywords” (labels) to the test which can be used to peform certain filtering actions.

Note

This test file is an example of a .pyt test file - a Python file with canary directives.

Running the test#

To run the test, navigate to examples/basic/first and execute canary run .:

$ canary run .
INFO: Collecting generator files from .
INFO: Instantiating generators from collected files
INFO: Generating test specs from generators
INFO: Searching for duplicated tests
INFO: Resolving test spec dependencies
INFO: Generated 1 test specs from 1 generators
INFO: Caching test specs
INFO: Created selection 'quartz-crystal'
INFO: Selecting test cases based on runtime environment
INFO: Excluded 1 test cases
                                     
  Reason                      Count  
 ─────────────────────────────────── 
  previous result = SUCCESS       1  
                                     
ERROR: no cases to run

A test is considered to have successfully completed if its exit code is 0. See Test statuses for more details on test statuses.

Inspecting the results#

Test execution was conducted within the “workspace”. The workspace is created in .canary. After the tests are finished, a “view” of the workspace is created in TestResults. The test session tree mirrors the layout of the source tree used to generate the test session:

$ canary tree examples/basic/first/TestResults
ERROR: [Errno 2] No such file or directory: 'examples/basic/first/TestResults'
examples/basic/first/TestResults

Details of the session can be obtained by navigating to the test session directory and executing canary status:

By default, only details of the failed tests appear in the output. To see the results of each test in the session, including passed tests, pass -rA: