canary hpc#

Manage and run job batches on an HPC scheduler

canary hpc [-h]  ...

Optional arguments

-h, --help

Show this help message and exit.

Subcommands

canary hpc run#

canary hpc run [-h] [-w] [-f file] [-o option] [-k expression] [--owner OWNERS] [-p expression] [--regex regex]
           [--tag TAG] [--only {changed,not_pass,failed,not_run,all}] [--fail-fast] [-P {permissive,pedantic}]
           [--copy-all-resources] [--empty-ok] [-s CONSOLE_STYLE] [--workers N] [--timeout type=T]
           [--no-incremental] [--backend BACKEND] [--scheduler-args ARGS] [--batch-spec SPEC]
           [--batch-workers WORKERS] [--batch-timeout-strategy STRATEGY] [--queue-timeout T]
           ...

Positional arguments

pathspec [–] [user args…]

Test file[s] or directories to search. See ‘canary help –pathspec’ for help on the path specification

Optional arguments

-h, --help

show this help message and exit

-w

Remove test execution directory, if it exists [default: None]

-f file

Read test paths from a json or yaml file. See ‘canary help –pathfile’ for help on the file schema

-o option

Turn option(s) on, such as ‘-o dbg’ or ‘-o intel’

-k expression

Restrict selection to tests matching expression. For example: -k ‘key1 and not key2’. The keyword :all: matches all tests

--owner OWNERS

Restrict selection to tests owned by ‘owner’

-p expression

Restrict selection to tests matching the paramter expression. For example: ‘-p cpus=8’ or ‘-p cpus<8’

--regex regex

Restrict selection to tests containing the regular expression regex in at least 1 of its file assets. regex is a python regular expression, see https://docs.python.org/3/library/re.html

--tag TAG

Name this selection ‘tag’

--only {changed,not_pass,failed,not_run,all}
Which tests to run after selection

all - run all selected tests, even if already passing failed - run only previously failing tests not_run - run tests that have never been executed changed - run tests that whose specs have newer modification time not_pass - run tests whose status is not ‘SUCCESS’ (default)

--fail-fast

Stop after first failed test [default: None]

-P {permissive,pedantic}, --parsing-policy {permissive,pedantic}

If pedantic (default), stop if file parsing errors occur, else ignore parsing errors

--copy-all-resources

Do not link resources to the test directory, only copy [default: None]

--empty-ok

Exit normally when the test set is empty. By default, an empty test set is an error (exit code 7)

-s CONSOLE_STYLE, --style CONSOLE_STYLE

Configure live console display style. Given as key=value pairs: live={yes,no}[yes]: live console updating name={short,long}[short]: print short (default) names or long

--workers N

Execute the test session asynchronously using a pool of at most N workers

--timeout type=T

Set the timeout for type (accepts Go’s duration format, eg, 40s, 1h20m, 2h, 4h30m30s). • type=**session**, the timeout T is applied to the entire test session. • type=**multiplier**, the multiplier T is applied to each test’s timeout. • type=*****, the timeout T is applied to all test cases. • type=**batch**, choices for T are ‘conservative’ to use a conservative estimate for batch timeouts (queue times) or ‘aggressive’. Otherwise, a timeout of T is applied to tests having keyword type. For example, –timeout fast=2 would apply a timeout of 2 seconds to all tests having the ‘fast’ keyword; common types are fast, long, default, and ctest.

--no-incremental

Don’t use the .canary_cache to infer testcase runtimes

--backend BACKEND, --scheduler BACKEND

Submit batches to this HPC scheduler [alias: -b backend=BACKEND] [default: None]

--scheduler-args ARGS

Comma separated list of options to pass directly to the scheduler [alias: -b options=ARGS]

--batch-spec SPEC

Comma separated list of options to partition test cases into batches. See canary batch help –spec for help on batch specification syntax [alias: -b spec=SPEC]

--batch-workers WORKERS

Run test cases in batches using WORKERS workers [alias: -b workers=WORKERS]

--batch-timeout-strategy STRATEGY

Estimate batch runtime (queue time) conservatively or aggressively [alias: -b timeout=STRATEGY] [default: aggressive]

--queue-timeout T

Maximum time to wait in queue [alias: -b queue_timeout=T] [default: 30min]

canary hpc exec#

canary hpc exec [-h] [--workers WORKERS] [--backend CANARY_HPC_BACKEND] [--case CANARY_HPC_CASE] --workspace
            CANARY_HPC_WORKSPACE

Optional arguments

-h, --help

show this help message and exit

--workers WORKERS

Run tests in batch using this many workers

--backend CANARY_HPC_BACKEND

The HPC connect backend name

--case CANARY_HPC_CASE

Run only this case

--workspace CANARY_HPC_WORKSPACE

The batch’s workspace

canary hpc info#

canary hpc info [-h] backend

Positional arguments

backend

Show information on this backend

Optional arguments

-h, --help

show this help message and exit

canary hpc help#

canary hpc help [-h] [--spec]

Optional arguments

-h, --help

show this help message and exit

--spec

Help on the batch specification syntax