parameterize#

parameterize(names: str | Sequence[str], values: Sequence[Sequence[Any] | Any], *, when: str | dict[str, str] | None = None, type: enums = enums.list_parameter_space, samples: int = 10, random_seed: float = 1234.0) → None#

Add new invocations to the test using the list of argvalues for the given argnames.

Usage#

.pyt:

import canary
canary.directives.parametrize(argnames, argvalues, when=..., type=None)

.vvt:

#VVT: parametrize (options=...,platforms=...,testname=...) : argnames = argvalues

Parameters#

  • argnames: A comma-separated string denoting one or more argument names, or a list/tuple of argument strings.

  • argvalues: If only one argname was specified, argvalues is a list of values. If N argnames were specified, argvalues is a 2D list of values where each column are the values for its respective argname.

  • when: Restrict processing of the directive to this condition

  • type: (.pyt only) Generate parameters using this type

  • samples: (.pyt only) Generate this many random elements (applicable only if type=canary.random_parameter_space)

The when expression is limited to the following conditions:

  • testname: Restrict processing of the directive to this test name

  • platform: Restrict processing of the directive to certain platform or platforms

  • option: Restrict processing of the directive to command line -o options

Special argnames#

  • cpus interpreted to mean “number of processing cores”. If the cpus parameter is not defined, the test is assumed to use 1 processing core.

  • gpus interpreted to mean “number of gpus”. If the gpus parameter is not defined, the test is assumed to use 0 gpus.

References#

Examples#

The following equivalent test specifications result in 4 test instantiations

test1.pyt:

# test1
canary.directives.parameterize("cpus", (4, 8, 12, 32))

test1.vvt:

# test1
#VVT: parameterize : np = 4 8 12 32

4 test cases:
├── test1[cpus=4]
├── test1[cpus=8]
├── test1[cpus=12]
├── test1[cpus=32]

argnames can be a list of parameters with associated argvalues, e.g.

test1.pyt:

# test1
canary.directives.parameterize("a,b", ((1, 2), (3, 4), (5, 6)))

test1.vvt:

# test1
#VVT: parameterize : a,b = 1,2 3,4 5,6

3 test cases:
├── test1[a=1,b=2]
├── test1[a=3,b=4]
├── test1[a=5,b=6]

parameterize can be called multiple times. When multiple parameterize directives are given, the Cartesian product of each is taken to form the set of parameters, e.g.

test1.pyt:

# test1
canary.directives.parameterize("a,b", [("a1", "b1"), ("a2", "b2")])
canary.directives.parameterize("x", ["x1", "x2"])

results in the following test invocations:

4 test cases:
├── test1[a=a1,b=b1,x=x1]
├── test1[a=a1,b=b1,x=x2]
├── test1[a=a2,b=b2,x=x1]
├── test1[a=a2,b=b2,x=x2]
On this page