canary user’s guide

Getting started

• About canary
• Installing canary
• Getting help
• Getting started

Basics and usage

• canary basics
  • Canary basics: the Canary workspace
  • The test file
  • The Job object
  • Test statuses
  • Resource allocation
  • Time resources
• canary usage
  • Finding and listing test files
  • Describing a test file
  • Running tests
  • Filtering tests
  • Parameterizing tests
  • Getting the status of a test session
  • Inspecting log files
  • Rerunning tests
  • Rebaselining tests
  • Finding locations of test assets
  • Enabling/disabling tests
  • Copying and linking resource files into the test execution directory
  • The execute and analyze pattern
  • Marking a test expected to diff or fail
  • Loading modules in your test environment
  • Sourcing rc scripts during test execution
  • CMake, CTest, and CDash
  • Running tests via an HPC batch scheduler
  • VVTest test scripts
  • GitLab reporting

Configuration

• Configuration
  • Overview
  • Configuration file
  • Configuration settings
  • Setting configuration variables on the command line

Creating reports

• Reporting test results
  • JUnit style reports
  • Multi-page markdown report
  • Multi-page HTML report
  • JSON report

Extending canary through plugins

• Extending canary
  • Extend canary with plugins
  • User defined reports
  • User defined commands
  • User defined test generator

Commands and directives

• Command Reference
• Test Directives
• Utilities