collect#

Collector module for discovering and instantiating test generators.

This module provides the infrastructure to scan various sources—including local filesystem paths and version control systems (Git and Google Repo)—to find files that match registered test generator patterns. Once discovered, these files are processed in parallel using a process pool to instantiate AbstractTestGenerator objects.

Usage patterns:

  1. Integration with the CLI: The Collector.setup_parser method adds arguments to allow users to specify scan paths via -r or a configuration file via -f.

  2. Programmatic usage: Use find_generators_in_path(path) for a simple way to retrieve all generators within a specific directory.

  3. Plugin-based extension: The module uses hooks (e.g., canary_collectstart) to allow plugins to modify the collection behavior or exclude specific directories.

recursedirs_type(arg: str) → dict[str, list[str]]#

Parses a path string into a root directory and optional file list.

Parameters:

arg – Path string, potentially in ‘root:path’ format.

Returns:

A dictionary mapping the root directory to a list of relative paths.

read_recursedirs(file: str) → dict[str, list[str]]#

Reads test paths from a JSON or YAML file.

Parameters:

file – Path to the configuration file.

Returns:

A dictionary mapping root directories to lists of relative paths.

class update_action(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)#

Bases: Action

Action to update a dictionary in the argparse namespace.

class Collector#

Bases: object

Collects and instantiates test generators from various sources.

static setup_parser(parser: Parser) → None#
run() → list[AbstractSpecGenerator]#

Executes the collection process.

Returns:

A list of instantiated test generators.

finalize() → None#

Instantiates generators from the collected files using a process pool.

collect_from_path(scanpath: ScanPath) → None#

Collects generator files from a local filesystem path.

Parameters:

scanpath – The path configuration to scan.

collect_from_vc(root: str) → None#

Collects generator files from a version control system.

Parameters:

root – The VC root string (e.g., ‘git@path’).

add_generator(generator: Type[AbstractSpecGenerator]) → None#

Adds a generator type to the collection criteria.

Parameters:

generator – The generator class to add.

property file_patterns: set[str]#

Returns the set of all file patterns associated with registered generator types.

add_skip_dirs(dirs: list[str]) → None#

Adds directories to be skipped during collection.

Parameters:

dirs – A list of directory names or patterns to skip.

skip(dirname: str) → bool#
add_scanpaths(scanpaths: dict[str, list[str]]) → None#

Adds multiple scan paths to the collector.

Parameters:

scanpaths – A dictionary of root paths and their associated relative paths.

add_scanpath(root: str, paths: list[str]) → None#

Adds a single scan path to the collector.

Parameters:

  • root – The root directory of the scan path.

  • paths – A list of relative paths to include.

iter_scanpaths() → Iterator[ScanPath]#

Iterates over all configured scan paths.

Returns:

An iterator of ScanPath objects.

add_file(root: str, path: str) → None#

Adds a single file to the collection.

Parameters:

  • root – The root directory.

  • path – The relative path to the file.

add_files(root: str, paths: list[str]) → None#

Adds multiple files to the collection.

Parameters:

  • root – The root directory.

  • paths – A list of relative paths to the files.

remove_file(root: str, path: str) → None#

Removes a file from the collection.

Parameters:

  • root – The root directory.

  • path – The relative path to the file.

iter_files() → Iterator[tuple[str, str]]#

Iterates over all collected files.

Returns:

An iterator of (root, path) tuples.

matches(f: str) → bool#

Checks if a filename matches any of the registered generator types.

Parameters:

f – The filename to check.

Returns:

True if it matches any generator type, False otherwise.

worker_init(snapshot: dict[str, Any])#

Initializes a worker process with a configuration snapshot.

Parameters:

snapshot – The configuration snapshot.

canary_collectstart(collector: Collector) → None#

Hook to configure the collector at the start of collection.

Parameters:

collector – The Collector instance.

git_ls(root: str, patterns: Iterable[str]) → list[str]#

Lists files in a git repository matching specific patterns.

Parameters:

  • root – Path to the git repository.

  • patterns – File patterns to match.

Returns:

A list of matching file paths.

repo_ls(root: str, patterns: Iterable[str]) → list[str]#

Lists files across all projects in a Google repo manifest.

Parameters:

  • root – Path to the repo root.

  • patterns – File patterns to match.

Returns:

A list of matching file paths across all projects.

class ScanPath(root: str, path: str | None = None)#

Bases: object

root: str#
path: str | None = None#
generate_one(args) → tuple[bool, AbstractSpecGenerator | None]#

Attempts to create a generator for a single file.

Parameters:

args – A tuple containing (generator_types, root, file_path).

Returns:

A tuple of (success_boolean, generator_instance_or_none).

find_generators_in_path(path: str | Path) → list[AbstractSpecGenerator]#

Convenience function to find and instantiate generators in a given path.

Parameters:

path – The path to scan.

Returns:

A list of instantiated AbstractTestGenerator objects.