Runner API

The runner module handles test discovery, execution, and result collection.

Test Discovery

discover_test_binaries

#![allow(unused)]
fn main() {
pub fn discover_test_binaries(
    project_root: &Path,
) -> Result<Vec<TestBinary>, NinetyNineError>
}

Discovers all test binaries in a Cargo project by running cargo test --no-run --message-format json-render-diagnostics and parsing the output.

Returns: A list of TestBinary structs, each containing the binary path, package name, and kind.

Errors: Returns BinaryDiscovery if cargo fails or output cannot be parsed.

list_tests_parallel

#![allow(unused)]
fn main() {
pub async fn list_tests_parallel(
    binaries: &[TestBinary],
    concurrency: usize,
) -> Result<Vec<TestCase>, NinetyNineError>
}

Lists all tests across multiple binaries concurrently. Each binary is invoked with --list --format terse and output lines ending with : test or : benchmark are parsed.

Uses tokio::sync::Semaphore for concurrency control.

Errors: Returns TestListing if a binary fails to produce test listings.

cargo_available

#![allow(unused)]
fn main() {
pub fn cargo_available() -> bool
}

Returns whether cargo is on PATH. The native runner builds test binaries via cargo test --no-run and executes them directly, so cargo is the only external tool required.

Test Execution

Executor

#![allow(unused)]
fn main() {
pub struct Executor<'a> {
    config: &'a ExecutionConfig,
}
}

Runs individual test cases with retry support.

Constructor:

#![allow(unused)]
fn main() {
pub fn new(config: &'a ExecutionConfig) -> Self
}

run_single

#![allow(unused)]
fn main() {
pub fn run_single(
    &self,
    test_case: &TestCase,
) -> Result<TestResult, NinetyNineError>
}

Executes a single test case with retries. Spawns the test binary with the --exact flag targeting the specific test.

Retry behavior:

• Retries up to config.retries times on failure
• Stops immediately on first pass
• Applies config.retry_delay between attempts

Timeout: Uses polling-based detection (50ms intervals). Kills the process when the deadline is exceeded, returning TestOutcome::Timeout.

Outcome classification:

ExecutionConfig

#![allow(unused)]
fn main() {
pub struct ExecutionConfig {
    pub concurrency: usize,
    pub timeout: Duration,
    pub retries: u32,
    pub retry_delay: Duration,
}
}

TestResult

#![allow(unused)]
fn main() {
pub struct TestResult {
    pub test_case: TestCase,
    pub outcome: TestOutcome,
    pub duration: Duration,
    pub stdout: String,
    pub stderr: String,
    pub attempt: u32,
}
}

Test Case Types

TestCase

#![allow(unused)]
fn main() {
pub struct TestCase {
    pub name: TestName,
    pub binary_path: PathBuf,
    pub binary_name: String,
    pub package_name: String,
    pub binary_kind: BinaryKind,
    pub kind: TestKind,
}
}

TestKind

#![allow(unused)]
fn main() {
pub enum TestKind {
    Test,
    Benchmark,
}
}

BinaryKind

#![allow(unused)]
fn main() {
pub enum BinaryKind {
    Lib,
    Bin,
    Test,
    Example,
}
}

Derived from Cargo metadata target kinds.

TestBinary

#![allow(unused)]
fn main() {
pub struct TestBinary {
    pub path: PathBuf,
    pub package_name: String,
    pub binary_name: String,
    pub kind: BinaryKind,
}
}

High-Level Runner

NativeRunner

#![allow(unused)]
fn main() {
pub struct NativeRunner { /* private fields */ }
}

Constructor:

#![allow(unused)]
fn main() {
pub fn new(project_root: &Path, config: ExecutionConfig) -> Self
}

Methods:

RunnerBackend

#![allow(unused)]
fn main() {
pub enum RunnerBackend {
    Native(NativeRunner),
}
}

Extensible enum wrapping runner implementations. Currently supports native Cargo test execution.

Methods: native(), execution_config(), discover_tests() — all delegate to the inner NativeRunner.

Standalone Function

execute_iterations

#![allow(unused)]
fn main() {
pub fn execute_iterations(
    test_case: &TestCase,
    iterations: u32,
    config: &ExecutionConfig,
    environment: &TestEnvironment,
) -> Result<Vec<TestRun>, NinetyNineError>
}

Convenience function that runs a test for N iterations and converts results to TestRun records. Used by the main command handler.