mutation_testing — Logtalk 3.100.1

# `mutation_testing`

This tool provides mutation testing support for loaded Logtalk entities. It’s still under development and breaking changes should be expected.

## API documentation

This tool API documentation is available at:

../../apis/library_index.html#mutation-testing

## Loading

This tool can be loaded using the query:

    | ?- logtalk_load(mutation_testing(loader)).

## Testing

To test this tool, load the `tester.lgt` file:

    | ?- logtalk_load(mutation_testing(tester)).

## Features

- Deterministic mutant discovery per entity predicate directive, predicate/non-terminal clause/rule, and mutator.

- Configurable mutator sets and campaign size limits.

- Deterministic sampled execution (`sampling(all|count(N)|rate(R))` plus `seed/1`).

- Mutation score computation (killed versus survived mutants).

- Mutation generation guided by code coverage stats.

- Threshold gating suitable for CI/CD checks.

- Exporting of mutation campaign reports in plain text and JSON formats.

## Default mutators

The default mutators are:

- `fail_insertion`
  Inserts failure in the selected predicate/non-terminal rule bodies.
- `body_goal_negation`
  Negates the selected predicate/non-terminal clause behavior.
- `relational_operator_replacement`
  Replaces a relational operator (e.g. `>`, `<`, `@=<`, `==`) with a complementary alternative.
- `arithmetic_operator_replacement`
  Replaces an arithmetic operator in selected expressions.
- `truth_literal_flip`
  Flips truth literals (`true` <-> `fail`) in selected goals.
- `head_arguments_mutation`
  Mutates one compile-time bound head argument using `type::mutation/3`.
- `head_arguments_reordering`
  Reorders head arguments (swapping the first two arguments).
- `clauses_reordering`
  Reorders predicate/non-terminal clauses/rules by swapping a clause with its successor.
- `scope_directive_replacement`
  Replaces a matching predicate/non-terminal scope directive visibility with an alternative visibility.
- `predicate_directive_suppression`
  Suppresses a matching predicate/non-terminal directive.
- `uses_directive_resource_deletion`
  Deletes one matching resource from a `uses/2` directive.

## Usage

Mutation testing requires an existing set of tests for the code being mutated. These tests are used to verify if they are able to kill the generated mutants or if they are too weak allowing the mutants to survive.

By default, the tool looks for the tests driver file (usually, `tester.lgt`) in the directory of the entity, library, or directory being tested. If the tests driver file is not found there, the tool looks into the startup directory of the Logtalk process.

Both the name of the tests driver file and its directory can be set using the `tester_file_name/1` and `tester_directory/1` options if the defaults don’t work in your particular case.

In this tool documentation, campaign predicates refers to the following public predicates:

- `entity/1-2`

- `predicate/2-3`

- `library/1-2`

- `directory/1-2`

These predicates run mutants and report results in one step.

Run mutation testing for one entity using defaults:

    | ?- mutation_testing::entity(my_object).

List generated mutants for one entity:

    | ?- mutation_testing::entity_mutants(my_object, Mutants).

Run with custom options:

    | ?- mutation_testing::entity(my_object, [
        mutators([fail_insertion, body_goal_negation]),
        max_mutations_per_mutator(5),
        sampling(count(100)),
        seed(20260303),
        max_mutators(100),
        threshold(60.0),
        verbose(true),
        print_mutation(true)
    ]).

Run campaigns for all loaded entities from a specific library:

    | ?- mutation_testing::library(core).

Run campaigns for all loaded entities from a specific directory:

    | ?- mutation_testing::directory('/path/to/sources').

Pretty-print a report term using the default text format:

    | ?- mutation_testing::report_entity(my_object, Report, []),
         mutation_testing::format_report(Report).

Suppress report formatting output for campaign predicates:

    | ?- mutation_testing::entity(my_object, [format(none)]).

Execution uses `lgtunit` test sets only. When using `timeout(Timeout)`, timeout handling is best-effort.

When running mutation campaigns inside another `lgtunit` test run, per-mutant `killed` versus `survived` classification may be affected by nested test-run message and event handling. Campaign summary accounting remains deterministic.

## Limitations

- Mutation points are currently mostly clause-level (one occurrence per clause per mutator), except `head_arguments_mutation` which creates one occurrence per compile-time bound head argument with supported type; internal expression-level candidate enumeration is not implemented.

- Equivalent mutant detection and duplicate mutant pruning are not implemented.

- Built-in mutation apply/revert currently targets loaded source entities (objects/categories), not protocols.

- Nested `lgtunit` runs may affect per-mutant `killed` vs `survived` status classification.

## Options

- `include_entities(Entities)`
  List of loaded entities to include (default `[]` meaning all).
- `exclude_entities(Entities)`
  List of loaded entities to exclude (default `[]`).
- `max_mutators(Max)`
  Maximum number of discovered mutators to use when `mutators/1` is not explicitly provided (`all` or positive integer; default `all`).
- `max_mutations_per_mutator(Max)`
  Maximum number of mutations generated per mutator and predicate/non-terminal (`all` or positive integer; default `5`).
- `mutators(Mutators)`
  Names of the mutators to use. When set to `[]` (default), mutators are auto-discovered and optionally limited by `max_mutators/1`.
- `sampling(Sampling)`
  Mutant selection strategy (`all`, `count(N)`, or `rate(R)` where `0.0 =< R =< 1.0`; default `all`). See below for full details.
- `seed(Seed)`
  Pseudo-random generator seed for reproducible sampling (default `123456789`; ignored when `sampling(all)`).
- `timeout(Timeout)`
  Maximum time in seconds for each mutant execution (positive number; default `300`). Timeout handling is best-effort.
- `threshold(Threshold)`
  Minimum mutation score in range `0.0..100.0` (default `0.0`).
- `verbose(Boolean)`
  Print per-mutant results (default `false`).

- `format(Format)` Controls report formatting output (`none`, `text`, or `json`; default `text`). When set to `text` or `json`, a report file is generated automatically for campaign predicates. The `json` format implements the Stryker Mutation Testing Framework report format: https://stryker-mutator.io/

- `report_file_name(FileName)` Report output file base name or path without extension (atom; default `mutation_test_report`). The extension is inferred from `format/1` (`text` -> `.txt`, `json` -> `.json`). When not absolute, the file is saved in the tests driver directory.

- `print_mutation(Boolean)`
  When `true`, prints original and mutated terms with source location for mutators. This option is only effective when `verbose(true)` (default `false`).
- `tester_file_name(Tester)`
  Name of the tests driver file for the code being tested (default `tester.lgt`).
- `tester_directory(Directory)`
  Full path to the directory containing the tests driver file for the code being tested (no default).

## Sampling semantics

The `sampling(Sampling)` option controls which generated mutants are actually executed in a mutation campaign. It is a post-generation selection step:

1. Generate candidate mutants for the selected entities/predicates/mutators.

2. Apply `sampling(...)` to select a subset (or all).

3. Execute only the selected mutants.

Because of this design, `sampling(...)` affects campaign predicates (`entity/2`, `predicate/3`, `library/2`, `directory/2`).

The `format(Format)` option affects only campaign predicates that execute and report in one step (`entity/2`, `predicate/3`, `library/2`, `directory/2`). It does not affect the `*_mutants` predicates or the `report_*` predicates, which only compute and return terms.

For campaign predicates, a report file is written automatically unless `format(none)` is used.

For `report_library/3` and `report_directory/3`, call `format_report/2-3` explicitly if you want to persist the computed report term. A single `format_report(..., Report)` call always generates a single output document (including JSON).

The `mutants/2-3` predicates are still useful for inspecting the generated deterministic mutant list itself; they are not execution/reporting predicates.

### Sampling modes

- `sampling(all)` Execute all generated mutants.

- `sampling(count(N))` Shuffle the generated mutant list and execute up to `N` mutants. If `N` is greater than the total number of generated mutants, all mutants are executed.

- `sampling(rate(R))` Compute `round(R * TotalGeneratedMutants)`, shuffle the mutant list, and execute that many mutants. If the computed value is greater than the total number of generated mutants, all mutants are executed.

The randomization used by `count/1` and `rate/1` is controlled by the `seed/1` option, allowing reproducible selection.

### Sampling usage in practice

- Control campaign cost Run a representative subset when the full mutant set is large.

- Trade off speed vs. confidence Use smaller samples for quick feedback and larger/full runs for stronger confidence before release.

- Enable reproducible sampling With a fixed `seed/1`, repeated runs with the same generated mutant set and same sampling options select the same subset. When all mutants are executed, (using `sampling(all)`) changing the seed has no effect on selection.

- Complement other limits `max_mutators/1` and `max_mutations_per_mutator/1` constrain mutant generation; `sampling/1` controls execution selection after generation.

## Defining new mutators

Define a new mutator as parametric object implementing the `expanding` protocol, either the `clause_mutator_protocol` or the `directive_mutator_protocol` marker protocol, and importing the `mutator_common` category:

    :- object(my_mutator(_Entity_, _Predicate_, _TargetIndex_, _Occurrence_, _PrintMutation_),
        implements((expanding, clause_mutator_protocol)),
        imports(mutator_common)).

        % mutate code when loading it using this object as a hook object
        term_expansion(Term, Mutation) :-
            ...

        % compute a term mutation; generate multiple mutations by backtracking
        mutation(Term, Mutation) :-
            ...

        % reset any required internal state
        reset :-
            ...

    :- end_object.

New mutators are found by dynamically looking for objects that conform to either the `clause_mutator_protocol` or the `directive_mutator_protocol` marker protocols.

For implementation examples, see the default mutators:

- mutators/fail_insertion.lgt

- mutators/body_goal_negation.lgt

- mutators/relational_operator_replacement.lgt

- mutators/arithmetic_operator_replacement.lgt

- mutators/truth_literal_flip.lgt

- mutators/head_arguments_mutation.lgt

- mutators/head_arguments_reordering.lgt

- mutators/clauses_reordering.lgt

- mutators/scope_directive_replacement.lgt