Testing an annotator

This page tests the FollowupAnnotator of Writing an annotator, the class from “A minimal annotator” on that page. Every example below is part of a pytest module that runs in GAIn’s own test suite, and it uses the declared API only. The test module defines FollowupAnnotator and build_followup_annotator exactly as that page shows them.

Registering in a test

A pipeline resolves an annotator’s type through the annotator registry. An installed package puts its type there through an entry point (Annotator plugins). A test runs the pipeline in its own process, so it does not need an installed entry point: register_annotator_factory() adds the annotator factory to the registry of that process. Call it at import time of the test module, so the type is registered before the first test loads a pipeline:

register_annotator_factory("followup", build_followup_annotator)

The “Discovering” section of Annotator plugins says why this is not a substitute for the entry point in a package meant to be installed.

A unit test with a seeded context

The followup annotator reads only the context, so its unit test needs no genomic resource. The pipeline loader still takes a repository: an empty directory repository in tmp_path is enough. The test loads a one-annotator pipeline and passes the context that the earlier annotators would have produced as the second argument of annotate():

def test_followup_answers_from_the_context(tmp_path: pathlib.Path) -> None:
    """The annotator answers from the context that the test seeds."""
    repo = grr.build_genomic_resource_repository({
        "id": "empty",
        "type": "directory",
        "directory": str(tmp_path / "grr"),
    })
    pipeline = load_pipeline_from_yaml(
        "- followup: {}", repo, work_dir=tmp_path / "work")

    with pipeline.open() as pipeline:
        allele = VCFAllele("chr1", 10, "A", "T")
        selected = pipeline.annotate(
            allele,
            {"phyloP7way": 1.5, "clinical_significance": "Pathogenic"})
        not_selected = pipeline.annotate(allele, {})
        no_annotatable = pipeline.annotate(None, {})

    assert selected == {
        "phyloP7way": 1.5,
        "clinical_significance": "Pathogenic",
        "followup": "yes",
    }
    assert not_selected == {"followup": "no"}
    assert no_annotatable == {"followup": None}

The pipeline answers the context with the annotator’s attributes merged into it. With the seeded context, the first answer is {'phyloP7way': 1.5, 'clinical_significance': 'Pathogenic', 'followup': 'yes'}. An empty context gives {'followup': 'no'}, and a None annotatable gives {'followup': None}.

An integration test over a resource

An integration test runs the annotator after the annotator that produces its input. Here a position_score_annotator produces phyloP7way from a position score that the test writes into tmp_path. The score table is a plain tab-separated file: format: tsv needs no bgzip and no tabix index. Position scores describes the keys of the genomic_resource.yaml.

def write_position_score(directory: pathlib.Path) -> None:
    """Write a ``phyloP7way`` position score as a plain TSV resource."""
    directory.mkdir(parents=True)
    (directory / "genomic_resource.yaml").write_text(textwrap.dedent("""
        type: position_score
        table:
          filename: phyloP7way.tsv
          format: tsv
          chrom:
            name: chrom
          pos_begin:
            name: pos_begin
          pos_end:
            name: pos_end
        scores:
        - id: phyloP7way
          type: float
          name: phyloP7way
    """))
    (directory / "phyloP7way.tsv").write_text(
        "chrom\tpos_begin\tpos_end\tphyloP7way\n"
        "chr1\t10\t10\t1.5\n"
        "chr1\t11\t11\t-0.5\n")

The repository is the directory that holds the resource directory, so the resource id is the name of that directory. The test loads a two-entry pipeline and asserts that the plugin read what the upstream annotator produced:

def test_followup_reads_the_upstream_score(tmp_path: pathlib.Path) -> None:
    """The annotator reads the score that the annotator before it wrote."""
    write_position_score(tmp_path / "grr" / "phyloP7way")
    repo = grr.build_genomic_resource_repository({
        "id": "local",
        "type": "directory",
        "directory": str(tmp_path / "grr"),
    })
    pipeline = load_pipeline_from_yaml(textwrap.dedent("""
        - position_score_annotator:
            resource_id: phyloP7way
        - followup: {}
    """), repo, work_dir=tmp_path / "work")

    with pipeline.open() as pipeline:
        conserved = pipeline.annotate(
            VCFAllele("chr1", 10, "A", "T"),
            {"clinical_significance": "Pathogenic"})
        not_conserved = pipeline.annotate(
            VCFAllele("chr1", 11, "C", "G"),
            {"clinical_significance": "Pathogenic"})

    assert conserved["phyloP7way"] == 1.5
    assert conserved["followup"] == "yes"
    assert not_conserved["phyloP7way"] == -0.5
    assert not_conserved["followup"] == "no"

What to assert

One assertion for each attribute that the annotator declares in get_attribute_specs().
The answer for a None annotatable: the base answers None for each attribute.
used_context_attributes, compared with the context keys that the annotator reads. A reannotation builds its dependency graph from this tuple, and a plain annotation run does not notice a missing name.

Another test of the module checks the declared attributes and the declared reads of the built annotator:

def test_followup_declares_its_context_reads(tmp_path: pathlib.Path) -> None:
    """The annotator declares one attribute and two context reads."""
    repo = grr.build_genomic_resource_repository({
        "id": "empty",
        "type": "directory",
        "directory": str(tmp_path / "grr"),
    })
    pipeline = load_pipeline_from_yaml(
        "- followup: {}", repo, work_dir=tmp_path / "work")

    annotator = pipeline.annotators[0]

    assert [attr.name for attr in annotator.attributes] == ["followup"]
    assert annotator.used_context_attributes == (
        "phyloP7way", "clinical_significance")

Unsupported fixtures

gain.genomic_resources.testing holds the fixture builders that GAIn’s own tests use: they are outside the declared API and can change without notice.