gain.genomic_resources.testing package

Submodules

gain.genomic_resources.testing.ann_data_builder module

Fluent, immutable test-data builder for ann_data resources.

A sibling of gain.genomic_resources.testing.builders for the same reason data_frame_builder is one: builders sits two lines under pylint’s max-module-lines ceiling, so a new builder has nowhere to go inside it. The dependency runs ONE WAY – this module imports the shared single-realize seam from builders and builders does not import back – so an_ann_data is imported from here.

The axis this builder exists to vary is the FORMAT, exactly as DataFrameBuilder varies csv/tsv/xlsx: one authored pair of annotation tables realized as an h5ad, as a 10x Matrix Market triple in whichever of the two 10x layouts a test needs, or as a 10x-Genomics HDF5. Hand-rolling any of them per test is what makes ann_data tests tedious otherwise, and the 10x triple in particular has to be realized consistently across three files for the sidecar resolution to be worth testing at all.

Of the 10x_h5 format only ONE layout is realized: the modern single matrix group, feature barcode, one genome, which is what both such resources we have are. The legacy root-genome-group layout, the probe-barcode variant and multi-genome files are deliberately absent (#707) – see docs/adr/0014.

Like DataFrameBuilder, this exposes NO expected AnnData. Realizing an h5ad forces this builder to go through anndata itself, and handing that object back as a test’s oracle would check anndata against anndata on exactly the axes an ann_data test varies – builder and loader would have to be wrong in the same way for the test to stay green. Tests state their expectations independently.

class gain.genomic_resources.testing.ann_data_builder.AnnDataBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, obs_data: str | None = None, var_data: str | None = None, drop_obs_columns: bool = False, drop_var_columns: bool = False, file_format: str = 'h5ad', legacy_layout: bool = False, uncompressed_layout: bool = False, prefix: str = '', declared_format: str | None = None, omit_format_key: bool = False, filename: str | None = None, parameters: dict[str, Any] | None = None, omit_file_key: bool = False)

Bases: MetaMixin

Immutable builder for a single ann_data resource.

A bare builder realizes a valid minimal readable h5ad resource.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

declared_format: str | None = None

drop_obs_columns: bool = False

drop_var_columns: bool = False

file_format: str = 'h5ad'

filename: str | None = None

legacy_layout: bool = False

obs_data: str | None = None

omit_file_key: bool = False

omit_format_key: bool = False

parameters: dict[str, Any] | None = None

prefix: str = ''

realize_into(resource_dir: Path) → None

Write this ann_data resource into resource_dir.

uncompressed_layout: bool = False

var_data: str | None = None

with_declared_format(file_format: str) → AnnDataBuilder

Override the config’s format: only, leaving realization.

Unvalidated on purpose: this is how a test builds a resource declaring an unknown format, or one whose declared format disagrees with the bytes on disk.

with_file(filename: str) → AnnDataBuilder

Override the realized filename (default: per format).

with_format(file_format: str) → AnnDataBuilder

Select the realized format AND the declared format: key.

One of h5ad, 10x_mtx or 10x_h5; the realized filename follows. To declare a format that does NOT match what is on disk, use with_declared_format().

with_legacy_layout() → AnnDataBuilder

Realize the 10x triple in the CellRanger v2 layout.

matrix.mtx/barcodes.tsv/genes.tsv, all plain text – as against v3’s gzipped matrix.mtx.gz/barcodes.tsv.gz/ features.tsv.gz. The distinction is not cosmetic: it is the one scanpy itself probes for, and the sidecar resolution has to make the same call.

with_obs(data: str) → AnnDataBuilder

Author the obs (per-cell) table as a whitespace block.

The first column is the index; the block is normalized by convert_to_tab_separated.

with_parameters(parameters: dict[str, Any]) → AnnDataBuilder

Emit a parameters: block passed through to the reader.

with_prefix(prefix: str) → AnnDataBuilder

Give the 10x triple a shared filename prefix.

scanpy.read_10x_mtx addresses the triple as a directory plus the prefix its three members share, so a resource carrying more than one matrix distinguishes them this way.

with_uncompressed_layout() → AnnDataBuilder

Realize the v3 triple as plain text – the STARsolo layout.

matrix.mtx/barcodes.tsv/features.tsv, with the same three-column feature table v3 always has. This is NOT the legacy layout: it still carries feature types, so gex_only still has something to filter on.

with_var(data: str) → AnnDataBuilder

Author the var (per-gene) table as a whitespace block.

Four column names are conventional, because a 10x feature table names its fields by position or by dataset and this block is what has to fill them: gene_name is the gene symbol, feature_type is the feature type, and genome/interval are the per-feature metadata a 10x_h5 carries (the triple has nowhere to put them, and drops them). All are optional – see _feature_table() and _feature_metadata() for what stands in. An h5ad realization carries every authored column as an ordinary var column.

without_file_key() → AnnDataBuilder

Omit file: from the config, keeping the data file.

without_format_key() → AnnDataBuilder

Omit format:, exercising the loader’s suffix default.

without_obs_columns() → AnnDataBuilder

Keep the index but drop every obs annotation column.

The shape that makes describe of nothing an empty frame, which the implementation declines to write as a statistic.

without_var_columns() → AnnDataBuilder

Keep the index but drop every var annotation column.

gain.genomic_resources.testing.ann_data_builder.an_ann_data() → AnnDataBuilder

Return an immutable ann_data builder.

gain.genomic_resources.testing.builders module

Fluent, immutable test-data builders for GRR resources.

This module offers a small builder DSL for composing genomic resources into a filesystem GRR that a test can open and read back. Builders are immutable (frozen dataclasses); every with_* method returns a NEW builder, so a partly-configured builder can be shared across test variations without leaking state.

The builders assemble a pure in-memory recipe with no side effects; the build_* methods delegate the actual file writing and repository construction to the existing helpers in gain.genomic_resources.testing (setup_directories and build_filesystem_test_repository, or build_inmemory_test_resource for the one builder with an in-memory exit).

Example:

def test_it(tmp_path):
    repo = (
        a_grr()
        .with_resource(
            "scores/pos",
            a_position_score()
            .with_score("phastCons", "float")
            .with_data('''
                chrom  pos_begin  phastCons
                1      10         0.1
                1      11         0.2
            '''),
        )
        .build_repo(tmp_path)
    )
    score = PositionScore(repo.get_resource("scores/pos")).open()
    assert score.get_scores_at_position("1", 10) == (0.1,)

class gain.genomic_resources.testing.builders.AlleleScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, scores: tuple[ScoreSpec, ...] = (), data: str | None = None, rows: tuple[tuple[tuple[str, str], ...], ...] = (), tabix: bool = False, csi: bool = False, index_filename: str | None = None, keep_conventional_index: bool = False, chrom_mapping: dict[str, Any] | None = None, chrom_mapping_rows: tuple[tuple[str, str], ...] | None = None, zero_based: bool = False, header_mode: str | None = None, omit_header_mode: bool = False, resource_type: str | None = None, dropped_key_columns: frozenset[str] = frozenset({}))

Bases: _TableScoreBuilder

Immutable builder for a single allele_score resource.

Requires reference/alternative columns and reads back through AlleleScore.

Had a twin, NPScoreBuilder, differing only in emitting type: np_score. It was retired with the type itself in 2026.8.5 (gain#920): a builder can only produce resources GAIn still reads.

Its fixtures moved here unchanged apart from the type. None needed allele_score_mode: substitutions to keep behaving the same, even though the mode default differs between the two spellings – nothing in gain reads AlleleScore.mode outside substitutions_mode() and alleles_mode(), and no migrated fixture consults either. The mode hazard the removal warns about is real for a GRR holder whose own code asks; it was inert in this suite.

DEFAULT_DATA: ClassVar[str] = '\n    chrom  pos_begin  reference  alternative  score\n    1      10         A          G            0.1\n    1      10         A          C            0.2\n    1      16         C          T            0.3\n'

SCORE_TYPE: ClassVar[str] = 'allele_score'

TABLE_EXTRA_CONFIG: ClassVar[str] = '    reference:\n      name: reference\n    alternative:\n      name: alternative\n'

TRAILING_COLUMNS: ClassVar[tuple[str, ...]] = ('reference', 'alternative')

class gain.genomic_resources.testing.builders.BasicResourceBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, files: tuple[tuple[str, str], ...] = (('data.txt', 'alabala'),))

Bases: MetaMixin

Immutable builder for a single basic resource.

basic is the catch-all type: no schema, no type-specific file list, whatever files the author put there. A bare builder realizes type: basic plus one placeholder data.txt payload – the resource every repository-layout test reaches for when it needs a resource and does not care which – and the config it renders for that is byte-identical to the hand-written "type: basic\n" literal it replaces, so fixtures pinning that literal’s size and md5 can move onto it. with_meta and its siblings come from MetaMixin; with_file() is the only knob of its own.

Two exits: build_resource() writes the resource under a tmp_path like every other builder, and build_inmemory() hands it back with no directory at all, for a fixture that only renders the resource’s page. Both realize the same content dict.

build_inmemory() → GenomicResource

Realize this single resource in memory, with no tmp_path.

For a fixture that has no directory to write into – the template tests render a resource’s page and never touch a file.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

files: tuple[tuple[str, str], ...] = (('data.txt', 'alabala'),)

realize_into(resource_dir: Path) → None

Write this basic resource into resource_dir.

with_file(filename: str, content: str) → Self

Add a payload file, or replace the one already carrying its name.

A basic resource ships arbitrary files, so this is the only content knob it has. with_file("data.txt", ...) overwrites the default payload in place rather than adding a second entry under the same name, which the realized directory could not hold anyway. The config is not a payload: it is rendered from the type and the declared meta:, and a file under its name would silently win over both, so that name is refused here – the GRRBuilder duplicate-id precedent.

class gain.genomic_resources.testing.builders.BigWigScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, score_id: str = 'score', value_type: str = 'float', filename: str | None = None, table_format: str | None = None, data: str | None = None, chrom_lens: dict[str, int] | None = None, histogram: dict[str, Any] | None = None, na_values: str | list[str] | None = None, fetch_budgets: dict[str, int] | None = None, zero_based: bool = False, resource_type: str | None = None)

Bases: MetaMixin

Immutable builder for a bigWig-backed position_score.

Authored as bedGraph rows (chrom start end value), whose intervals are 0-based half-open – 1-based position p reads the interval containing p - 1. Unlike the tabular builders this one declares exactly one score, because a bigWig carries a single value – and a resource declaring more than one is refused at open (bigwig_scores.validate_bigwig_scoredefs).

The emitted score block addresses no column, which is the canonical bigWig config: a bigWig record’s payload IS its value, so there is nothing to address. An index: key is a deprecated no-op, accepted with a warning for the deployed resources that carry it; a test that wants that path authors it as yaml – see test_bigwig_scores.py.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

chrom_lens: dict[str, int] | None = None

data: str | None = None

fetch_budgets: dict[str, int] | None = None

filename: str | None = None

histogram: dict[str, Any] | None = None

na_values: str | list[str] | None = None

realize_into(resource_dir: Path) → None

Write the resource config and the bigWig into resource_dir.

resource_type: str | None = None

score_id: str = 'score'

table_format: str | None = None

value_type: str = 'float'

with_chrom_lens(chrom_lens: dict[str, int]) → Self

Declare the chromosome lengths written into the bigWig header.

with_data(data: str) → Self

Author the bedGraph rows as a whitespace-separated block.

with_fetch_size(fetch_size: int) → Self

Emit fetch_size: in the table: config.

A budget in records per range query, not base pairs. The only fetch knob offered: this builder is ours rather than config surface, so it offers no key that does nothing (see docs/adr/0002-remove-bigwig-fetch-buffering.md).

with_filename(filename: str) → Self

Name the bigWig file, overriding the default data.bw.

The suffix is not decoration: with no format: key it is the whole input to the backend decision (build_genomic_position_table), so a test about that decision has to author it. Any name is accepted, including one whose suffix maps elsewhere – proving that an explicit format: overrides the suffix needs exactly such a resource. The bigWig payload itself is written by pyBigWig, which reads the handle rather than the name (gain#348).

with_format(table_format: str) → Self

Emit an explicit format: in the table: config.

The key the suffix only supplies a default for. Authoring it makes a test able to state the precedence between the two inputs – pairing it with a with_filename() whose suffix maps elsewhere is the only way to observe that the explicit key wins (gain#348). Emitted verbatim, so a test can also author the case variants the dispatch accepts.

with_histogram(histogram: dict[str, Any]) → Self

Attach a histogram block to the single bigWig score.

A bigWig exposes exactly one score, so no score_id is needed. The block is emitted verbatim under histogram: in the resource config; without it a numeric bigWig score relies on the resource’s auto-built default histogram.

with_na_values(na_values: str | list[str]) → Self

Declare the NA sentinel(s) for the single bigWig score.

A bigWig exposes exactly one score, so no score_id is needed. Accepts either a scalar ("-1") or a list, emitted verbatim under na_values: in the resource config – the schema permits both.

with_resource_type(resource_type: str) → Self

Render a type: other than position_score.

The backend a resource gets is chosen by its table FORMAT, not by its type (build_genomic_position_table), so a bigWig can sit under an allele_score just as well as under a position_score. Nobody publishes one – a bigWig has no reference or alternative to carry – and that is precisely what makes it worth building: it is how a test reaches an allele score whose backend serves column arrays and whose table has no key columns at all.

Restricted to the types that read back through a score class over this backend; fragment_score is not among them, since a bigWig record is a point value rather than an attributed interval.

with_score(score_id: str, value_type: str = 'float') → Self

Name the single score this bigWig exposes.

with_zero_based() → Self

Emit zero_based: true in the table: config.

A bigWig hard-codes its 0-based-half-open to closed-1-based conversion and never consults the key – authoring it here realizes, config-first, the misconfiguration the table-build warning is meant to surface, and lets a test carry the key through schema validation instead of injecting it into a table dict by hand.

zero_based: bool = False

class gain.genomic_resources.testing.builders.FragmentScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, scores: tuple[ScoreSpec, ...] = (), data: str | None = None, rows: tuple[tuple[tuple[str, str], ...], ...] = (), tabix: bool = False, csi: bool = False, index_filename: str | None = None, keep_conventional_index: bool = False, chrom_mapping: dict[str, Any] | None = None, chrom_mapping_rows: tuple[tuple[str, str], ...] | None = None, zero_based: bool = False, header_mode: str | None = None, omit_header_mode: bool = False, resource_type: str | None = None, dropped_key_columns: frozenset[str] = frozenset({}))

Bases: _TableScoreBuilder

Immutable builder for a single fragment score resource.

Shares the tabular-score machinery with the position/allele builders, differing only in the type value. A fragment is a region rather than a point, so the default data carries the optional pos_end column. Reads back through FragmentScore, which weights every record 1 however long it is.

DEFAULT_DATA: ClassVar[str] = '\n        chrom  pos_begin  pos_end  score\n        1      10         19       0.1\n        1      20         200      0.2\n    '

SCORE_TYPE: ClassVar[str] = 'fragment_score'

with_resource_type(resource_type: str) → Self

Render fragment_score or cnv_collection as the type:.

A bare builder already renders fragment_score, the preferred spelling; reach for this to pin the legacy cnv_collection. Raises if resource_type names no fragment score. Only a fragment score has two spellings, hence not on the base.

class gain.genomic_resources.testing.builders.GRRBuilder(resources: tuple[tuple[str, ResourceBuilder], ...] = (), public_url: str | None = None)

Bases: object

Immutable builder composing resources into a filesystem GRR.

Resources are held behind the shared ResourceBuilder seam, so a single GRR can compose heterogeneous resource types (e.g. a genome plus a position score). build_repo realizes each builder into its own root / resource_id directory; the id is known here, so any ValueError a builder raises is annotated with it centrally.

build_definition(root: Path, *, grr_id: str = 'test_grr') → Path

Realize into root/grr and write a root/grr.yaml.

Returns the path of the written definition file. A CLI tool such as annotate_tabular is given a --grr definition file, not a repository object, so build_repo alone cannot drive one. The definition is written OUTSIDE the resources directory: a stray grr.yaml sitting among the resources would be walked as though it were one.

build_repo(tmp_path: Path, *, proto_id: str | None = None) → GenomicResourceProtocolRepo

Realize a filesystem GRR into tmp_path.

proto_id names the repository, for a caller that has to look it up by name – a group child, whose id is how a resource is asked for. Left unset, the id is derived from the root and the advertised url.

definition(root: Path, *, grr_id: str = 'test_grr') → dict[str, Any]

Render this GRR as a dir repository definition.

Shared by build_repo and build_definition so the in-process repository and the written yaml cannot describe different GRRs. public_url is omitted entirely when none was advertised, rather than written as an explicit null. The schema accepts either and both mean “no public mirror”, but a definition a test reads back is also documentation of the shape a deployment writes – and a deployment with no mirror simply has no such key.

public_url: str | None = None

realize_all(root: Path) → None

Realize every attached resource into root/<resource_id>.

resources: tuple[tuple[str, ResourceBuilder], ...] = ()

with_public_url(public_url: str) → GRRBuilder

Advertise this GRR under public_url.

public_url is the address a deployment publishes its GRR at, and the only one that means anything once a rendered document or an API response leaves the server – the repository’s own url may be a directory mounted into a container. A resource’s public address is this url with the resource id joined to it.

with_resource(resource_id: str, resource_builder: ResourceBuilder) → GRRBuilder

Attach a resource, assigning its repo id here.

Rejects a duplicate id fast at the call site: two resources sharing an id would realize into the same directory with the second silently winning.

class gain.genomic_resources.testing.builders.GeneScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, scores: tuple[ScoreSpec, ...] = (), data: str | None = None, gene_column: str = 'gene', gzipped: bool = False)

Bases: MetaMixin

Immutable builder for a single gene_score resource.

Built on the shared score-declaration base (ScoreSpec): scores are declared with with_score() (column_name defaults to the score id) and validated for duplicate ids / column names exactly like a position score. The gene→value table is authored with with_data() as a whitespace block whose header must be {gene_column} plus each declared score’s column_name.

Realizes as a PLAIN (non-gzipped) tab-separated data.tsv table with a top-level filename: config – mirroring the simplest working gene_score fixture. A bare builder realizes a valid minimal readable gene score: one float score and a few gene rows, with NO histogram (the numeric default histogram is auto-built when the score is read).

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

data: str | None = None

gene_column: str = 'gene'

gzipped: bool = False

realize_into(resource_dir: Path) → None

Write this gene-score resource into resource_dir.

Raises a ResourceValidationError on invalid content; GRRBuilder annotates it with the resource id.

scores: tuple[ScoreSpec, ...] = ()

with_data(data: str) → GeneScoreBuilder

Author the gene→value table as a whitespace-separated block.

Validated at the header level only: it must contain the gene column plus each declared score’s column_name; a missing declared column or an undeclared extra column raises ResourceValidationError.

with_gene_column(name: str) → GeneScoreBuilder

Set the gene-id column name (default "gene").

with_gzip() → GeneScoreBuilder

Realize the gene table gzipped (.tsv.gz) instead of plain.

The default is a plain .tsv table; with_gzip gzips the TSV to data.tsv.gz and points filename: at it. The resource reads back identically to the plain form.

with_histogram(histogram: dict[str, Any], *, score_id: str | None = None) → GeneScoreBuilder

Attach a histogram block to a declared score.

With score_id omitted the histogram is attached to the most-recently-declared score; passing score_id targets that score. Omitted by default: a numeric score relies on the resource’s auto-built default histogram, so no histogram: block is emitted unless one is declared here.

with_score(score_id: str, value_type: str = 'float', *, column_name: str | None = None, desc: str | None = None) → GeneScoreBuilder

Declare a gene score; column_name defaults to score_id.

class gain.genomic_resources.testing.builders.PositionScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, scores: tuple[ScoreSpec, ...] = (), data: str | None = None, rows: tuple[tuple[tuple[str, str], ...], ...] = (), tabix: bool = False, csi: bool = False, index_filename: str | None = None, keep_conventional_index: bool = False, chrom_mapping: dict[str, Any] | None = None, chrom_mapping_rows: tuple[tuple[str, str], ...] | None = None, zero_based: bool = False, header_mode: str | None = None, omit_header_mode: bool = False, resource_type: str | None = None, dropped_key_columns: frozenset[str] = frozenset({}))

Bases: _TableScoreBuilder

Immutable builder for a single position_score resource.

DEFAULT_DATA: ClassVar[str] = '\n        chrom  pos_begin  score\n        1      10         0.1\n        1      11         0.2\n        1      15         0.3\n    '

SCORE_TYPE: ClassVar[str] = 'position_score'

class gain.genomic_resources.testing.builders.ReferenceGenomeBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, fasta: str | None = None, chromosomes: tuple[tuple[str, str], ...] = (), line_width: int = 60, bgzip: bool = True, index_file: str | None = None)

Bases: MetaMixin

Immutable builder for a single genome resource.

Two authoring modes:

• with_fasta(raw) – author the FASTA text. The content is normalized via convert_to_tab_separated (leading indentation and blank lines are stripped, internal whitespace within a line becomes a TAB), so write single-token headers and put each chromosome’s sequence on its own line(s) with no internal spaces.

• with_chromosome(id, seq) – accumulate chromosomes; the FASTA is synthesized (>id header + the sequence wrapped at with_line_width).

The two modes are mutually exclusive: setting both raises when the genome is realized. A bare builder (neither set) realizes a valid minimal genome – one chromosome "1" with a short deterministic sequence.

Realization is bgzipped by default (.fa.gz + .fai + .gzi); as_plain() switches to a plain .fa + .fai.

as_plain() → ReferenceGenomeBuilder

Realize a plain (uncompressed) .fa genome instead of bgz.

bgzip: bool = True

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

chromosomes: tuple[tuple[str, str], ...] = ()

fasta: str | None = None

index_file: str | None = None

line_width: int = 60

realize_into(resource_dir: Path) → None

Write this genome resource into resource_dir.

Delegates compression/indexing and the genomic_resource.yaml to the existing setup_genome/setup_genome_bgz helpers.

with_chromosome(chrom_id: str, sequence: str) → ReferenceGenomeBuilder

Accumulate one chromosome; FASTA is synthesized on realize.

Rejects an empty or whitespace-only sequence fast at the call site (a pysam SamtoolsError otherwise surfaces with no resource context deep inside faidx).

with_fasta(raw: str) → ReferenceGenomeBuilder

Author the genome as FASTA text (primary mode).

The content is not byte-exact: it is normalized via convert_to_tab_separated (leading indentation and blank lines are stripped; internal whitespace within a line becomes a TAB). Write single-token headers (>1, not >1 description) and put each chromosome’s sequence on its own line(s) with no internal spaces.

Rejects empty or whitespace-only content fast at the call site (a pysam SamtoolsError otherwise surfaces with no resource context deep inside faidx), mirroring the with_chromosome guard.

with_index_file(name: str) → ReferenceGenomeBuilder

Publish the .fai index under name via index_file.

The default <filename>.fai is renamed, not copied, so the realized resource carries exactly one FASTA index. That is what makes a test of the index_file override meaningful: htslib and the protocol layer both fall back to the adjacent default name, so a resource that still has a <filename>.fai sitting next to the genome is read successfully even when the override is ignored entirely.

with_line_width(n: int) → ReferenceGenomeBuilder

Set the FASTA wrapping width for the synthesized-FASTA path.

class gain.genomic_resources.testing.builders.ResourceBuilder(*args, **kwargs)

Bases: Protocol

Structural interface for a single-resource test builder.

Every resource builder knows how to realize exactly one resource – its config plus data/index files – into a directory. GRRBuilder composes heterogeneous builders through this one seam; each implementation delegates to the appropriate setup_* helper from gain.genomic_resources.testing.

realize_into(resource_dir: Path) → None

Write this resource’s directory into resource_dir.

class gain.genomic_resources.testing.builders.VcfInfoScoreBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, data: str | None = None, scores: tuple[ScoreSpec, ...] = (), merge_vcf_scores: bool = False, zero_based: bool = False, csi: bool = False, index_filename: str | None = None, realize_index_filename: bool = True, header_index: bool = True)

Bases: MetaMixin

Immutable builder for a VCF-backed allele_score resource.

The score definitions are derived by the resource from the VCF’s ##INFO header, so a bare builder declares none: author the INFO metadata in the VCF text and the scores follow. with_score() AMENDS one of them through a scores: entry – the desc, na_values, aggregator and histogram a config may give a header-derived score, and a type: where the header declares a scalar (dbSNP’s Flag typed bool is the canonical case). Its value_type defaults to None, an entry that states no type: at all, because that is a first-class shape here (gain#1221) and the header already carries one. The block is a FILTER by default – the resource defines only the fields it names; with_merge_vcf_scores() turns it into an override, merging every unnamed header field back in.

Nothing declared is checked against the VCF text: an entry naming no INFO field, or stating a type the header’s Number denies, is what a test authors to watch the RESOURCE refuse it (gain#1336), so the builder renders it as given. The block itself is checked as the table builders check theirs – a field declared twice is refused at realize. Reads back through AlleleScore on the vcf_info table backend, which the .vcf.gz filename selects.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

csi: bool = False

data: str | None = None

header_index: bool = True

index_filename: str | None = None

merge_vcf_scores: bool = False

realize_index_filename: bool = True

realize_into(resource_dir: Path) → None

Write the resource config and the bgzipped VCF + index.

scores: tuple[ScoreSpec, ...] = ()

with_aggregator(aggregator: str, *, score_id: str | None = None) → Self

Declare the default aggregator of a score declared with_score.

with_csi_index() → Self

Index the bgzipped VCF as .csi, not the default .tbi.

with_data(data: str) → Self

Author the whole VCF, ## header lines included.

with_histogram(histogram: dict[str, Any], *, score_id: str | None = None) → Self

Attach a histogram block to a score declared with_score.

with_index_filename(index_filename: str) → Self

Realize the index at index_filename and declare it in config.

The index htslib writes next to the data file is MOVED to index_filename – so no adjacent data.vcf.gz.tbi / data.vcf.gz.csi is left behind – and the table config declares index_filename: pointing at it.

Moving it is the whole point: htslib auto-probes for an adjacent index and opens happily without being told about it, so a resource whose index sits at its default name proves nothing about whether the configured index_filename was honoured (gain#596).

with_merge_vcf_scores() → Self

Emit a top-level merge_vcf_scores: true.

With the block an override rather than a filter, a refusal keyed on a header field (gain#1258) is reachable through a block that never names that field.

with_missing_index_filename(index_filename: str) → Self

Declare index_filename: in the config with no such file.

The realized index stays at its default adjacent name, so htslib could still auto-probe its way to a working file – which is exactly what an open must NOT do here: an explicitly configured index that does not exist has to fail, naming the configured path rather than quietly reading through some other one (gain#596).

with_na_values(na_values: str | list[str], *, score_id: str | None = None) → Self

Declare the NA sentinel(s) of a score declared with_score.

with_score(score_id: str, value_type: str | None = None, *, desc: str | None = None) → Self

Amend the INFO field score_id through a scores: entry.

With no value_type the entry states no type: and reads what the header-only resource reads (gain#1221).

with_zero_based() → Self

Emit zero_based: true in the table: config.

A VCF is always 1-based, so the vcf_info backend ignores the key – authoring it here realizes, config-first, the misconfiguration the table-build warning is meant to surface, and lets a test carry the key through schema validation instead of injecting it into a table dict by hand.

without_header_index() → Self

Ship the *.header.vcf.gz sidecar with no index of its own.

The realize path indexes the sidecar because it bgzips it the same way as the data file; real score resources (dbSNP is the canonical one) ship it unindexed. Use this to realize that shape: the table reads its INFO metadata without an index, and opened by name it is a file whose index resolution resolves to NOTHING and must still open (gain#596).

zero_based: bool = False

gain.genomic_resources.testing.builders.a_basic_resource() → BasicResourceBuilder

Return an immutable basic resource builder.

gain.genomic_resources.testing.builders.a_bigwig_score() → BigWigScoreBuilder

Return an immutable bigWig-backed position-score builder.

gain.genomic_resources.testing.builders.a_fragment_score() → FragmentScoreBuilder

Return an immutable fragment-score builder.

gain.genomic_resources.testing.builders.a_gene_score() → GeneScoreBuilder

Return an immutable gene-score builder.

gain.genomic_resources.testing.builders.a_grr() → GRRBuilder

Return an immutable GRR-composition builder.

gain.genomic_resources.testing.builders.a_position_score() → PositionScoreBuilder

Return an immutable position-score builder.

gain.genomic_resources.testing.builders.a_reference_genome() → ReferenceGenomeBuilder

Return an immutable reference-genome builder.

gain.genomic_resources.testing.builders.a_vcf_info_score() → VcfInfoScoreBuilder

Return an immutable VCF-backed allele-score builder.

gain.genomic_resources.testing.builders.an_allele_score() → AlleleScoreBuilder

Return an immutable allele-score builder.

gain.genomic_resources.testing.builders.build_repo_tempdir(grr_builder: GRRBuilder) → Generator[GenomicResourceProtocolRepo, None, None]

Realize grr_builder into a self-managed temporary directory.

A tmp_path-free realize form for non-pytest callers: the GRR is realized into a fresh tempfile.TemporaryDirectory, yielded as an open repository, and the directory is removed on exit.

gain.genomic_resources.testing.builders.build_resource_tempdir(builder: ResourceBuilder) → Generator[GenomicResource, None, None]

Realize one builder into a self-managed temporary directory.

The single-resource counterpart of build_repo_tempdir(): yields the sole realized resource and cleans the temporary directory up on exit.

gain.genomic_resources.testing.builders.write_grr_definition(root: Path, definition: dict[str, Any]) → Path

Write definition as root/grr.yaml and return its path.

Shared by the GRR and group builders so the serialization policy and the “definition sits OUTSIDE the directory it points at” rule are stated once. A stray grr.yaml among the resources would be walked as though it were one.

gain.genomic_resources.testing.data_frame_builder module

Fluent, immutable test-data builder for data_frame resources.

A sibling of gain.genomic_resources.testing.builders rather than a member of it: that module was already within nine lines of its size ceiling before this builder existed, so the next builder to be added had to live somewhere. The dependency runs ONE WAY – this module imports the shared single-realize seam from builders, and builders does not import back – so a_data_frame is imported from here, not from builders with the other factories.

class gain.genomic_resources.testing.data_frame_builder.DataFrameBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, data: str | None = None, raw_content: str | bytes | None = None, file_format: str = 'csv', declared_format: str | None = None, omit_format_key: bool = False, filename: str | None = None, parameters: dict[str, Any] | None = None, omit_file_key: bool = False)

Bases: MetaMixin

Immutable builder for a single data_frame resource.

A data_frame config declares no columns at all – only file, format and a parameters passthrough – so unlike the score builders there is nothing here for a data header to be validated against. What this builder buys instead is the FORMAT axis: one authored table realized as csv, tsv or xlsx. That is the axis data_frame tests vary, and hand-rolling an xlsx fixture per test is what makes them tedious otherwise.

Two authoring modes:

• with_data() – a whitespace-separated block, normalized by convert_to_tab_separated and rendered into the target format.

• with_raw_content() – verbatim file content, text or bytes. Not a luxury: the parameters: passthrough (skiprows, comment, na_values, quoted separators) describes file shapes a whitespace block cannot express, and a compressed table has no whitespace-block spelling at all.

A bare builder realizes a valid minimal readable csv resource.

Deliberately exposes NO expected DataFrame. Realizing xlsx forces this builder to parse the authored block with pandas, and handing that frame back as a test’s assertion oracle would be circular on exactly the separator and dtype axes a data_frame test varies – builder and loader would have to be wrong in the same way for the test to stay green, which is precisely the shape of gain#434’s tsv-parsed-as-csv bug. Tests state their expectations independently.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

data: str | None = None

declared_format: str | None = None

file_format: str = 'csv'

filename: str | None = None

omit_file_key: bool = False

omit_format_key: bool = False

parameters: dict[str, Any] | None = None

raw_content: str | bytes | None = None

realize_into(resource_dir: Path) → None

Write this data_frame resource into resource_dir.

Raises a ResourceValidationError on invalid content; GRRBuilder annotates it with the resource id.

with_data(data: str) → DataFrameBuilder

Author the table as a whitespace-separated block.

The block is normalized by convert_to_tab_separated (so || becomes a space and EMPTY a dot) and then rendered into whatever with_format() selected.

with_declared_format(file_format: str) → DataFrameBuilder

Override the config’s format: only, leaving realization.

Unvalidated on purpose: this is how a test builds a resource declaring an unknown format, or one whose declared format disagrees with the bytes on disk.

with_file(filename: str) → DataFrameBuilder

Override the realized filename (default: per format).

with_format(file_format: str) → DataFrameBuilder

Select the realized format AND the declared format: key.

One of csv, tsv, excel; the filename follows (data.csv / data.tsv / data.xlsx) unless with_file() overrides it. To declare a format that does NOT match what is on disk – an unknown format, or a mismatch – use with_declared_format().

with_parameters(parameters: dict[str, Any]) → DataFrameBuilder

Emit a parameters: block passed through to the reader.

with_raw_content(content: str | bytes) → DataFrameBuilder

Write content to the data file verbatim.

The escape hatch for file shapes a whitespace block cannot express – comment lines, leading junk rows, quoted separators, explicit NA markers – i.e. everything the parameters: passthrough exists to handle. bytes for a table the loader is meant to decompress, paired with with_file() to give it a name pandas can infer the compression from. Mutually exclusive with with_data(), and unavailable for excel (with_data() renders the workbook).

without_file_key() → DataFrameBuilder

Omit file: from the config, keeping the data file.

Realizes the gain#434 misconfiguration the loader rejects, in the spirit of with_missing_header_mode: the resource is complete on disk but does not say which file to read.

without_format_key() → DataFrameBuilder

Omit format:, exercising the loader’s csv default.

gain.genomic_resources.testing.data_frame_builder.a_data_frame() → DataFrameBuilder

Return an immutable data_frame builder.

gain.genomic_resources.testing.faulty_filesystem module

A test-only fsspec filesystem that can be scripted to fail.

The repository protocol’s failure paths used to be reachable only by patching protocol methods – mocker.patch.object(res, "open_raw_file") and friends. That seam pins internal method names, skips the code between the public API and the patch site, and can only fault the read side: a publish write, a .state write or a cleanup rm had no injection point at all.

The protocol’s real contract boundary is the fsspec AbstractFileSystem it is handed (FsspecReadWriteProtocol(..., filesystem=...)), so that is where faults belong. See docs/adr/0021-protocol-fault-tests-inject-at- the-filesystem-and-tier-by-observability.md and #874.

class gain.genomic_resources.testing.faulty_filesystem.FaultyFileSystem(*args, **kwargs)

Bases: AbstractFileSystem

An AbstractFileSystem that delegates, and fails where told to.

Generic over the filesystem it wraps – MemoryFileSystem by default, matching the inmemory scheme the protocol tests already use, but any AbstractFileSystem will do.

cachable = False because fsspec otherwise memoizes filesystem instances by constructor arguments and would hand a scripted filesystem to an unrelated test.

cachable = False

consume_fault(operation: str, path: str) → _ScriptedFault | None

Return the first scripted fault firing on this call, if any.

Every matching fault is counted, not just the one that fires, so one fault’s ordinals do not shift because another was scripted over the same operation and path.

corrupt_read(pattern: str, *, on_call: int | None = None) → None

Deliver the full length of pattern, with the wrong bytes.

cp_file(path1: str, path2: str, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

created(path: str) → Any

Delegate to the wrapped filesystem.

delete(path: str, recursive: bool = False, maxdepth: int | None = None) → Any

Delegate to the wrapped filesystem.

exists(path: str, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

fail_close(pattern: str, error: BaseException, *, on_call: int | None = None) → None

Fail closing any path matching pattern.

fail_open(pattern: str, error: BaseException, *, on_call: int | None = None) → None

Fail opening any path matching pattern.

fail_read(pattern: str, error: BaseException, *, on_call: int | None = None) → None

Fail reading from any path matching pattern.

stall_read’s general form: the caller names the error rather than taking the timeout that models a dropped link. What needs it is a read that fails the way a remote store fails – an aiohttp error, whose message carries the fetch url – which no scripted open can stand in for, because the protocol redacts the open and not the reads on the handle it returns (gain#620).

fail_rm(pattern: str, error: BaseException, *, on_call: int | None = None) → None

Fail removing any path matching pattern.

fail_write(pattern: str, error: BaseException, *, on_call: int | None = None) → None

Fail writing to any path matching pattern.

find(path: str, maxdepth: int | None = None, withdirs: bool = False, detail: bool = False, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

property fsid: str

Delegate to the wrapped filesystem.

info(path: str, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

invalidate_cache(path: str | None = None) → Any

Delegate to the wrapped filesystem.

isdir(path: str) → Any

Delegate to the wrapped filesystem.

ls(path: str, detail: bool = True, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

makedirs(path: str, exist_ok: bool = False) → Any

Delegate to the wrapped filesystem.

mkdir(path: str, create_parents: bool = True, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

modified(path: str) → Any

Delegate to the wrapped filesystem.

mv(path1: str, path2: str, recursive: bool = False, maxdepth: int | None = None, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

open(path: str, mode: str = 'rb', block_size: int | None = None, cache_options: dict[str, Any] | None = None, compression: str | None = None, **kwargs: Any) → Any

Open through the inner filesystem, under the script.

The whole call is delegated rather than routed through _open, so path handling stays the inner filesystem’s – this wrapper never strips a scheme prefix of its own, and the protocol hands it fully-qualified urls.

put(lpath: str, rpath: str, recursive: bool = False, callback: Any = <fsspec.callbacks.NoOpCallback object>, maxdepth: int | None = None, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

rm(path: str, recursive: bool = False, maxdepth: int | None = None) → Any

Remove through the inner filesystem, under the script.

short_read(pattern: str, *, after_bytes: int, on_call: int | None = None) → None

End the stream of pattern early, silently (the #292 shape).

sign(path: str, expiration: int = 100, **kwargs: Any) → Any

Delegate to the wrapped filesystem.

stall_read(pattern: str, *, on_call: int | None = None) → None

Stall reads of pattern the way a dropped remote link does.

gain.genomic_resources.testing.faulty_filesystem.corrupt_same_length(data: Any) → Any

Return data of the same length with different content.

gain.genomic_resources.testing.gene_models_builder module

Fluent, immutable test-data builder for gene_models resources.

A sibling of gain.genomic_resources.testing.builders for the same reason data_frame_builder and ann_data_builder are ones: builders is already past pylint’s max-module-lines and carries a too-many-lines suppression to say so, so a new builder there would be growing a module that is over the limit rather than finding room in one. (Both siblings describe builders as sitting AT the ceiling; that stopped being true when the suppression went in.) The dependency runs ONE WAY – this module imports the shared single-realize seam from builders and builders does not import back – so a_gene_models is imported from here.

The axis this builder exists to vary is the INTERCHANGE FORMAT. A gene_models resource names its format in the config, and each of the seven formats gain parses spells the same transcript differently – in different columns, in different coordinate conventions, and in the columnar family across two different half-open bounds. A test that wants “the same genes, read through another format” therefore had to hand-roll a second file and a second config and keep the two in step by eye. Here the transcripts are authored ONCE, in gain’s own coordinates, and GeneModelsBuilder.with_format() decides how they are written down; the emitted config always names the format the data was actually rendered in, so the two cannot drift.

Coordinates are gain’s throughout the builder’s interface: 1-based and inclusive on both ends, the convention Exon documents and every parser converts to. The half-open shift the UCSC-derived formats need is applied by the renderer, not by the test author.

Like the other builders here, this exposes NO expected GeneModels: a test states what it expects the parse to be. Handing back a model built from the same records would check the builder against itself on exactly the axis a gene-models test varies.

gain.genomic_resources.testing.gene_models_builder.GENE_MODELS_FORMATS: frozenset[str] = frozenset({'ccds', 'default', 'gtf', 'knowngene', 'refflat', 'refseq', 'ucscgenepred'})

Every interchange format this builder can write, for a test that wants to run over all of them.

class gain.genomic_resources.testing.gene_models_builder.GeneModelsBuilder(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False, transcripts: tuple[TranscriptSpec, ...] = (), fileformat: str = 'refflat', no_genes: bool = False)

Bases: MetaMixin

Immutable builder for a single gene_models resource.

build_resource(tmp_path: Path) → GenomicResource

Realize this single resource (repo id "") into tmp_path.

fileformat: str = 'refflat'

no_genes: bool = False

realize_into(resource_dir: Path) → None

Write this gene-models resource into resource_dir.

Raises a ResourceValidationError on two authoring modes at once; GRRBuilder annotates it with the resource id.

transcripts: tuple[TranscriptSpec, ...] = ()

with_format(fileformat: str) → GeneModelsBuilder

Select the interchange format the transcripts are written in.

An unknown name is refused here, where the caller named it. A format that clashes with with_no_genes() is not: that is a conflict between two authoring modes, and it is reported when the resource is realized – see _effective_content().

with_no_genes() → GeneModelsBuilder

Build a resource whose gene models are empty.

The second authoring mode, and mutually exclusive with the first: realized by setup_empty_gene_models, which writes a refFlat header line and no records. It therefore combines with neither an authored transcript nor another format, and says so when the resource is realized rather than ignoring one.

with_transcript(tr_name: str, *, exons: list[tuple[int, int]], gene: str | None = None, chrom: str = 'chr1', strand: str = '+', cds: tuple[int, int] | None = None) → GeneModelsBuilder

Author one transcript, replacing the default transcripts.

Coordinates – exons and cds alike – are gain’s: 1-based and inclusive at both ends, whatever format they end up written in. gene defaults to the transcript name, and cds omitted means a non-coding transcript.

class gain.genomic_resources.testing.gene_models_builder.TranscriptSpec(tr_name: str, gene: str, chrom: str, strand: str, exons: tuple[tuple[int, int], ...], cds: tuple[int, int] | None)

Bases: object

One authored transcript, in gain’s 1-based inclusive coordinates.

cds is the coding interval; None means a non-coding transcript, which each format spells its own way.

cds: tuple[int, int] | None

chrom: str

exons: tuple[tuple[int, int], ...]

gene: str

strand: str

tr_name: str

property tx: tuple[int, int]

The transcript bounds – the span of its exons.

gain.genomic_resources.testing.gene_models_builder.a_gene_models() → GeneModelsBuilder

Return an immutable gene-models builder.

gain.genomic_resources.testing.group_builder module

Immutable builder composing several GRRs into a group repository.

A group of directory repositories – each with its own resources and its own advertised public_url – is the shape a deployment actually runs, and the one a test needs to prove that a resource’s public address comes from the child repository it was found in rather than from a single base url.

This lives beside gain.genomic_resources.testing.builders rather than inside it: that module is already over pylint’s max-module-lines and carries a suppression for it, so the convention is that a new builder gets a sibling module importing the shared seam one way. builders does not import back, and there is no re-export.

Example:

a_grr_group().with_child("main", a_grr()...).build_repo(tmp_path)

class gain.genomic_resources.testing.group_builder.GRRGroupBuilder(children: tuple[tuple[str, GRRBuilder], ...] = ())

Bases: object

Immutable builder composing whole GRRs into a group repository.

Children are GRRBuilder s, so a child expresses everything a standalone GRR does – its resources and its advertised public_url. Each realizes into its own root / child_id directory, which is what keeps two children carrying the same resource id from realizing over each other.

build_definition(root: Path, *, grr_id: str = 'test_grr') → Path

Realize the children into root/grr and write root/grr.yaml.

Returns the path of the written definition file. As for a single GRR, the definition is written OUTSIDE the directory holding the children, so it is not walked as though it were a resource.

build_repo(tmp_path: Path) → GenomicResourceRepo

Realize every child under tmp_path and build the group.

Returns the plain GenomicResourceRepo seam rather than a protocol repository: a group is not one protocol, and narrowing the annotation would be a promise this cannot keep.

Each child is built through its own GRRBuilder, so a child of a group is realized, repaired and named exactly as the same builder would be on its own – a fixture does not change shape by being composed into a group.

children: tuple[tuple[str, GRRBuilder], ...] = ()

definition(root: Path, *, grr_id: str = 'test_grr') → dict[str, Any]

Render this group as a group repository definition.

Each child is described exactly as a standalone GRR describes itself: GRRBuilder owns both halves, realizing its resources and rendering its own definition.

realize_all(root: Path) → None

Realize every child GRR under its own directory in root.

with_child(repo_id: str, grr_builder: GRRBuilder) → GRRGroupBuilder

Attach a child GRR under repo_id.

Rejects a duplicate id fast at the call site: the group repository refuses duplicate child ids anyway, and a child id is also a cache directory name, so two children sharing one would realize into the same directory with the second silently winning.

gain.genomic_resources.testing.group_builder.a_grr_group() → GRRGroupBuilder

Start building a group of GRRs.

gain.genomic_resources.testing.info_page_fixtures module

The fixtures the info pages’ browser tests are built on.

Two of them. a_coverage_repo is a single resource whose statistics table both sortable-table suites sort; a_browse_repo is a repository shaped to be navigated – folders to descend through and terms to search for – which is what the index page’s own tests need.

Two rather than one because the coverage fixture’s traps are tuned to a sorter and nothing else should perturb them: adding folders to it would change the table the sort assertions read, and adding a sort trap to the browse fixture would make its search assertions depend on row order.

Two suites drive the same table from opposite sides. core/tests/small/genomic_resources/test_info_page_sortable_tables.py pins what the templates emit – which <th> carries data-sort, which <td> carries a data-sort-value. The info_pages_e2e Playwright project generates the page and pins what a browser does with it when a header is clicked.

Neither suite is worth much without the traps below, and those traps are what makes this module exist rather than a copy on each side: the two suites live in different projects, and each <project>/Dockerfile copies only its own directory, so the Playwright project cannot import anything from core’s test tree. It can import this, because gain.genomic_resources.testing ships in the wheel its image installs.

Duplicating the shape instead would give two independently tunable fixtures whose assertions only mean anything while they happen to agree – retune one and the other’s assertions go vacuous with nothing turning red.

gain.genomic_resources.testing.info_page_fixtures.BROWSE_CAPITALISED_FOLDER = 'Zoo'

The browse fixture’s top-level folders, in the order the tree sorts them. Several of them, so a search that matches inside one leaves others that must disappear – a pruned tree with nothing to prune proves nothing.

The capitalised one sorts last here, on purpose. Comparing names as UTF-16 code units – which is what < does, and what the tree did before iossifovlab/gain#579 – puts every capitalised name ahead of every lowercase one, so it would come first; the table has always used localeCompare, which puts it last. The two views ordered a mixed-case repository differently (iossifovlab/gain#564), and a fixture whose names are all lowercase cannot tell the two comparators apart.

The declared order is pinned on the TypeScript side, where the tree’s own ordering is asserted: test_info_page_browse_fixture compares this tuple sorted() against the repository sorted(), so what it pins is the membership, not the sequence written here.

gain.genomic_resources.testing.info_page_fixtures.BROWSE_GENOME_RESOURCE_ID = 'genomes/g984'

a tree with one type in it cannot show that the type filter narrows anything.

Type:

The genome, which is the fixture’s second resource type

gain.genomic_resources.testing.info_page_fixtures.BROWSE_ID_ONLY_TERM = 'phylop'

a term carried only by a resource’s id. Together the two pin the index’s two routes independently – stop indexing summaries and the first goes red while this one stays green.

Type:

The mirror of it

gain.genomic_resources.testing.info_page_fixtures.BROWSE_ORDERING_RESOURCE_IDS = ('Zoo/alpha', 'Zoo/Track')

Two resources sharing that folder, to settle the order of resources.

A pair rather than one, and capitalised against lowercase, because the tree sorts folders and resources with the same comparator: without two resources in one folder, nothing pins that the comparator reached the resources too. Track sorts after alpha by locale and before it by code unit, which is the same disagreement Zoo creates among the folders.

No resource here carries a name needing URL-escaping, and none built this way can: _scan_path_for_resources parses each candidate path with parse_gr_id_version_token, which matches it against [a-zA-Z0-9/._-]+, so a directory with a space, a percent or a non-ASCII letter in it is skipped by the scan with a warning – it is never published. Every character that grammar does allow is unreserved in encodeURIComponent, so percent-encoding a legal folder segment is the identity.

The page encodes anyway, but no fixture can exercise it: a page is built only by the scanning protocol, and a remote .CONTENTS – the one other way a wider id could enter – now drops it at enumeration too (iossifovlab/gain#1352). What info_pages_e2e asserts is the decoding half, reachable from any address a reader can type.

gain.genomic_resources.testing.info_page_fixtures.BROWSE_PHASTCONS_RESOURCE_ID = 'hg38/scores/conservation/phastcons'

The two resources with nothing special about them. They are what gives hg38 a subtree to prune down to and the type filter more than one row to work on.

gain.genomic_resources.testing.info_page_fixtures.BROWSE_RESOURCE_IDS = ('hg38/scores/conservation/phylop', 'hg38/scores/conservation/phastcons', 'hg38/scores/coverage', 'hg19/legacy/allele_frequencies', 'genomes/g984', 'Zoo/alpha', 'Zoo/Track')

Every resource the browse fixture carries. The deepest id is four segments, so the tree has a folder inside a folder inside a folder to descend through and walk back up.

gain.genomic_resources.testing.info_page_fixtures.BROWSE_SUMMARY_ONLY_TERM = 'marmoset'

A term that reaches its resource through the resource’s summary and through nothing else.

An unqualified FTS5 MATCH searches every indexed column, so “found via the summary” is only distinguishable from “found via the id” while this word appears in no id, type, description, score id or label anywhere in the fixture. Nothing in the data enforces that; test_info_page_browse_fixture.py does.

gain.genomic_resources.testing.info_page_fixtures.CONTIGS = ['chr1', 'chr2', 'chr10']

The contigs the fixture carries, in natural order.

gain.genomic_resources.testing.info_page_fixtures.COVERAGE_RESOURCE_ID = 'scores/coverage'

The resource whose Coverage table both suites drive, and the genome it is labelled with. The label is the rung that lets the coverage denominator resolve, which is what gives two rows a fraction and one none.

gain.genomic_resources.testing.info_page_fixtures.COVERED_POSITIONS = [9, 10, 2]

The covered-position counts, in the order the page renders them. 9, 10 and 2 are chosen so that comparing them as text (“10” < “2” < “9”) differs from comparing them as numbers – a column that lost its data-sort="number" would still sort, just wrongly, and only a fixture with this shape notices.

gain.genomic_resources.testing.info_page_fixtures.GENOME_LENGTHS = {'chr1': 100, 'chr2': 50}

chr1 and chr2 resolve a length; chr10 deliberately does not, so the Coverage table carries one row whose fraction is None – its Covered % cell gets no data-sort-value, and the sorter has to treat that as “no value” rather than as zero.

gain.genomic_resources.testing.info_page_fixtures.a_browse_repo(where: Path) → GenomicResourceRepo

A repository shaped to be navigated rather than sorted.

Four top-level folders, a four-segment path to descend, two resource types, and the two search terms above – one reaching its resource only through a summary, the other only through an id.

One of those folders is capitalised, and two resources share it under names that disagree about their order: between them they make the tree’s sort order decidable, which a repository of lowercase names cannot settle.

The summaries are deliberately plain prose: each has to stay clear of both terms except for the one resource that carries it, and prose naming its own resource is exactly how that stops being true.

No labels and no statistics: every column an unqualified MATCH can search is a column one of the two terms could leak into, so the fixture carries the fewest of them it can and still be a repository.

gain.genomic_resources.testing.info_page_fixtures.a_coverage_repo(where: Path) → GenomicResourceRepo

A three-contig score whose genome knows only two of the contigs.

gain.genomic_resources.testing.resource_meta module

The meta: block shared by every GRR test-data builder.

Every resource type may carry a meta: block – summary, description and a free-form labels: mapping – and it is read back through the resource itself (get_summary / get_description / get_labels), by grr_manage/grr_browse, the resource statistics and the docs rendering.

Because it is a property of a resource, not of any one resource type, MetaMixin carries it once for all of the builders in builders and data_frame_builder instead of every _render_config growing its own copy of the block. A builder mixes it in and then either

• appends MetaMixin.render_meta() to the config text it renders itself, or

• calls MetaMixin.append_meta_into() after delegating the config writing to a setup_* helper (the reference-genome path).

Both are no-ops until one of MetaMixin.with_meta(), MetaMixin.with_labels(), MetaMixin.with_raw_labels() or MetaMixin.with_raw_meta() is called, so a builder that does not use them realizes byte-identical output.

It lives in its own module (like score_specs) so the builder DSL can keep growing without builders.py turning into an unreadable slab.

class gain.genomic_resources.testing.resource_meta.MetaMixin(meta_summary: str | None = None, meta_description: str | None = None, meta_labels: Any = None, meta_labels_declared: bool = False, meta_raw: Any = None, meta_raw_declared: bool = False)

Bases: object

Immutable meta: state shared by every resource builder.

A frozen dataclass carrying the meta: fields, so a builder that mixes it in gains with_meta/with_labels – and the rendering behind them – without redeclaring either. Nothing is declared by default, which is what keeps the block absent from the rendered config unless it was asked for.

A value and a declared flag rather than a value alone: labels: and meta: can each be declared as an explicit YAML null, and that renders differently from not declaring them at all – the one emits the key with nothing after it, the other emits no key. The flag carries that distinction, so the value field holds exactly what the caller passed and stays comparable, copyable and picklable.

append_meta_into(resource_dir: Path) → None

Append the meta: block to an already-written resource config.

For the builders that delegate the whole genomic_resource.yaml to a setup_* helper (setup_genome/setup_genome_bgz) rather than rendering it themselves. A no-op when no meta was declared, so the delegated config is left byte-identical.

meta_description: str | None = None

meta_labels: Any = None

meta_labels_declared: bool = False

meta_raw: Any = None

The whole meta: block, when with_raw_meta replaced it.

meta_raw_declared: bool = False

meta_summary: str | None = None

render_meta() → str

Render the meta: block, or "" when nothing was declared.

Emitted through yaml.safe_dump so a summary or a label value carrying a colon, a newline or leading whitespace stays the string it was authored as.

with_labels(**labels: Any) → Self

Emit a labels: mapping inside the resource’s meta: block.

Keys are passed through verbatim, e.g. with_labels(reference_genome="genome") – the label vocabulary is a convention of the GRR content, not of the resource schema, so the builder models none of it. The mapping REPLACES any previously declared one (the _TableScoreBuilder() with_chrom_mapping precedent) and is deep-copied, so neither the mapping nor a mutable value inside it stays shared with the caller.

with_meta(*, summary: str | None = None, description: str | None = None) → Self

Declare the resource’s summary and/or description.

Only the fields passed are set, so the two can be declared in separate calls without the second one clearing the first. Calling with neither is a validation error rather than a silent no-op – emitting an empty meta: block is never what was meant.

with_raw_labels(labels: Any) → Self

Emit labels: as an arbitrary YAML value, mapping or not.

meta.labels is free-form YAML, so what a curator writes there is not necessarily a mapping – a scalar, a list and an explicit null are all things a resource can carry, and each one has to be expressible for the readers of meta.labels to be tested against it (gain#654). with_labels() takes keyword arguments and therefore only ever builds a mapping; this is its sibling for everything else, with_raw_labels(None) being the explicit labels: null spelling. The value REPLACES any previously declared one and is deep-copied, on the same terms.

with_raw_meta(meta: Any) → Self

Emit the whole meta: block as an arbitrary YAML value.

meta: is as free-form as the labels: inside it, so a resource can declare it as a scalar or a list too, and a reader of meta.labels has to be tested against that shape as well (gain#654). Where with_raw_labels() replaces one field, this replaces the block: whatever is passed becomes the value of meta: verbatim, and any summary/description/labels declared alongside it is not rendered. with_raw_meta(None) is the explicit meta: null spelling, as with_raw_labels(None) is for the field. Deep-copied on the same terms as its sibling.

gain.genomic_resources.testing.resource_meta.append_config_block(resource_dir: Path, rendered: str) → None

Append a rendered YAML block to an already-written resource config.

The shared tail for every builder that delegates the whole genomic_resource.yaml to a setup_* helper and then has to add a key the helper does not write. A no-op for an empty block, so the delegated config is left byte-identical.

gain.genomic_resources.testing.score_specs module

Score declarations shared by the GRR test-data builders.

The lowest layer of the builder DSL in builders: the representation of a single declared score column (ScoreSpec), the pure functions that add to and amend a tuple of them, and the renderer that turns them into the scores: block of a genomic_resource.yaml.

Every score builder – position/np/allele/fragment, bigWig, VCF-info, gene – declares its scores through this one representation; they differ only in the base (non-score) columns their data tables require. It lives in its own module so the builder DSL can keep growing without either half of it turning into an unreadable slab.

ResourceValidationError is raised from here, so it is defined here too – the builders re-export it as part of the DSL’s public surface.

exception gain.genomic_resources.testing.score_specs.ResourceValidationError

Bases: ValueError

Raised for a builder-owned validation error.

Subclasses ValueError so existing pytest.raises(ValueError, ...) call sites keep matching. GRRBuilder.build_repo catches only this type when annotating an error with the resource id, so a genuine, non-validation ValueError surfacing from realize_into (e.g. a lower-level failure inside a setup_* helper) passes through un-relabeled instead of being silently recast as a validation error.

class gain.genomic_resources.testing.score_specs.ScoreSpec(score_id: str, value_type: str | None, column_name: str | None, column_index: int | None = None, desc: str | None = None, histogram: dict[str, Any] | None = None, na_values: str | list[str] | None = None, aggregator: str | None = None)

Bases: object

A single declared score column.

The shared score-declaration representation used by the position-score, the gene-score and the VCF-info builders: an id, a column_name (defaulting to the id), a value type, an optional desc and an optional histogram block. The builders differ only in what their data files require of a declaration; the declarations themselves, their column_name defaulting, duplicate-id / duplicate-column_name validation and YAML rendering are all shared through this type.

A score is addressed EITHER by column_name or by column_index, never both. When column_index is set, column_name is None and the column the index points at is resolved from the data header at realize time (see _resolve_column_names()).

value_type is None for an entry that states no type: at all – a legal shape (type: is optional in the resource schema) that a VCF-backed score reaches for on purpose, because there the header already declares the type and an unstated one means “the header’s” (gain#1221). The table-backed builders always state one.

aggregator: str | None = None

column_index: int | None = None

column_name: str | None

desc: str | None = None

histogram: dict[str, Any] | None = None

na_values: str | list[str] | None = None

score_id: str

value_type: str | None

gain.genomic_resources.testing.score_specs.append_score(scores: tuple[ScoreSpec, ...], score_id: str, value_type: str | None, *, column_name: str | None = None, column_index: int | None = None, desc: str | None = None) → tuple[ScoreSpec, ...]

Return scores with one more declared score appended.

Shared by every builder’s with_score. With neither addressing mode given, column_name defaults to score_id; the two modes are mutually exclusive, matching the resource schema, which declares column_index as excluding name/column_name/index.

gain.genomic_resources.testing.score_specs.render_score_specs_yaml(scores: tuple[ScoreSpec, ...]) → str

Render declared scores as a YAML scores: list body (0-indent).

Optional desc/histogram are emitted only when set, so a score with neither renders exactly the three id/type/column_name lines the position-score builder emitted before the shared base. A value_type of None emits no type: line at all.

gain.genomic_resources.testing.score_specs.scores_or_default(scores: tuple[ScoreSpec, ...]) → tuple[ScoreSpec, ...]

Return scores or, when empty, a single default float score.

Shared fallback for every score builder (position/np/allele/gene): a bare builder with no declared score realizes one "score" float column.

gain.genomic_resources.testing.score_specs.set_aggregator(scores: tuple[ScoreSpec, ...], aggregator: str, *, score_id: str | None = None) → tuple[ScoreSpec, ...]

Return scores with aggregator set on one score.

There used to be two fields to choose between – position_aggregator and allele_aggregator. A score has one aggregator now; which reduction it names is fixed by the resource type. With score_id omitted the aggregator is attached to the most-recently-declared score; passing score_id targets that specific score. The value is rendered verbatim, so a test can author an INVALID aggregator on purpose and watch the resource schema reject it.

gain.genomic_resources.testing.score_specs.set_histogram(scores: tuple[ScoreSpec, ...], histogram: dict[str, Any], *, score_id: str | None = None) → tuple[ScoreSpec, ...]

Return scores with histogram set on one declared score.

Shared by every builder’s with_histogram. With score_id omitted the histogram is attached to the most-recently-declared score; passing score_id targets that specific score. Declaring a histogram before any score, or for an unknown score id, is a validation error.

gain.genomic_resources.testing.score_specs.set_na_values(scores: tuple[ScoreSpec, ...], na_values: str | list[str], *, score_id: str | None = None) → tuple[ScoreSpec, ...]

Return scores with na_values set on one declared score.

Shared by every builder’s with_na_values. With score_id omitted the sentinel(s) are attached to the most-recently-declared score; passing score_id targets that specific score. Setting na_values before any score, or for an unknown score id, is a validation error. The value is rendered verbatim under na_values: – either a scalar (na_values: "-1") or a list – matching the resource schema’s ["string", "list"].

Module contents

Provides tools usefult for testing.

gain.genomic_resources.testing.build_faulty_test_protocol(root_path: Path, content: dict[str, Any] | None = None) → tuple[FsspecReadWriteProtocol, FaultyFileSystem]

Build a protocol whose filesystem can be scripted to fail.

The protocol is constructed directly, with its filesystem handed to it, rather than through build_fsspec_protocol() – that builder makes a filesystem of its own from the url and would drop the scripted one.

root_path is what keeps one test’s scripted filesystem out of the next one’s protocol. Protocols are memoized on (proto_id, url) and a rebuild re-runs __init__ on the live instance, rebinding its filesystem: two tests sharing a root would share one protocol, and the second test’s script would be answering the first test’s holder. A per-test tmp_path gives both halves of the key their uniqueness, the same discipline build_filesystem_test_protocol() follows.

content, when given, populates the repository before anything is scripted, so a test scripts faults onto a repository that is already whole.

Returns the protocol and its filesystem, because the filesystem is what a test scripts and proto.filesystem is typed as the fsspec base.

A root is refused the second time it is asked for. Nothing else would catch the mistake: _refuse_a_reconfiguring_rebuild compares the credential kwargs and the public url, not filesystem, so a repeat root is answered with the incumbent protocol carrying the new script – a silent wrong-reason pass rather than an error. The natural slip is wanting a source and a destination and reaching for tmp_path for both; give them tmp_path / "src" and tmp_path / "dst".

gain.genomic_resources.testing.build_filesystem_test_protocol(root_path: Path, *, repair: bool = True, proto_id: str | None = None, public_url: str | None = None, read_only: Literal[False] = False) → FsspecReadWriteProtocol

gain.genomic_resources.testing.build_filesystem_test_protocol(root_path: Path, *, repair: bool = True, proto_id: str | None = None, public_url: str | None = None, read_only: Literal[True]) → FsspecReadOnlyProtocol

Build and return an filesystem fsspec protocol for testing.

The root_path is expected to point to a directory structure with all the resources.

Unless proto_id says otherwise the protocol is named by derive_test_proto_id(), so it can be wrapped in a GenomicResourceCachedRepo without ceremony.

A read_only protocol is the shape a repository served from a remote is read through – it is what a test wanting to hand the protocol hand-written .CONTENTS asks for. It cannot repair what it cannot write, so repair must be turned off along with it.

The derived id is a function of the root and of the mode, and protocols are memoized on (proto_id, url): a second build over a root that already has a protocol of that mode returns that same instance, while a build in the other mode gets an id – and so an instance – of its own. Pass an explicit proto_id when a test wants a genuinely separate protocol over one root; an explicit id names one memoized instance, so build_fsspec_protocol refuses to reuse it in the other mode rather than answering with the mode built first (#514).

public_url is the address a deployment advertises the repository at. It is part of a protocol’s identity – a rebuild that would repoint it is refused – so it joins the derived id too, and two protocols over one root advertising different mirrors are two protocols, exactly as the two modes are.

gain.genomic_resources.testing.build_filesystem_test_repository(root_path: Path, *, proto_id: str | None = None, public_url: str | None = None) → GenomicResourceProtocolRepo

Build and return an filesystem fsspec repository for testing.

The root_path is expected to point to a directory structure with all the resources.

gain.genomic_resources.testing.build_filesystem_test_resource(root_path: Path) → GenomicResource

gain.genomic_resources.testing.build_http_test_protocol(root_path: Path, *, repair: bool = True) → Generator[FsspecReadOnlyProtocol, None, None]

Populate Apache2 directory and construct HTTP genomic resource protocol.

The Apache2 is used to serve the GRR. This root_path directory should be a valid filesystem genomic resource repository.

gain.genomic_resources.testing.build_inmemory_test_protocol(content: dict[str, Any]) → FsspecReadWriteProtocol

Build and return an embedded fsspec protocol for testing.

gain.genomic_resources.testing.build_inmemory_test_repository(content: dict[str, Any]) → GenomicResourceProtocolRepo

Create an embedded GRR repository using passed content.

gain.genomic_resources.testing.build_inmemory_test_resource(content: dict[str, Any]) → GenomicResource

Create a test resource based on content passed.

The passed content should appropriate for a single resource. Example content:

{
    "genomic_resource.yaml": textwrap.dedent('''
        type: position_score
        table:
            filename: data.txt
        scores:
            - id: aaaa
                type: float
                desc: ""
                name: sc
    '''),
    "data.txt": convert_to_tab_separated('''
        #chrom start end sc
        1      10    12  1.1
        2      13    14  1.2
    ''')
}

gain.genomic_resources.testing.build_s3_test_bucket(s3filesystem: S3FileSystem | None = None) → str

Create an s3 test buckent.

gain.genomic_resources.testing.build_s3_test_filesystem(endpoint_url: str | None = None) → S3FileSystem

Create an S3 fsspec filesystem connected to the S3 server.

gain.genomic_resources.testing.build_s3_test_protocol(root_path: Path) → Generator[FsspecReadWriteProtocol, None, None]

Construct fsspec genomic resource protocol.

The S3 bucket is populated with resource from filesystem GRR pointed by the root_path.

gain.genomic_resources.testing.convert_to_tab_separated(content: str) → str

Convert a string into tab separated file content.

Useful for testing purposes. If you need to have a space in the file content use ‘||’.

gain.genomic_resources.testing.copy_proto_genomic_resources(dest_proto: FsspecReadWriteProtocol, src_proto: FsspecReadOnlyProtocol) → None

Publish every resource of src_proto into dest_proto.

Populating a fresh s3 protocol takes a bulk path – see _bulk_populate_genomic_resources() – which is the same repository for a fraction of the round trips (gain#862). Every other destination is populated resource by resource through the protocol.

The bulk path is only taken for a destination that is still empty: it uploads what the source has and so, unlike ReadWriteRepositoryProtocol.copy_resource(), cannot remove a file that has left the manifest since.

gain.genomic_resources.testing.derive_test_proto_id(root: str, *, read_only: bool = False, public_url: str | None = None) → str

Derive a cache-compatible protocol id from a protocol’s root.

The id a testing protocol gets by default must satisfy three constraints at once, and the <name>-<digest> shape is what satisfies all three:

• it is a single path segment, so GenomicResourceCachedRepo accepts it as a cache directory name (#460) – the sanitized name cannot introduce a separator and the appended digest keeps the whole from ever being . or ..;

• it is unique per distinct root, so two protocols built under identically-named temp directories do not trip the group repository’s duplicate-child-id guard (#445);

• it is deterministic, so FsspecReadOnlyProtocol.__new__’s (proto_id, url) memo keeps returning one instance per root. A random or counter-based id would silently change that identity.

The leading name is decoration – it is what makes a cache directory readable while debugging; the digest is what carries the uniqueness.

A read-only protocol gets its own -ro id over the same root, because the memo is keyed on the id and the url alone. Sharing one id between the two modes does not yield two protocols – it is refused (#514) – and a test that wants both modes over one root wants two protocols.

public_url folds into the digest for exactly the same reason: it is part of a protocol’s identity, and a rebuild that would repoint it is refused rather than honoured (#841). Two GRRs over one root advertising different mirrors are therefore two protocols, not one contested one – which is what a test comparing two spellings of an advertised address is asking for.

gain.genomic_resources.testing.proto_builder(scheme: str, content: dict) → Generator[FsspecReadOnlyProtocol | FsspecReadWriteProtocol, None, None]

Build a test genomic resource protocol with specified content.

gain.genomic_resources.testing.resource_builder(scheme: str, content: dict) → Generator[GenomicResource, None, None]

gain.genomic_resources.testing.s3_test_protocol() → FsspecReadWriteProtocol

Build an S3 fsspec testing protocol on top of existing S3 server.

gain.genomic_resources.testing.s3_test_server_endpoint() → str

gain.genomic_resources.testing.setup_bigwig(out_path: Path, content: str, chrom_lens: dict[str, int]) → Path

Setup a bigwig format variants file using bedGraph-style content.

Example: chr1 0 100 0.0 chr1 100 120 1.0 chr1 125 126 200.0

gain.genomic_resources.testing.setup_dae_transmitted(root_path: Path, summary_content: str, toomany_content: str) → tuple[Path, Path]

Set up a DAE transmitted variants file using passed content.

gain.genomic_resources.testing.setup_denovo(denovo_path: Path, content: str) → Path

gain.genomic_resources.testing.setup_directories(root_dir: Path, content: str | dict[str, Any]) → None

Set up directory and subdirectory structures using the content.

gain.genomic_resources.testing.setup_empty_gene_models(out_path: Path) → GeneModels

Set up empty gene models.

gain.genomic_resources.testing.setup_gene_models(out_path: Path, content: str, fileformat: str | None = None, config: str | None = None) → GeneModels

Set up gene models in refflat format using the passed content.

gain.genomic_resources.testing.setup_genome(out_path: Path, content: str) → ReferenceGenome

Set up reference genome using the content.

gain.genomic_resources.testing.setup_genome_bgz(out_path: Path, content: str) → ReferenceGenome

Set up a bgzipped reference genome using the content.

Writes a BGZF-compressed FASTA at out_path (expected to end in .fa.gz/.fa.bgz) together with its .fai and .gzi indexes.

gain.genomic_resources.testing.setup_gzip(gzip_path: Path, gzip_content: str) → Path

Set up a gzipped TSV file.

gain.genomic_resources.testing.setup_pedigree(ped_path: Path, content: str) → Path

gain.genomic_resources.testing.setup_tabix(tabix_path: Path, tabix_content: str, **kwargs: bool | str | int) → tuple[str, str]

Set up a tabix file.

gain.genomic_resources.testing.setup_vcf(out_path: Path, content: str, *, csi: bool = False) → Path

Set up a VCF file using the content.

gain.genomic_resources.testing.short_identity_digest(identity: str) → str

Return the short digest the testing helpers name things by.

One spelling of “distinguish these by content” – the width and the hash are decided here rather than at each call site, so widening it for collisions is one edit.