gain.genomic_resources.genomic_scores package

Submodules

gain.genomic_resources.genomic_scores.aggregation module

Reducing a region to one value per requested score.

The machinery the per-kind aggregating reads share – get_scores_in_region_agg(), get_allele_scores_in_region_agg() and get_fragment_scores_overlapping_region_agg() – apart from any score class: resolving a caller’s queries to (score_id, aggregator) pairs, building a fresh aggregator for each, and folding a stream of fetched segments into one value per request.

fold_into_bins() is the same reduction per grid bin rather than per region: a dense block, one row a bin and one column an aggregator, with the empty-bin rule (EMPTY_BIN_VALUES) stated there and nowhere else.

Resolving and building are separate steps here because they are separate questions. score_def_for() and resolve_aggregator_name() answer “is this request answerable, and by what”, which a caller may ask without meaning to read – since gain#1131 resolve_aggregation_queries() asks exactly that when an annotation pipeline loads, so a misconfigured attribute is refused before any annotatable arrives. build_region_aggregator() answers “give me somewhere to accumulate”, which only a read needs, and which a read needs freshly every time.

Nothing here knows what a score KIND is. The one thing that differs between kinds – how many times a record’s value counts – reaches fold_region_segments() as weigh, the kind’s own record_weight(). The fold applies what it is handed and does not re-derive or cross-check it: the rule is stated once per kind, on the kind (see test_the_weight_rule_is_stated_once_per_kind). Whether the segments were clipped to the query window before they got here is settled the same way, by the kind’s own read, so the fold carries no flag saying which kind it serves.

Every BACKEND feeds one path here – this generic weighted stream over fetched records – and a backend-specific fast path was considered and rejected to keep it that way; see .out-of-scope/bigwig-stats-pushdown.md. That is a claim about backends, not about the package: PositionScore folds its own segments for the aggregated plane, and converging the two is gain#1027’s remaining work, not this module’s promise.

Every refusal here is of a REQUEST, never of a definition: a score the resource does not define, an aggregator a caller names that does not build, a bool score asked for with no aggregator named. The definitions a request is resolved against were held to their own rules when the score was built (a scores: aggregator: that cannot be built is refused there, through score_configuration_error), so by the time a request arrives every definition is one the resource can reduce by. That is why these refusals carry the request family’s address – which score, of which resource – and not the configuration prefix: the file to edit is the caller’s, and Invalid configuration: <resource> would send the reader to a yaml that is fine. The undefined-score sentence is not worded here: every surface that refuses that mistake shares it, from undefined_scores_message().

AggregateValue = str | int | float | bool | None | list[str | int | float | bool | None] | dict[str | int | float | bool | None, int]: The value that one aggregator reduces a region to. The aggregator sets the type: list and most_common give a list, value_count gives a dict that maps each value to its count, and every other aggregator gives a score value.

EMPTY_BIN_VALUES: dict[str, float] = {'count': 0.0, 'max': nan, 'mean': nan, 'median': nan, 'min': nan, 'product': nan, 'sum': 0.0}: What a cell of fold_into_bins() holds when its aggregator received no non-null value, per aggregator: 0 where nothing summed or counted IS zero, NaN where there is nothing to answer. The keys are also the whole set of aggregators the fold accepts – the ones whose answer is a float. This is the fragment surfaces’ rule (a bin with no fragment has nothing to fold), and it differs on purpose from the position-score binner’s, which stores NaN for an uncovered bin unless the track’s none_value_replacement says otherwise; that key does not exist on the fragment kind. The two rules agree on count: its aggregator answers 0 for no values, so the position-score binner stores 0.0 for an uncovered bin of a count track too.

build_region_aggregator(score_id: str, aggregator: str, *, resource_id: str) → Aggregator[source]

Build a FRESH aggregator, naming the resource if it cannot.

Fresh per call, not reused: an aggregator is a mutable accumulator and explicitly not thread-safe, and every caller in gain builds one per fold (see Aggregator). A score holds none between reads.

Aggregator.build raises a bare KeyError('mediann') for an unknown name, saying nothing about which score asked for it. The name refused is the CALLER’s: a definition’s own aggregator: was built once when the score was constructed (score_def.refuse_unbuildable_aggregators), so a resolved request never carries a definition-side spelling that fails here.

build_region_aggregators(requests: Sequence[tuple[str, str]], *, resource_id: str) → list[Aggregator][source]

One FRESH aggregator per request, parallel to the request list.

build_region_aggregator() over a resolved request list – the shape every folding read hands fold_region_segments(), built once per READ and never held on the score, for the reason that function gives. Fresh accumulators remove one hazard, not the class of them: two concurrent region reads of one open score still share the table’s line iterator, which is each read’s “one live region read at a time” rule and not this function’s promise.

distinct_score_ids(score_ids: Iterable[str]) → list[str][source]

The DISTINCT ids among score_ids, in the order asked for.

One fetch serves every aggregation request, so the same list must both name what is fetched and index the values that come back – which is why the ORDER is part of the answer, and why every aggregating read in this package derives it here. A second spelling that ordered the scores differently would not fail; it would have every aggregator quietly reading its neighbour’s column.

Takes the ids rather than the requests they came off, because a request is shaped differently on each surface – a pair here, a PositionScoreAggregationQuery resolved to a triple on the position score’s plane – and none of that is what the derivation is about. Each surface projects its own shape at its call site; the request list gets a named projection, request_score_ids(), because it is the one shape TWO readers (the allele and fragment kinds’ folding reads) project, and each hands the result on to the fold.

Note this is the aggregating reads’ derivation, not a package-wide one: nothing else in the package fetches one list and indexes it twice. The position annotator’s point read asks one score per attribute, a score named twice included, and pairs the answers back by position, so it has nothing to dedupe.

fold_into_bins(records: Iterable[tuple[int, int, str | int | float | bool | None]], *, start: int, end: int, bin_size: int, aggregators: Sequence[str]) → ndarray[source]

Fold position-sorted records into a dense block of global-grid bins.

records are (position, column, value): each value is added to a fresh aggregator of its column in the bin holding its position. aggregators names one aggregator per column, which is how a caller gives each column its meaning – one per query for a score’s binned read, one per group for a binner.

The answer is a float64 array of shape (n_bins, n_columns). Row i is bin calc_bin_index(bin_size, start) + i of the grid anchored at position 1, so n_bins is calc_bin_index(bin_size, end) - calc_bin_index(bin_size, start) + 1 and blocks of adjacent regions split on a bin edge stack into the block of their union. Bins are not clipped and no bounds are returned: an edge row is a whole grid bin, holding only the records whose position lies in [start, end].

The empty-bin rule. Null values are skipped, as every aggregator skips them, and a cell whose aggregator received no non-null value holds EMPTY_BIN_VALUES for that aggregator: 0 for count and sum, NaN for mean, max, min, median and product. Those seven are the whole set this fold accepts; any other aggregator cannot answer a float and is refused before the first record is read, as is a bin_size below 1.

Preconditions. records are sorted by position, and every position lies in [start, end]. The fold holds only the current bin’s aggregators, so it consumes the stream once, in order; a record that breaks either precondition is refused when it is reached. Values are whatever the aggregators accept – numbers, or anything at all for count.

Fold one region read into one value per request.

segments is a stream of (left, right, values) as fetch_region_segments_scores() yields it, with values positional and parallel to score_ids – the list the caller derived (request_score_ids()) to name what it fetched, handed on here so the fold indexes exactly the columns the fetch carried; one derivation cannot disagree with itself, where two (the fold once derived its own) had to be kept agreeing. That is how a request finds its column: two requests for one score share the fetch and keep separate accumulators. Any Sequence of values will do – the fold only ever indexes values[column] – so a kind that hands over tuples folds exactly as one that hands over lists.

aggregators is parallel to requests and built by the CALLER, which is what lets an invalid aggregator name be refused before the region is read at all: a read that refuses an unknown contig when it is called would otherwise mask a misspelled aggregator behind that complaint.

weigh is the caller’s per-kind weight rule, applied as handed: it turns a segment’s span into the number of times that record’s value counts. It is not second-guessed here, and neither is the stream – whether a record was first cut down to the query window is the caller’s business, settled before the segments arrive. A kind that counts a record once counts it wherever the point it collapses to falls, window or not.

request_score_ids(requests: list[tuple[str, str]]) → list[str][source]

distinct_score_ids() of a request list’s scores.

Derived ONCE per aggregating read and handed to both ends of it: the reader names the scores to fetch with it and passes the same list to fold_region_segments() as score_ids, which says why one list serves both. The position score’s plane projects its own shape inline instead – one reader, nothing to hand on.

resolve_aggregation_queries(queries: Sequence[ScoreAggregationQuery] | None, *, score_definitions: dict[str, GenomicScoreDef], all_scores: list[str], resource_id: str) → list[tuple[str, str]][source]

Resolve kind-neutral queries to (score_id, aggregator NAME) pairs.

Asks the two questions every aggregation request asks (score_def_for(), resolve_aggregator_name()), and expands None to every score with its own default. Stops at the NAME, as resolve_aggregation_queries() does, so a caller can ask whether a query list is answerable without reading – an annotator asks exactly that as its pipeline loads. Building the accumulators is the READ’s business, per call (build_region_aggregator()).

Shared by the fragment and allele folding reads, which is what promoted it here from the fragment kind: this package keeps a derivation on the one kind that needs it and moves it here when a second does. The position kind keeps its own resolver because its query carries a third field, none_value_replacement, that has to be judged between the two questions – see that method for why the two must not be merged. A pair has nowhere to put that field, and the signature keeps its query out: a PositionScoreAggregationQuery is a sibling of the neutral query, not a subclass, so it does not type-check here – see ScoreAggregationQuery for why (gain#1302).

resolve_aggregator_name(aggregator: str | None, score_def: GenomicScoreDef, *, resource_id: str) → str[source]

The aggregator to reduce a score with: the caller’s, else its own.

The second question, and the one statement of the rule that a score with neither is refused. Every surface takes the aggregator as a field on its query, so every refusal ends with QUERY_AGGREGATOR_REMEDY. The neutral resolver and the position score’s resolver share the whole sentence, and test_both_surfaces_state_the_missing_default_rule_identically pins that.

The score is named by score_def rather than beside it: score_definitions is keyed by score_id at every construction path, so a separate argument would be one the caller could contradict.

score_def_for(score_id: str, *, score_definitions: dict[str, GenomicScoreDef], resource_id: str) → GenomicScoreDef[source]

The definition an aggregation request names, refusing an unknown one.

The first of the two questions every aggregation request asks, and the one place aggregation refuses when the answer is no. Whether the request arrived as a bare score id, as a (score_id, aggregator) pair, as a ScoreAggregationQuery or as a PositionScoreAggregationQuery changes nothing about it: the resource either defines that score or it does not, and the caller is told which ones it has either way.

gain.genomic_resources.genomic_scores.allele module

AlleleScore – one value per (position, ref, alt) allele.

The kind keyed by a variant rather than a position, in either of two modes (substitutions and alleles). Its reads widen the shared batch with the two key columns a position row does not have – see AlleleRecordArrays for why that widening is a slice of the shared type rather than a separate one.

Bases: object

The answer of a folding read: one value per query, and the allele keys.

Variables:

values (tuple[AggregateValue, ...]) – One value for each query of the read, in the order asked. A query is a score with an aggregator, so one score asked with two aggregators gives two values. The aggregator sets the type of its value: list gives a list and value_count gives a dict.
allele_keys (tuple[str, ...] | None) – The distinct allele keys of the rows that were folded, in the order the rows were first seen. The value is None when the read was not asked for keys. See allele_key() for the format.

allele_keys: tuple[str, ...] | None

values: tuple[str | int | float | bool | None | list[str | int | float | bool | None] | dict[str | int | float | bool | None, int], ...]

class AlleleEntry(pos: int, ref: str | None, alt: str | None, values: tuple[ScoreValue, ...])[source]

Bases: NamedTuple

One allele row of a region, as the unreduced region read yields it.

Variables:

pos (int) – The position of the row.
ref (str | None) – The reference nucleotides of the row, as the table holds them. The value is None when the table has no reference column.
alt (str | None) – The alternative nucleotides of the row, as the table holds them. The value is None when the table has no alternative column.
values (tuple[ScoreValue, ...]) – The values of the scores that the read was asked for, in the order asked.

alt: str | None: Alias for field number 2

pos: int: Alias for field number 0

ref: str | None: Alias for field number 1

values: tuple[str | int | float | bool | None, ...]: Alias for field number 3

exception AlleleMultiplicityError(resource_id: str, detail: str, *, allele: tuple[str, int, str | None, str | None] | None = None)[source]

Bases: ValueError

The rows of an allele score disagree with its allele_multiplicity.

The message is detail with a prefix: the resource id and, when the refusal concerns one allele, the allele spelled by allele_key():

AlleleMultiplicityError(
    "scores/x", "holds 2 rows but declares allele_multiplicity: one",
    allele=("1", 10, "A", "G"))
# allele score 'scores/x', allele 1:10:A:G: holds 2 rows ...

Variables:

resource_id (str) – The id of the resource that the error is about.
allele (tuple[str, int, str | None, str | None] | None) – The allele as (chrom, pos, ref, alt). The value is None when the refusal does not concern one allele.
detail (str) – The message without the prefix.

class AlleleScore(resource: GenomicResource)[source]

Bases: GenomicScore

Allele-specific genomic score resource.

An AlleleScore provides scores that depend on specific alleles at genomic positions. Unlike PositionScore, AlleleScore considers both the reference and alternative alleles when computing scores. This makes it suitable for variant-specific predictions and annotations.

AlleleScore supports two operational modes:

SUBSTITUTIONS mode: Scores are specific to nucleotide substitutions (e.g., A>T, C>G). This mode is optimized for single nucleotide variants and considers the directionality of the change. Used by resources like CADD, which provide substitution-specific scores.
ALLELES mode: Scores are associated with specific alleles at positions, without considering the reference allele. This mode supports insertions, deletions, and more complex variants. The score depends on the alternative allele itself rather than the substitution pattern.

Typical use cases include: - Variant pathogenicity scores (e.g., CADD, DANN) - Functional impact predictions (e.g., PolyPhen, SIFT scores) - Splice site predictions - Regulatory variant scores - Any metric that depends on specific alleles

The score data is typically stored in VCF files or tabix-indexed tables with reference and alternative allele columns.

Example

>>> from gain.genomic_resources.repository_factory import (
...     build_genomic_resource_repository
... )
>>> repo = build_genomic_resource_repository()
>>> resource = repo.get_resource("cadd_v1_6")
>>> score = build_score_from_resource(resource)
>>> with score.open() as score:
...     # Fetch the scores of a specific variant's row
...     values = score.get_allele_scores_for_allele(
...         "chr1", 12345, "A", "T"
...     )
...     # Iterate over the alleles in a region.  The nucleotides
...     # come off the record; the values come off the score.
...     for record in score.fetch_records("chr1", 10000, 20000):
...         values = score.get_score_values_from_record(
...             record, score_defs
...         )
...         print(f"{record[POS_BEGIN]} "
...               f"{record[REF]}>{record[ALT]}: {values}")

Reducing those values over a region is the resource’s own job: get_allele_scores_in_region_agg() folds a region in one streaming walk, one value per ScoreAggregationQuery, with the allele keys beside them when asked, and the allele annotator’s region mode reads through it (gain.annotation.allele_score_annotator).

No read consults the mode: an exact-match read answers an indel’s own line in either mode. The mode is for the caller to route on, and the allele score annotator does – on a substitutions resource it folds any allele other than a substitution over its span instead of matching it.

Variables:

resource (gain.genomic_resources.repository.GenomicResource) – The underlying GenomicResource object
resource_id – Unique identifier for the resource
config (dict[str, Any]) – Configuration dictionary for the score
table – GenomicPositionTable for data access (typically VCF)
score_definitions (dict[str, ScoreDefT]) – Dictionary mapping score IDs to their definitions
mode – Operating mode (SUBSTITUTIONS or ALLELES)

Key Methods:

get_allele_scores_for_allele: Get the values of one allele’s only row, on a resource declaring one row per allele get_allele_score_for_allele: The same for one score get_allele_scores_in_region_agg: Reduce the alleles of a region to one value per query – and their keys – in one walk, telling the same two answers apart get_allele_scores_for_allele_agg: Reduce the rows of one allele to one value per query, telling an allele no row holds apart from one whose rows were all rejected get_allele_score_for_allele_agg: The same for one score get_allele_scores_for_allele_rows: Get the values of every row of one allele, unreduced get_allele_score_for_allele_rows: The same for one score get_allele_scores_in_region_rows: Stream an AlleleEntry per row of a region, unreduced, off the walk the folding read reduces get_allele_score_in_region_rows: The same for one score, as (pos, ref, alt, value) fetch_region_segments_scores: Iterate over allele scores in a genomic region substitutions_mode: Check if operating in SUBSTITUTIONS mode alleles_mode: Check if operating in ALLELES mode

Configuration:

The resource configuration should specify:

table.filename: Path to the data file (usually VCF)
table.reference: Column/field containing reference alleles
table.alternative: Column/field containing alternative alleles
allele_score_mode: Either “substitutions” or “alleles” (optional)
allele_multiplicity: Either “one” (the default) or “many” – how many table rows one allele key may have (optional)
scores: List of score definitions with an optional aggregator specification

DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'max', 'int': 'max', 'str': 'list'}: The aggregator that a query uses when it names none, by the value type of the score: max for numbers, list for strings and none for booleans.

HISTOGRAM_COUNT_UNIT: ClassVar[str] = 'alleles'

"alleles".

Type:: The unit that the histograms of this kind count

class Mode(*values)[source]

Bases: Enum

The allele score mode, as the resource declares it.

SUBSTITUTIONS is the mode substitutions and ALLELES is the mode alleles. No read consults the mode. The allele score annotator uses it to route an allele.

ALLELES = 2

SUBSTITUTIONS = 1

static from_name(name: str) → Mode[source]

Return the mode that has the given configuration name.

Parameters:: name (str) – "substitutions" or "alleles".
Returns:: The mode of the name.
Return type:: AlleleScore.Mode
Raises:: ValueError – If the name is neither of the two.

class Multiplicity(*values)[source]

Bases: Enum

How many table rows one allele may have.

The resource declares it with the top-level key allele_multiplicity. ONE is the default. MANY means that several rows may hold one (chrom, pos, ref, alt), for example one row for each transcript.

MANY = 'many'

ONE = 'one'

alleles_mode() → bool[source]: Return whether the mode of the resource is alleles.

fetch_region_allele_arrays(chrom: str, pos_begin: int | None, pos_end: int | None, scores: list[str], *, batch_size: int = 100000) → Generator[AlleleRecordArrays, None, None][source]

Fetch a region as column arrays, with the nucleotides.

The read widens fetch_region_value_arrays() with the reference and alternative columns. It is meant for a caller that scans a whole region for allele content, such as the allele statistics.

The nucleotides are raw and the scores are parsed. An NA sentinel of a score arrives as the non-value of the score. The two nucleotide columns go through no parsing: no upper-casing, no stripping and no sentinel handling. A caller that wants them normalised does it once, where the meaning is known.

Parameters:

chrom (str) – The chromosome name.
pos_begin (int or None) – The first position of the region, 1-based. None means the start of the chromosome.
pos_end (int or None) – The last position of the region. None means the end of the chromosome.
scores (list[str]) – The score ids to read.
batch_size (int, optional) – The most rows in a batch. The reference and alternative columns count toward VALUE_ARRAYS_CELL_BUDGET like two scores, so a batch holds at most budget // (len(scores) + 2) rows. The default is 100000.

Returns:

The batches, in position order. A batch has the position arrays and the value arrays of fetch_region_value_arrays(), followed by the reference and alternative arrays. A key column that the table does not declare is an array of None.

Return type:

Generator[AlleleRecordArrays, None, None]

Raises:

TypeError – If the score does not serve the read for scores. Ask supports_region_allele_arrays() first.
ValueError – If the score is not open, or if the contig is not in the resource.

Notes

The call checks the request. The errors above are raised when the method is called and not on the first next().

Return the value of one score at the only row of one allele.

The singular form of get_allele_scores_for_allele().

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
score (str or None, optional) – The score id. None, the default, is accepted only when the resource defines exactly one score.
score_filter (ScoreFilter or None, optional) – Keep the row only if this filter accepts it.

Returns:

The value of the score. None when no row holds the allele, or when score_filter rejects the row. The value itself may also be None when the row has no value for the score.

Return type:

ScoreValue or None

Raises:

AlleleMultiplicityError – As get_allele_scores_for_allele().
ValueError – As get_allele_scores_for_allele(). Also when score is None and the resource defines several scores.
ScoreFilterError – If score_filter was compiled for another resource.

get_allele_score_for_allele_agg(chrom: str, pos: int, ref: str, alt: str, *, score: str | None = None, aggregator: str | None = None, score_filter: ScoreFilter | None = None) → AlleleAggregate | None[source]

Reduce the rows of one allele for one score.

The singular form of get_allele_scores_for_allele_agg(). It does not unwrap the answer: the AlleleAggregate has a values tuple of one element.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
score (str or None, optional) – The score id. None, the default, is accepted only when the resource defines exactly one score.
aggregator (str or None, optional) – The aggregator name. None, the default, is the default aggregator of the score.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts.

Returns:

As get_allele_scores_for_allele_agg().

Return type:

AlleleAggregate or None

Raises:

ValueError – As get_allele_scores_for_allele_agg(). Also when score is None and the resource defines several scores.
ScoreFilterError – If score_filter was compiled for another resource.

Return the value of one score for every row of one allele.

The singular form of get_allele_scores_for_allele_rows().

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
score (str or None, optional) – The score id. None, the default, is accepted only when the resource defines exactly one score.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts.

Returns:

One value for each row that the filter keeps, in file order. The list is empty when no row carries the allele or when the filter rejects every row.

Return type:

list[ScoreValue]

Raises:

ValueError – As get_allele_scores_for_allele_rows(). Also when score is None and the resource defines several scores.
ScoreFilterError – If score_filter was compiled for another resource.

Yield (pos, ref, alt, value) for each row that overlaps a region.

The singular form of get_allele_scores_in_region_rows().

Parameters:

chrom (str) – The chromosome name.
start (int) – The first position of the region, 1-based.
end (int) – The last position of the region.
score (str or None, optional) – The score id. None, the default, is accepted only when the resource defines exactly one score.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts.

Returns:

One tuple for each row that the filter keeps. The generator yields nothing when no row overlaps the region or when the filter rejects every row. ref or alt is None when the table has no such column.

Return type:

Generator[tuple[int, str or None, str or None, ScoreValue], None, None]

Raises:

ValueError – As get_allele_scores_in_region_rows(). Also when score is None and the resource defines several scores.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request. The table is read lazily, as in get_allele_scores_in_region_rows().

Return the values of the only row of one allele.

The match is exact on the position and on the nucleotides: a row that starts earlier and whose span reaches pos is not this allele. The read does not consult allele_score_mode.

The read answers one row, as the multiplicity of the resource promises. A resource that declares allele_multiplicity: many is refused, because one allele may hold several rows. On a one resource, several rows that hold the allele are a data error. The rows are counted before score_filter, so a filter that hides one of them does not hide the error. Until allele_multiplicity_enforced() answers true, the read warns once for each process and answers the first row. The filter applies to that row alone.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
scores (Sequence[str] or None, optional) – The score ids to read. None, the default, means every score of the resource, in definition order.
score_filter (ScoreFilter or None, optional) – Keep the row only if this filter accepts it. None, the default, accepts every row.

Returns:

The values of the row, parallel to scores. None when no row holds the allele, or when score_filter rejects the row.

Return type:

tuple[ScoreValue, …] or None

Raises:

AlleleMultiplicityError – If the resource declares allele_multiplicity: many. Also if several rows hold the allele on a one resource, once the enforcement release is reached. Read every row with get_allele_scores_for_allele_rows(), or reduce the rows with get_allele_scores_for_allele_agg().
ValueError – If a score id is unknown, or if the contig is not in the resource.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request before it reads a row, and before the many refusal: an unknown score id, an unknown contig and a foreign filter are refused on any resource.

get_allele_scores_for_allele_agg(chrom: str, pos: int, ref: str, alt: str, *, queries: Sequence[ScoreAggregationQuery] | None = None, allele_keys: Sequence[str] | None = None, score_filter: ScoreFilter | None = None) → AlleleAggregate | None[source]

Reduce the rows of one allele to one value per query.

Fold the rows at exactly (chrom, pos, ref, alt). A row that starts earlier and whose span reaches pos is not folded. The read folds the rows that get_allele_scores_for_allele_rows() answers, with the queries, the aggregators and the AlleleAggregate of get_allele_scores_in_region_agg(). The read is allowed on both multiplicities and does not consult allele_score_mode. On a one resource it folds the single row.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
queries (Sequence[ScoreAggregationQuery] or None, optional) – What to compute. None, the default, means every score of the resource with its default aggregator.
allele_keys (Sequence[str] or None, optional) – As in get_allele_scores_in_region_agg(). The answer has the one key chrom:pos:ref:alt, or several keys when a suffixed score differs across the rows.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts. None, the default, keeps every row.

Returns:

None when no row holds the allele. The check happens before the filter. When the filter rejects every row, the answer is an AlleleAggregate in which each aggregator answers for no rows: max gives None, list gives [] and count gives 0.

Return type:

AlleleAggregate or None

Raises:

ValueError – If a score id is unknown, if a query has no aggregator to resolve to, if an aggregator is bad, or if the contig is not in the resource.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request before it reads a row.

Return the values of every row of one allele, in file order.

Several rows may carry one allele, and all of them are answered. The read does not reduce the rows and does not consult allele_score_mode. The match is exact on the position and on the nucleotides: a row that starts earlier and whose span reaches pos is not this allele.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str) – The reference nucleotides.
alt (str) – The alternative nucleotides.
scores (Sequence[str] or None, optional) – The score ids to read. None, the default, means every score of the resource.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts. The filter is evaluated on every row that overlaps the position, not only on the rows of the allele. None, the default, keeps every row.

Returns:

One tuple for each row that the filter keeps, parallel to scores. The list is empty both when no row carries the allele and when the filter rejects every row. The read does not tell the two cases apart.

Return type:

list[tuple[ScoreValue, …]]

Raises:

ValueError – If a score id is unknown, or if the contig is not in the resource.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request before it reads a row.

get_allele_scores_in_region_agg(chrom: str, start: int, end: int, *, queries: Sequence[ScoreAggregationQuery] | None = None, allele_keys: Sequence[str] | None = None, score_filter: ScoreFilter | None = None) → AlleleAggregate | None[source]

Reduce the alleles in a region to one value per query.

Fold the rows that overlap the region and that score_filter keeps, in one walk that holds no row. An allele row weighs 1.

Parameters:

chrom (str) – The chromosome name.
start (int) – The first position of the region, 1-based.
end (int) – The last position of the region.
queries (Sequence[ScoreAggregationQuery] or None, optional) – What to compute. Each query is a score with an optional aggregator, and the aggregator of the query wins over the default aggregator of the score. None, the default, means every score of the resource with its default aggregator.
allele_keys (Sequence[str] or None, optional) – The score ids that suffix each allele key. None, the default, builds no keys. An empty sequence builds the bare chrom:pos:ref:alt keys. See allele_key().
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts. Build the filter with compile_filter() of this score. None, the default, keeps every row.

Returns:

None when no row overlaps the region. The check happens before the filter. Otherwise an AlleleAggregate. When the filter rejects every row, each aggregator answers for an empty selection: max gives None and list gives []. The keys are distinct and in the order the rows were first seen.

Return type:

AlleleAggregate or None

Raises:

ValueError – If a score id of queries or allele_keys is unknown, if a query has no aggregator to resolve to, or if the contig is not in the resource.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request before it reads a row.

get_allele_scores_in_region_rows(chrom: str, start: int, end: int, *, scores: Sequence[str] | None = None, score_filter: ScoreFilter | None = None) → Generator[AlleleEntry, None, None][source]

Yield an AlleleEntry for each row that overlaps a region.

The entries come in the position order of the table. Rows that share an allele are each yielded. The read does not reduce the rows and does not consult allele_score_mode.

Parameters:

chrom (str) – The chromosome name.
start (int) – The first position of the region, 1-based.
end (int) – The last position of the region.
scores (Sequence[str] or None, optional) – The score ids to read. None, the default, means every score of the resource.
score_filter (ScoreFilter or None, optional) – Keep only the rows that this filter accepts. None, the default, keeps every row.

Returns:

One entry for each row that the filter keeps, with values parallel to scores. The generator yields nothing both when no row overlaps the region and when the filter rejects every row. get_allele_scores_in_region_agg() tells the two cases apart.

Return type:

Generator[AlleleEntry, None, None]

Raises:

ValueError – If a score id is unknown, or if the contig is not in the resource.
ScoreFilterError – If score_filter was compiled for another resource.

Notes

The call checks the request, and the errors above are raised before the generator is returned. The table is read lazily, on the first next(). Iterate one generator at a time, as with every region read of a table, and materialise what has to outlive the next read.

static get_schema() → dict[str, Any][source]

Return the schema of the resource configuration.

The schema is the GenomicScore schema plus the keys allele_score_mode, allele_multiplicity and merge_vcf_scores, and the reference, alternative and variant column definitions of the table.

Returns:: A new schema in the Cerberus format. The caller may change it.
Return type:: dict[str, Any]

property multiplicity: Multiplicity

The number of rows that one allele may have in the table.

Only the bare reads consult it, such as get_allele_scores_for_allele().

substitutions_mode() → bool[source]: Return whether the mode of the resource is substitutions.

supports_region_allele_arrays(scores: list[str]) → bool[source]

Return whether fetch_region_allele_arrays() serves the scores.

The answer needs the conditions of supports_region_value_arrays() (the backend and the value types of the scores) and one more: the table must declare at least one of the reference and alternative columns. One column is enough. A bigWig-backed score has no such columns and is not served.

The method works on a score that is not open. In one case the answer is conservative there: a table with header_mode: file that names its key columns only in its data file answers False until the score is opened, and True after.

Parameters:: scores (list[str]) – The score ids that the caller wants to read.
Returns:: True if fetch_region_allele_arrays() accepts the scores. If the answer is False, read per record with fetch_records(). The rows are the same.
Return type:: bool

Build the allele key, chrom:pos[:ref:alt][:v1,v2].

The key is the identity of an allele as annotation output spells it.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The position of the allele.
ref (str or None) – The reference nucleotides. The key has no :ref:alt part when this or alt is None.
alt (str or None) – The alternative nucleotides.
suffix (Sequence[ScoreValue], optional) – The values of the scores to append, in the order asked. Each value is rendered as stringify() renders it for output, and the values are joined with ,. The suffix is part of the identity: two rows of one allele that differ in a suffixed score give two keys. The default is an empty sequence, which appends nothing.

Returns:

The key, for example 1:10:A:G or 1:10:A:G:0.5,2.

Return type:

str

gain.genomic_resources.genomic_scores.base module

The GenomicScore base class.

Everything the three score kinds share: config parsing and score-def construction, the open/close lifecycle over the position table, and the record and array reads. The kinds themselves live in position, allele and fragment, and the factories that dispatch between them in builders.

Decomposing this class – so that a kind’s author reads the handful of hooks their kind overrides rather than the whole base – is gain#1027. Its first extraction (gain#1044) moved the scoredef lifecycle to score_def and took this module under the 1500-line cap, so the file-scoped too-many-lines pragma gain#1007 added here when it restored that cap is gone; its second (gain#1074) moved the region-aggregation machinery to aggregation, which the per-kind aggregating reads drive, each handing it the kind’s weight rule; its third (gain#1114) moved the value-extraction seam – picking the per-record read, and addressing each score def to a payload column – to value_extraction, leaving GenomicScore.open() calling both in the order that seam requires and the two public per-record getters here. The remaining seams are #1027’s other children.

A score’s defs are finished in place at open: resolve_score_indices() writes score_index onto the definitions this class already holds rather than handing back new ones (it says there who reads them). The earlier half of the lifecycle differs: finish_scoredefs runs inside GenomicScore._build_scoredefs(), before there is a score_definitions to write onto, and so returns the mapping.

class GenomicScore(resource: GenomicResource)[source]

Bases: ScoreResource[GenomicScoreDef]

Base class of the position, allele and fragment scores.

A genomic score gives access to the values of a genomic resource that attaches scores to places in the genome. The class owns what the three kinds share: the score definitions, the open and close lifecycle, the filter compiler, and the record and column-array reads. A score is built closed, so call open() before the first read.

The data of the resource is a tabix file, a VCF, a bigWig or an in-memory table. A score is not safe for concurrent use.

Variables:

resource (GenomicResource) – The genomic resource of the score.
resource_id (str) – The id of the resource.
config (dict) – The validated and normalized configuration.
table (GenomicPositionTable) – The table that holds the data.
score_definitions (dict[str, GenomicScoreDef]) – The definitions of the scores, keyed by score id.
table_loaded (bool) – True while the table is open.
DEFAULT_AGGREGATORS (dict[str, str | None]) – The aggregator that each kind of score value gets when its config states none. The value types of a score are the keys. Each kind of score sets it.

gain.genomic_resources.genomic_scores.batch_budget module

How many rows one bulk column-array batch reads.

A bulk read (GenomicScore.fetch_region_value_arrays() and the allele read built on the same column loop) materialises every cell of the columns it fetches for the rows of a batch, then parses them while the raw cells are still alive. Its peak memory therefore grows with rows x columns, not with rows alone, and a row count on its own lets a score declaring hundreds of columns build a batch of tens of millions of cells. The rule here bounds the cells instead.

VALUE_ARRAYS_CELL_BUDGET = 2000000: The most raw cells – rows x the distinct columns a read fetches – one bulk batch may hold, whatever row count its caller asked for. It bounds the fetch, not the parse: score ids sharing one payload column each still get a parsed array of their own. 2M cells keeps the full 100,000-row default batch for a read of up to 20 columns, and gives a 454-column read about 4,400 rows.

value_arrays_batch_rows(batch_size: int, column_count: int) → int[source]

The rows one bulk batch reads: batch_size, capped by the budget.

The cap divides VALUE_ARRAYS_CELL_BUDGET by the number of columns the read fetches – every one of them, a non-score column such as an allele’s reference/alternative included, since each costs a raw cell per row all the same. Never below one row, so a read wider than the whole budget still makes progress.

gain.genomic_resources.genomic_scores.builders module

The factories that build a score from a resource.

One pair per kind (build_<kind>_score_from_resource and its _from_resource_id sibling) plus the dispatching pair that reads the resource’s type and picks the kind for you. The _from_resource_id half falls back to the default GRR when handed no repository.

build_allele_score_from_resource(resource: GenomicResource) → AlleleScore[source]

Build an allele score from an allele_score resource.

Every call builds a new score. The score is closed: call open() before the first read.

Parameters:: resource (GenomicResource) – A resource of type allele_score.
Returns:: A new, closed score. The mode is alleles unless the resource sets allele_score_mode.
Return type:: AlleleScore
Raises:: ValueError – If the resource is not of type allele_score. A resource that still declares the removed type np_score is also refused. The message names the replacement type and the mode key that keeps the old reading.

build_allele_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) → AlleleScore[source]

Build an allele score from the id of an allele_score resource.

Every call builds a new score. The score is closed: call open() before the first read.

Parameters:

resource_id (str) – The id of a resource of type allele_score in the repository.
grr (GenomicResourceRepo or None, optional) – The repository that resolves resource_id. None, the default, builds the default repository with build_genomic_resource_repository().

Returns:

A new, closed score, as build_allele_score_from_resource builds it.

Return type:

AlleleScore

Raises:

FileNotFoundError – If the repository has no resource with this id.
ValueError – If the resource is not of type allele_score.

build_fragment_score_from_resource(resource: GenomicResource) → FragmentScore[source]

Build a fragment score from a fragment-score resource.

Every call builds a new score. You own the score and can close it without any effect on other scores built from the same resource. The score is closed: call open() before the first read.

Parameters:: resource (GenomicResource) – A resource of type fragment_score, or of the older type cnv_collection, which gives a deprecation warning.
Returns:: A new, closed fragment score.
Return type:: FragmentScore
Raises:: ValueError – If the type of the resource is not a fragment score type.

build_fragment_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) → FragmentScore[source]

Build a fragment score from the id of a fragment-score resource.

Parameters:

resource_id (str) – The id of a resource of type fragment_score, or of the older type cnv_collection, which gives a deprecation warning.
grr (GenomicResourceRepo | None, default None) – The repository that holds the resource. None builds the default repository.

Returns:

A new, closed fragment score. Call open() before the first read.

Return type:

FragmentScore

Raises:

FileNotFoundError – If the repository has no resource with this id.
ValueError – If the type of the resource is not a fragment score type.

build_position_score_from_resource(resource: GenomicResource) → PositionScore[source]

Build a position score from a resource.

The score is closed. Call open() before the first read. Each call returns a new score, so the caller owns it and closing it affects no other score.

Parameters:: resource (GenomicResource) – A resource of type position_score.
Returns:: A closed position score for the resource.
Return type:: PositionScore
Raises:: ValueError – If the resource is not of type position_score.

build_position_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) → PositionScore[source]

Build a position score from the id of a resource.

The score is closed. Call open() before the first read. Each call returns a new score, so the caller owns it and closing it affects no other score.

Parameters:

resource_id (str) – The id of a resource of type position_score in the repository.
grr (GenomicResourceRepo | None) – The repository that resolves resource_id. When it is None, the function builds the default repository with build_genomic_resource_repository().

Returns:

A closed position score for the resource.

Return type:

PositionScore

Raises:

FileNotFoundError – If the repository has no resource with this id.
ValueError – If the resource is not of type position_score.

build_score_from_resource(resource: GenomicResource) → GenomicScore[source]

Build the score of a score resource, whatever its kind.

The function reads the type of the resource and calls the factory of that kind. Use the factory of the kind when the type is known. Use this function when the type is not.

Parameters:: resource (GenomicResource) – A resource of type position_score, allele_score or fragment_score. The legacy type cnv_collection also names a fragment score.
Returns:: A new PositionScore, AlleleScore or FragmentScore. Each call returns a new score that the caller owns, so closing it affects nothing else. The score is closed: call open() before the first read.
Return type:: GenomicScore
Raises:: ValueError – If the resource is not of a score type, or if it declares a retired type such as np_score.

build_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) → GenomicScore[source]

Build the score of the score resource with a given id.

The function finds the resource in the repository and calls build_score_from_resource().

Parameters:

resource_id (str) – The id of a score resource in the repository.
grr (GenomicResourceRepo | None, default None) – The repository that resolves resource_id. When it is None, the function builds the default repository with build_genomic_resource_repository().

Returns:

A new PositionScore, AlleleScore or FragmentScore. The score is closed: call open() before the first read.

Return type:

GenomicScore

Raises:

FileNotFoundError – If the repository has no resource with this id.
ValueError – If the resource is not of a score type, or if it declares a retired type such as np_score.

gain.genomic_resources.genomic_scores.chrom_lengths module

Chromosome lengths of a genomic score, and where each one came from.

A score’s table can usually say how long a contig is, but the backends answer with different confidence – a bigWig header is exact, the tabix probe brackets an upper bound, the in-memory backend only knows how far its rows reach – and the callers that split a contig into regions also need to know WHY there is no length when there is none (gain#509). So the record here keeps each number under its provenance, picks the best-trusted of them, and carries the ContigExtent reason the table gives when it has no number.

The ladder the epic (gain#1412) settles is genome label → bigWig header → tabix estimate, applied per contig of the score. The resolver here asks every rung live, and every rung answers (gain#1574): the genome rung from the ReferenceGenome the caller hands in, the table’s own rung through the table – a record holds each answer under its source, and best picks by the source’s rank. It holds no repository, so resolving the score’s reference_genome label into that genome is the caller’s job.

A repair stores what the resolver found as CHROM_LENGTHS_FILE – one block per source, contig to length, the table’s block carrying the reason for a contig it had no length for – together with a DerivedFrom key: the label the genome resolved from and the manifest md5 of every table file, which is what a later repair compares to tell a current file from a stale one without opening the table (gain#1576). save_chrom_lengths / load_chrom_lengths are the file’s two seams; loading never raises, a file that cannot be read as one reads as absent.

CHROM_LENGTHS_FILE = 'statistics/chrom_lengths.json': Where a repair stores the resolver’s answers, relative to the resource.

CHROM_LENGTHS_FORMAT = 1: The layout of that file. A stored file of another format is stale: read as absent and rewritten by the next repair.

class ChromLength(answers: Mapping[ChromLengthSource, int], extent: ContigExtent | None)[source]

Bases: object

One contig’s length from every source that had one.

answers is each source’s length; extent is the TABLE’s reason when it had none – EMPTY when it proved the contig holds no records, UNDETERMINED when the probe could not answer for a contig that may well hold some – and None when it answered. A record with no answer always carries the reason; one is refused without it.

answers: Mapping[ChromLengthSource, int]

property best: ChromLengthAnswer | None: The highest-ranked answer with its source; None when none.

extent: ContigExtent | None

property listed_by_genome: bool

a contig the genome lists carries its answer, whatever the table said. The one definition of “unlisted”, which the repair’s warning (gain#1575) reads.

Type:: Whether the genome rung answered

class ChromLengthAnswer(length: int, source: ChromLengthSource)[source]

Bases: object

One source’s length for a contig: the number and where it came from.

length: int

source: ChromLengthSource

class DerivedFrom(reference_genome: str | None, files_md5: dict[str, str | None])[source]

Bases: object

What a stored answer was computed from – the file’s freshness key.

reference_genome is the label the genome was actually resolved from, None when there was none to resolve: no label, a label that is not a resource id, or a genome the repository could not find. So a genome that turns up later reads as a change. files_md5 is the manifest md5 of every data file of the table, keyed by name – as the manifest records it, which for an entry it has not digested is none.

describes(resource: GenomicResource, manifest: Manifest, files: Collection[str], *, genome_resolves: Callable[[], bool] | None = None) → bool[source]

Whether resource, as it is now, is what this was derived from.

The one freshness test of the key: manifest’s md5 of every one of the table’s files (a file it does not list is a change), then the reference_genome label as the resource carries it today. A key derived from a genome stands while the label still names it, whether or not the genome is still there – the record is the record (gain#1573). One derived with none stands while there is still none: a genome that turns up later reads as a change, and whether a label that is now present names one is genome_resolves’s answer – the implementation’s, which holds a repository. A caller with none passes nothing, and any label then reads as a change: conservative on purpose (gain#1419), the answer costs the live probe and nothing more.

The manifest is the caller’s to hand in, so that a read path passes the stored one and never builds it.

files_md5: dict[str, str | None]

reference_genome: str | None

class StoredChromLengths(lengths: dict[str, ChromLength], derived_from: DerivedFrom, table_source: ChromLengthSource)[source]

Bases: object

Every source’s answer for every contig, and what they derive from.

table_source is the source the score’s table answers under: its block is the one that names every contig, in the table’s order, and the one that carries a contig’s reason when the table had no length for it.

derived_from: DerivedFrom

lengths: dict[str, ChromLength]

property sources: list[ChromLengthSource]: Every source with an answer, and the table’s, best first.

table_source: ChromLengthSource

as_chrom_length_source(source: ChromLengthSource | str | None) → ChromLengthSource | None[source]

The member source names – itself, or by its string value.

None stays None: the caller’s “the best-ranked one”. Raises ValueError naming the valid values on a string that is none of them.

best_chrom_length(chrom: str, resolved: ChromLength, contigs: list[str]) → ChromLengthAnswer[source]: resolved.best, or the table’s refusal when there is none – contigs being the table’s, which the refusal names.

derive_chrom_length(score: GenomicScore, chrom: str, ref_genome: ReferenceGenome | None = None) → ChromLength[source]

Resolve one contig of score through the ladder, every rung.

ref_genome is the top rung: a contig it lists is answered with its true length under REFERENCE_GENOME. The table is asked as well, genome or not, and answers under whatever source the backend declares its lengths to be, or with the reason it cannot. Raises ValueError when the score is not open or does not carry chrom – a bad question, as opposed to an absent answer – in the TABLE’s words, since it is the table that refuses; the genome rung sits behind the table’s own screen so that a contig only the genome knows is refused the same way.

derive_chrom_lengths(score: GenomicScore, ref_genome: ReferenceGenome | None = None) → dict[str, ChromLength][source]

Resolve every contig of score, in the table’s order.

The universe is the score’s contigs – get_all_chromosomes() – never the whole genome; a whole-reference denominator is a property of the genome and belongs to coverage (gain#1041). Keyed by contig so the consumer that splits regions (the statistics build) looks up by name; the dict keeps table order.

Not memoised: on a tabix score every call re-runs the probe per contig. Raises ValueError on a score that is not open.

files_md5_of(manifest: Manifest, files: Collection[str]) → dict[str, str | None][source]

The manifest md5 of every one of files, keyed by name.

One definition of “the same files” for every gate that asks – the statistics hash, the stored lengths’ key and the score’s own reader – so they cannot drift apart on what counts as a data file. As the manifest records it, which for an entry it has not digested is none; a file the manifest does not list is a KeyError.

load_chrom_lengths(resource: GenomicResource) → StoredChromLengths | None[source]

Read the resource’s CHROM_LENGTHS_FILE; None when it has none.

Absence is a normal state, not an error: a resource repaired before the file existed has nothing stored until its next repair. A file that cannot be read as one – of another format, or truncated by a repair killed mid-write, the write not being atomic – reads as absent too, with a WARNING naming it: absent, the ordinary repair rewrites it; raised, the gate would fail the resource over a file the repair is about to replace.

load_current_chrom_lengths(score: GenomicScore) → StoredChromLengths | None[source]

The stored lengths, if the file describes score as it is now.

Four ways for it not to, all normal and none a WARNING: no file (a resource repaired before the file existed, or never – the state of every resource until its next repair, so said at DEBUG, as anything louder would fire on every open of every one of them); no stored manifest to compare the key against; a derived_from that no longer matches (a label re-pointed and not yet repaired – trusted, it would answer the old genome’s lengths); and a contig list that is not the table’s (a chrom_mapping changed under it) – the last three INFO, since something is off and a repair is due. Until it runs the score’s reads resolve live. Asked of an open score.

ranked(sources: Collection[ChromLengthSource]) → list[ChromLengthSource][source]: sources best first: the members are declared in ladder order.

refuse_unanswered_source(chrom: str, source: ChromLengthSource, resolved: ChromLength, *, table_source: ChromLengthSource, contigs: list[str]) → None[source]

Raise for a source that has no answer in resolved.

The table’s own source is refused in the table’s words, which say whether the contig is empty or merely unmeasured; any other names the sources that do answer the contig.

save_chrom_lengths(resource: GenomicResource, stored: StoredChromLengths) → None[source]: Write stored as the resource’s CHROM_LENGTHS_FILE.

gain.genomic_resources.genomic_scores.fragment module

FragmentScore – one value per genomic interval.

The kind whose records span a region rather than a point, and the one that still answers to a legacy resource-type spelling; recognising that spelling announces it through warn_deprecated_spelling().

Bases: object

One reduction of the fragments in a region, and the fragment count.

Variables:

count (int) – The number of fragments that the read saw: the fragments that overlap the region, that pass the overlap fractions and that score_filter keeps. A region with no fragments gives 0. A filter that rejects every fragment gives 0 too.
values (tuple[AggregateValue, ...]) – One reduced value for each query, in the order of the queries. The same score asked twice with two aggregators gives two values. A value is None when its aggregator has nothing to reduce. The aggregator sets the type of its value: list gives a list and value_count gives a dict.

count: int

values: tuple[str | int | float | bool | None | list[str | int | float | bool | None] | dict[str | int | float | bool | None, int], ...]

class FragmentScore(resource: GenomicResource)[source]

Bases: GenomicScore

A genomic score over fragments: intervals that carry attributes.

A fragment keeps its own begin and end, and fragments may overlap. A copy-number variant collection is one use of this kind. The resource type is fragment_score. The older type name cnv_collection is still accepted, with a deprecation warning.

Build a fragment score with build_fragment_score_from_resource_id(). The score is built closed. Call open() before the first read.

The reads come in pairs. The plural form takes a scores list and returns values parallel to it. The singular form takes one score.

Parameters:: resource (GenomicResource) – The resource of type fragment_score or cnv_collection.
Raises:: ValueError – If the type of the resource is not a fragment score type.

DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'max', 'int': 'max', 'str': 'join(,)'}: The default aggregator for each value type, for a score whose configuration states no aggregator: max for float and int, join(,) for str and none for bool.

HISTOGRAM_COUNT_UNIT: ClassVar[str] = 'fragments'

"fragments".

Type:: The unit that the histograms of this kind count

get_fragment_arrays_overlapping_region(chrom: str, start: int, end: int, *, scores: list[str] | None = None, batch_size: int = 100000) → Generator[tuple[ndarray, ndarray, dict[str, ndarray]], None, None][source]

Yield the fragments that overlap a region, as column arrays.

The array form of get_fragment_scores_overlapping_region(). The concatenated batches hold the same fragments, in the same order, as that read. Each fragment keeps its own extent: the read does not clip a fragment to the region, also where the fragment runs past it.

The values follow the parse of fetch_region_value_arrays(): an int or a float score gives float64 with nan for no value, and a str score gives an object array with None for no value.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
scores (list[str] | None, default None) – The score ids to read. None reads every score of the resource.
batch_size (int, default 100000) – The hint for the rows of a batch that fetch_region_value_arrays() takes.

Returns:

Batches of (begins, ends, {score_id: values}). The keys of the mapping follow the order of scores. The mapping holds one column for each distinct score id: a score id that scores repeats gives one key, not one value for each repeat as the record read gives. A batch holds at least one fragment: the generator is empty when no fragment overlaps the region.

Return type:

Generator[RecordArrays, None, None]

Raises:

ValueError – If start is below 1, end is before start, the score is closed, or the contig or a score id is unknown.
TypeError – If supports_region_value_arrays() is False for the scores.

get_fragment_arrays_starting_in_region(chrom: str, start: int, end: int, *, scores: list[str] | None = None, batch_size: int = 100000) → Generator[tuple[ndarray, ndarray, dict[str, ndarray]], None, None][source]

Yield the fragments that begin in a region, as column arrays.

The array form of get_fragment_scores_starting_in_region(). The concatenated batches hold the same fragments, in the same order, as that read. Each fragment keeps its own extent: the read does not clip a fragment to the region.

The values follow the parse of fetch_region_value_arrays(): an int or a float score gives float64 with nan for no value, and a str score gives an object array with None for no value.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
scores (list[str] | None, default None) – The score ids to read. None reads every score of the resource.
batch_size (int, default 100000) – The hint for the rows of a batch that fetch_region_value_arrays() takes. A batch of this read can hold fewer rows.

Returns:

Batches of (begins, ends, {score_id: values}). The keys of the mapping follow the order of scores. The mapping holds one column for each distinct score id: a score id that scores repeats gives one key, not one value for each repeat as the record read gives. A batch holds at least one fragment: the generator is empty when no fragment begins in the region.

Return type:

Generator[RecordArrays, None, None]

Raises:

ValueError – If start is below 1, end is before start, the score is closed, or the contig or a score id is unknown.
TypeError – If supports_region_value_arrays() is False for the scores.

Return the fragments that cover a position, with the value of one score.

This is the singular form of get_fragment_scores_at_position().

Parameters:

chrom (str) – The contig name.
pos (int) – The position (1-based).
score (str | None, default None) – The score id. None is valid only when the resource defines exactly one score.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

One (begin, end, value) tuple for each fragment. The list is empty when no fragment covers the position.

Return type:

Sequence[tuple[int, int, ScoreValue]]

Raises:

ValueError – If score is None and the resource defines more than one score, or for any reason that get_fragment_scores_at_position() lists.

Yield the fragments that overlap a region, with the value of one score.

This is the singular form of get_fragment_scores_overlapping_region(). The same limits apply.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
score (str | None, default None) – The score id. None is valid only when the resource defines exactly one score.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.
min_region_overlap_fraction (float | None, default None) – The fragment must cover at least this fraction of the region. None sets no threshold.
min_fragment_overlap_fraction (float | None, default None) – At least this fraction of the fragment must fall in the region. None sets no threshold.

Returns:

One (begin, end, value) tuple for each fragment. The value is None when the fragment has no value for the score. The generator is empty when no fragment overlaps the region.

Return type:

Generator[tuple[int, int, ScoreValue], None, None]

Raises:

ValueError – If score is None and the resource defines more than one score, or for any reason that get_fragment_scores_overlapping_region() lists.

get_fragment_score_overlapping_region_agg(chrom: str, start: int, end: int, *, score: str | None = None, aggregator: str | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None, score_filter: ScoreFilter | None = None) → FragmentAggregate[source]

Reduce the fragments that overlap a region to one value for one score.

This is the singular form of get_fragment_scores_overlapping_region_agg(). It returns the same FragmentAggregate, and values has one element.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
score (str | None, default None) – The score id. None is valid only when the resource defines exactly one score.
aggregator (str | None, default None) – The aggregator. None uses the default aggregator of the score.
min_region_overlap_fraction (float | None, default None) – The fragment must cover at least this fraction of the region. None sets no threshold.
min_fragment_overlap_fraction (float | None, default None) – At least this fraction of the fragment must fall in the region. None sets no threshold.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

The number of fragments that the read saw, and a tuple with the one reduced value.

Return type:

FragmentAggregate

Raises:

ValueError – If score is None and the resource defines more than one score, or for any reason that get_fragment_scores_overlapping_region_agg() lists.

Yield the fragments that begin in a region, with the value of one score.

This is the singular form of get_fragment_scores_starting_in_region(). The same partition and the same limits apply.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
score (str | None, default None) – The score id. None is valid only when the resource defines exactly one score.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

One (begin, end, value) tuple for each fragment. The generator is empty when no fragment begins in the region.

Return type:

Generator[tuple[int, int, ScoreValue], None, None]

Raises:

ValueError – If score is None and the resource defines more than one score, or for any reason that get_fragment_scores_starting_in_region() lists.

Return the fragments that cover a position.

Each fragment is reported at its full extent, not clipped to the position.

Parameters:

chrom (str) – The contig name.
pos (int) – The position (1-based).
scores (list[str] | None, default None) – The score ids to read. None reads every score of the resource.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

One (begin, end, values) tuple for each fragment. The list is empty when no fragment covers the position. You can keep the list across a later read.

Return type:

Sequence[tuple[int, int, tuple[ScoreValue, …]]]

Raises:

ValueError – If pos is below 1, or for any other reason that get_fragment_scores_overlapping_region() lists.

Yield the fragments that overlap a region.

Each fragment is reported at its own extent. The extent is not clipped to the region, also where the fragment runs past it. The fragments come in table order.

The reads check the request when you call them, and read the data later. A closed score, an unknown contig, an unknown score id, an invalid region and an overlap fraction outside the range from 0 to 1 raise before the first next().

A score serves one region read at a time. Starting a second read invalidates a read that you still consume, and a tabix-backed table then answers with the records of the other read and no error. Close a held generator, or call list(...) on it, before the next read.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
scores (list[str] | None, default None) – The score ids to read. None reads every score of the resource.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). The read drops the fragments that the filter rejects. The filter may name a score that is not in scores. None keeps every fragment.
min_region_overlap_fraction (float | None, default None) – The fragment must cover at least this fraction of the region. None sets no threshold.
min_fragment_overlap_fraction (float | None, default None) – At least this fraction of the fragment must fall in the region. None sets no threshold.

Returns:

One (begin, end, values) tuple for each fragment. values is parallel to scores. A value is None when the fragment has no value for that score. The generator is empty when no fragment overlaps the region.

Return type:

Generator[tuple[int, int, tuple[ScoreValue, …]], None, None]

Raises:

ValueError – If the score is closed, the contig or a score id is unknown, start is below 1, end is before start, or an overlap fraction is outside the range from 0 to 1.

get_fragment_scores_overlapping_region_agg(chrom: str, start: int, end: int, *, queries: Sequence[ScoreAggregationQuery] | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None, score_filter: ScoreFilter | None = None) → FragmentAggregate[source]

Reduce the fragments that overlap a region to one value for each query.

The read selects the fragments as get_fragment_scores_overlapping_region() does. The overlap fractions and score_filter have the same meaning. Each fragment counts once, however long it is. The read folds the fragments as they arrive and does not keep them. A few aggregators keep one entry for each fragment while they run: list, median, concatenate and join. The default aggregator of a str score is join(,), which is one of them.

The count of the result counts fragments. A query with the count aggregator counts the non-null values instead, and gives 0 for an empty region.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
queries (Sequence[ScoreAggregationQuery] | None, default None) – The scores to reduce, each with an aggregator. A query with no aggregator uses the default of the score. None reduces every score of the resource with its default aggregator.
min_region_overlap_fraction (float | None, default None) – The fragment must cover at least this fraction of the region. None sets no threshold.
min_fragment_overlap_fraction (float | None, default None) – At least this fraction of the fragment must fall in the region. None sets no threshold.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

The number of fragments that the read saw, and one reduced value for each query. When no fragment overlaps the region, count is 0.

Return type:

FragmentAggregate

Raises:

ValueError – If a query names an unknown score or an aggregator that the score does not accept, or for any reason that get_fragment_scores_overlapping_region() lists.

Yield the fragments that begin in a region.

The read yields the fragments whose begin lies in [start, end]. A set of adjacent windows answers each fragment from exactly one window, with no duplicate and no gap. Use this read for chunked and parallel work. get_fragment_scores_overlapping_region() answers a fragment from every window that it reaches into.

Each fragment is reported at its own extent. The limit of one live read per score applies, as for get_fragment_scores_overlapping_region(). The read has no overlap fractions, because a fraction filter could let a fragment fall out of every window.

Parameters:

chrom (str) – The contig name.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
scores (list[str] | None, default None) – The score ids to read. None reads every score of the resource.
score_filter (ScoreFilter | None, default None) – A filter from compile_filter(). None keeps every fragment.

Returns:

One (begin, end, values) tuple for each fragment. values is parallel to scores. The generator is empty when no fragment begins in the region.

Return type:

Generator[tuple[int, int, tuple[ScoreValue, …]], None, None]

Raises:

ValueError – If start is below 1, end is before start, the score is closed, or the contig or a score id is unknown.

static get_schema() → dict[str, Any][source]

Return the configuration schema of a fragment score resource.

Returns:: The schema of a genomic score, with an aggregator entry added to each score definition.
Return type:: dict[str, Any]

get_scores_in_bins(chrom: str, start: int, end: int, bin_size: int, queries: Sequence[ScoreAggregationQuery]) → ndarray[source]

Reduce the fragments that start in each bin, one column for each query.

The grid is global and starts at position 1. Row i is the bin with index calc_bin_index(bin_size, start) + i. The rows are whole bins. The read does not clip them to the region.

A fragment belongs to the bin that holds its begin, however far it runs. The blocks of two adjacent calls that split on a bin edge stack into the block of their union. The fragments are those that get_fragment_scores_starting_in_region() yields for [start, end].

The fragment count of a bin is the query ScoreAggregationQuery(<score>, "count"). It counts the fragments that start in the bin and have a value for the score. For a score that every fragment carries, it is the number of fragments that start in the bin.

The read answers all queries from one read of the region. It materialises the block, so the limit of one live read applies for the duration of the call only.

Parameters:

chrom (str) – The contig name. A contig that the resource lacks gives the block of empty cells, not an error.
start (int) – The first position of the region (1-based, inclusive).
end (int) – The last position of the region (1-based, inclusive).
bin_size (int) – The width of a bin in positions. It must be at least 1.
queries (Sequence[ScoreAggregationQuery]) – The scores to reduce, each with an aggregator. A query with no aggregator uses the default of the score.

Returns:

A float64 array of shape (n_bins, len(queries)). Column j reduces queries[j]. An empty cell is 0 for count and sum. It is NaN for mean, max, min, median and product.

Return type:

numpy.ndarray

Raises:

ValueError – If start is below 1, end is before start, bin_size is below 1, a score is unknown, or an aggregator is not accepted for the score. The read raises before it reads the first fragment.

gain.genomic_resources.genomic_scores.position module

PositionScore – one value per genomic position.

The kind whose records carry no reference or alternative allele, so a position is the whole key. Adds the position-run reads and the binned and aggregated region queries built on them.

class PositionScore(resource: GenomicResource)[source]

Bases: GenomicScore

Position-based genomic score resource.

A position score gives one or more scores for a genomic position or a run of positions. Unlike an allele score, it ignores the reference and alternative alleles. The data lives in a tabix-indexed file, a bigWig file or an in-memory table.

Typical resources are:

conservation scores, such as phastCons and phyloP
mappability scores
GC content
recombination rates
any metric that depends only on the genomic position

A score is built closed. Call open() before the first read.

Examples

import gain.grr as grr

repo = grr.build_genomic_resource_repository()
score = grr.build_position_score_from_resource_id(
    "hg38/scores/phastCons100way", repo)
with score.open() as score:
    values = score.get_scores_at_position("chr1", 12345)
    for begin, end, row in score.fetch_region_segments_scores(
            "chr1", 10000, 20000):
        print(f"{begin}-{end}: {row}")

Variables:

resource (GenomicResource) – The underlying genomic resource.
resource_id (str) – The identifier of the resource.
config (dict) – The validated configuration of the score.
score_definitions (dict[str, GenomicScoreDef]) – The definition of each score, by score id.
HISTOGRAM_COUNT_UNIT (str) – The unit of the histogram counts. For a position score it is "base pairs".
DEFAULT_AGGREGATORS (dict[str, str | None]) – The aggregator name that a region read uses for each value type, when the configuration of the score names none. A value of None means that the type takes no default.

DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'mean', 'int': 'mean', 'str': 'list'}

HISTOGRAM_COUNT_UNIT: ClassVar[str] = 'base pairs': The noun for what a histogram’s n counts – the weighting the statistics scan gives each record of this family: base pairs for a position score (record_weight), alleles, fragments, genes. Every concrete family states its own; the info pages read it.

static get_schema() → dict[str, Any][source]

Return the configuration schema of a position score.

The schema is the one of GenomicScore, with an aggregator key added to the definition of each score.

Returns:: A new Cerberus schema dictionary, which the caller may modify.
Return type:: dict[str, Any]

Return the value of one score at one position.

This is the singular form of get_scores_at_position().

Parameters:

chrom (str) – The chromosome name.
pos (int) – The 1-based position.
score (str | None) – The id of the score. None is accepted only when the resource defines exactly one score.

Returns:

The value of the score. None means that no record covers the position, or that the record has no value for the score.

Return type:

ScoreValue | None

Raises:

ValueError – If the score is closed, the chromosome is unknown, the score id is unknown, or score is None and the resource defines more than one score.

Yield the aggregated value of one score for each bin of a region.

This is the singular form of get_scores_in_bins().

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
bin_size (int) – The width of a bin in positions. It must be at least 1.
score (str | None) – The id of the score. None is accepted only when the resource defines exactly one score.
aggregator (str | None) – The name of the aggregator. None selects the aggregator that the configuration of the score names, or the default one for its value type.
none_value_replacement (ScoreValue | None) – The value that stands for an uncovered position. None means that an uncovered position is skipped by the aggregator.

Yields:

tuple[int, int, ScoreValue] – The first position, the last position and the aggregated value of a bin, in position order. The value is None when the aggregator has no value to report for the bin.

Raises:

ValueError – If the score is closed, the region is not valid, bin_size is less than 1, the score id is unknown, no aggregator can be chosen, or none_value_replacement does not match the value type.

Yield the value of one score at each position of a region.

This is the singular form of get_scores_in_region().

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
score (str | None) – The id of the score. None is accepted only when the resource defines exactly one score.

Yields:

ScoreValue | None – One value for each position, in position order. A None value means that no record covers the position, or that the record has no value for the score.

Raises:

ValueError – If the score is closed, the region is not valid, the chromosome is unknown, or the score id is unknown. The function raises before the first value, not during the iteration.

Reduce a region to one value for one score.

This is the singular form of get_scores_in_region_agg().

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
score (str | None) – The id of the score. None is accepted only when the resource defines exactly one score.
aggregator (str | None) – The name of the aggregator. None selects the aggregator that the configuration of the score names, or the default one for its value type.
none_value_replacement (ScoreValue | None) – The value that stands for an uncovered position. None means that an uncovered position is skipped by the aggregator.

Returns:

The aggregated value. It is None when the aggregator has no value to report, for example when no position of the region has a value. The aggregator sets the type of the value: list gives a list and value_count gives a dict.

Return type:

AggregateValue

Raises:

ValueError – If the score is closed, the region is not valid, the score id is unknown, no aggregator can be chosen, or none_value_replacement does not match the value type.

Return the values of the scores at one position.

Parameters:

chrom (str) – The chromosome name.
pos (int) – The 1-based position.
scores (Sequence[str] | None) – The ids of the scores. None asks for every score that the resource defines, in definition order.

Returns:

One value for each requested score, in the order of scores. A None value means that no record covers the position, or that the record has no value for that score. The tuple is empty when scores is an empty sequence.

Return type:

tuple[ScoreValue | None, …]

Raises:

ValueError – If the score is closed, the chromosome is unknown, or a score id is unknown.

get_scores_in_bins(chrom: str, start: int, end: int, bin_size: int, queries: Sequence[PositionScoreAggregationQuery]) → Generator[tuple[int, int, tuple[str | int | float | bool | None, ...]], None, None][source]

Yield the aggregated values of each bin of a region.

The bins follow the global grid that starts at position 1, so the results of adjacent regions tile and are comparable across calls. The first and the last bin are clipped to the region. The function emits every bin of the region, also a bin that no record touches.

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
bin_size (int) – The width of a bin in positions. It must be at least 1.
queries (Sequence[PositionScoreAggregationQuery]) – The queries. Each names a score, an aggregator and a replacement value for the uncovered positions.

Yields:

tuple[int, int, tuple[ScoreValue, …]] – The first position, the last position and one aggregated value for each query of a bin, in position order. A value is None when the aggregator has no value to report.

Raises:

ValueError – If the score is closed, the region is not valid, bin_size is less than 1, or a query is not valid (see resolve_aggregation_queries()).

Notes

A chromosome that the resource never mentions is not an error. The function emits every bin of it, as for a region that no record covers.

Yield the values of the scores at each position of a region.

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
scores (Sequence[str] | None) – The ids of the scores. None asks for every score that the resource defines, in definition order.

Yields:

tuple[ScoreValue | None, …] – Exactly end - start + 1 tuples, in position order. A tuple has one value for each requested score. A None value means that no record covers the position, or that the record has no value for that score.

Raises:

ValueError – If the score is closed, the region is not valid, the chromosome is unknown, or a score id is unknown. The function raises before the first tuple, not during the iteration.

Reduce a region to one value for each query.

Each position of the region counts once. A record counts once for each position of the region that it covers.

Parameters:

chrom (str) – The chromosome name.
start (int) – The first 1-based position of the region.
end (int) – The last 1-based position of the region.
queries (Sequence[PositionScoreAggregationQuery]) – The queries. Each names a score, an aggregator and a replacement value for the uncovered positions.

Returns:

One value for each query, in the order of queries. A value is None when the aggregator has no value to report. The aggregator sets the type of its value: list gives a list and value_count gives a dict. The tuple is empty when queries is empty.

Return type:

tuple[AggregateValue, …]

Raises:

ValueError – If the score is closed, the region is not valid, or a query is not valid (see resolve_aggregation_queries()).

Notes

A chromosome that the resource never mentions is not an error. It answers as a region that no record covers.

resolve_aggregation_queries(queries: Sequence[PositionScoreAggregationQuery]) → list[tuple[str, str, str | int | float | bool | None]][source]

Resolve each query to a score id, an aggregator name and a value.

Use it to find out whether a query is answerable without reading. The position score annotator calls it once when the pipeline loads.

Parameters:: queries (Sequence[PositionScoreAggregationQuery]) – The queries. Each names a score, an aggregator, and a replacement value for the uncovered positions.
Returns:: One tuple for each query, in the order of queries: the id of the score, the name of the aggregator, and the none_value_replacement of the query. The list is empty when queries is empty.
Return type:: list[tuple[str, str, ScoreValue]]
Raises:: ValueError – If a score id is unknown, a none_value_replacement has a type that the score cannot take, or a query names no aggregator and the value type of the score has no default one.

gain.genomic_resources.genomic_scores.records module

Batch array types and the region/record algebra over them.

The half of the score layer that knows nothing about score resources: the shape of a read batch (RecordArrays, AlleleRecordArrays) and the five functions that decide which part of a record a region gets, or whether it gets it at all.

Two of those partition different things and are deliberately neighbours – clip_span() partitions POSITIONS, owns_record() partitions RECORDS by where they BEGIN – so that the two halves of the algebra have one home and a caller picks the one it means. overlap_fractions_admit() sits beside them without partitioning anything: it SELECTS records by how much of the region, or of themselves, the two share, and a record it rejects is answered by no region at all.

Nothing here imports a score class. That is a property of this module, not yet a saving for its callers: the scan and the statistics layer still reach these names through the package facade, which imports every submodule, so GenomicScore is loaded either way. What it buys is that those callers CAN be pointed at genomic_scores.records directly, one at a time, without anything else moving – migrating them was out of scope for the gain#902 split that created this module.

class AlleleRecordArrays(pos_begin: np.ndarray, pos_end: np.ndarray, values: dict[str, np.ndarray], reference: np.ndarray, alternative: np.ndarray)[source]

Bases: NamedTuple

One batch as AlleleScore.fetch_region_allele_arrays() makes it.

RecordArrays widened by the two key columns an allele row has and a position row does not. The first three fields are that tuple exactly, in the same order, so batch[:3] is a RecordArrays.

That slice is required, not decorative: every consumer of the shared read unpacks three names (the scan’s array door validate_record_arrays() and its coverage accumulator among them), and handing one of them a batch of five raises too many values to unpack. A caller feeding this read into machinery written for the shared one passes batch[:3], and mypy says so too – this type is not a RecordArrays.

reference and alternative are the cells as stored – see the fetch method for why they are the one part of a batch that is not parsed.

alternative: ndarray: Alias for field number 4

pos_begin: ndarray: Alias for field number 0

pos_end: ndarray: Alias for field number 1

reference: ndarray: Alias for field number 3

values: dict[str, ndarray]: Alias for field number 2

RecordArrays

GenomicScore.fetch_region_value_arrays() <.base.GenomicScore.fetch_region_value_arrays> produces it: the RAW one-based begin and end columns, plus one parsed value array per requested score id. Named because the vectorized scan validators are transducers over a stream of these.

Type:: One batch as
Type:: meth

alias of tuple[ndarray, ndarray, dict[str, ndarray]]

clip_span(rec_begin: int, rec_end: int, pos_begin: int | None, pos_end: int | None) → tuple[int, int] | None[source]

Clip a record’s span to a queried window: skip, clip, or refuse.

Returns the part of [rec_begin, rec_end] inside [pos_begin, pos_end], where a None bound means unbounded on that side, or None for a record with no part inside the region: one ending before it (the skip) or one starting past it (which naive clipping would turn into an inverted span, whose width as a weight is negative).

clip_to_region(segments: Iterator[tuple[int, int, T]], pos_begin: int | None, pos_end: int | None) → Generator[tuple[int, int, T], None, None][source]: Clip a segment stream to a region, dropping what falls outside.

overlap_fractions_admit(rec_begin: int, rec_end: int, start: int, end: int, min_region_fraction: float | None, min_record_fraction: float | None) → bool[source]

Whether a record overlaps a region by enough of either side.

With overlap the length of the intersection, min_region_fraction is overlap / region_length – “the record covers at least this much of MY region” – and min_record_fraction is overlap / record_length – “at least this much of the RECORD falls in my region”. The two answer different questions: a 10 bp record inside a 1 Mb region scores ~0.00001 on the first and 1.0 on the second.

Every threshold supplied must hold, and each is compared with >=, so 1.0 means full containment of the side it is about. Both None admits everything, which is what a region read does with no thresholds at all.

0.0 is not the same request as None, though both admit every record a region query actually answers: 0 / length >= 0.0 holds, so 0.0 also admits a record with NO overlap at all. A region query answering with such a row is a backend that over-returns; one cause of it – a tabix table whose index and pos_end name different columns – ADR 0008 refuses at open(), and no threshold here is a check for the rest.

A SELECTION predicate, not a reshaping one: it says whether the record is answered, never what span is reported for it.

rec_end is assumed not to precede rec_begin, so that the record length it divides by is at least 1. This does not check it: the record reads refuse an inverted span at _score_segments(), before any consumer sees the record. Called with one directly, it divides by zero or worse.

owned_records_mask(pos_begin: ndarray, start: int | None, end: int | None) → ndarray[source]: owns_record() over a whole batch’s begin column.

owns_record(begin: int, start: int | None, end: int | None) → bool[source]

Whether a region owns a record, by where that record BEGINS.

The scan’s partition of RECORDS, and the one statement of it (gain#816). Ownership is total, unique and reachable: the regions tile a contig contiguously from position 1, so every record’s begin falls in exactly one of them, and the owning region’s query always returns it – begin inside [start, end] is by itself an overlap. An unbounded side owns everything on that side, which is the same rule with one region.

Contrast clip_span(), which partitions POSITIONS. A statistic that sums over records wants this; one that unions positions wants that. Both live here so the two halves of the algebra have one home.

gain.genomic_resources.genomic_scores.value_extraction module

How a record’s cell becomes a value: the two decisions taken at open.

The seam between a score and its table’s payload. Both decisions are taken once per open, from the table’s declared payload_kind and the score definitions, and neither needs a GenomicScore:

select_value_extractor() picks the per-record read, before table.open();
resolve_score_indices() addresses each definition to a payload column, after it.

That order is load-bearing – each function’s docstring says what forces its half – and it belongs to the caller, open().

The extractors themselves are not here: they live with the backends that know what a payload IS – vcf_scores, bigwig_scores, and score_def for the column read, which also owns the ValueExtractor alias. This module only chooses between them, which is why it sits ABOVE all three rather than inside one of them: hosting the choice in score_def, where gain#1044 put the scoredef lifecycle, would have it import vcf_scores and bigwig_scores, both of which import score_def – the same cycle that kept GenomicScore._build_scoredefs on the class. resolve_score_indices alone could have gone there; it is here so that the seam reads as one module rather than two homes.

resolve_score_indices() mutates the definitions in place and returns nothing; its docstring says who reads what it wrote.

resolve_score_indices(score_definitions: dict[str, GenomicScoreDef], *, table: GenomicPositionTable, resource_id: str) → None[source]

Resolve each score’s configured address to a payload column.

Runs after table.open(), because the by-NAME case is the one thing here that has to consult the table’s header. What a definition’s address means – an INFO key, nothing, or a column – is the table’s PayloadKind, read off the table.

Writes score_index onto the definitions it is handed, and returns nothing: “the defs are finished in place at open” is the contract base states. Two paths read what it wrote – extract_column_value(), on every record of the tabular per-record read, and so the very extractor this module’s other half binds; and fetch_region_value_arrays, which the statistics scan reaches it through. Handing back a mapping would only mean every caller writes it back.

A definition it cannot resolve – no address, two addresses, a name over a table with no header, a name the header does not have, a VCF definition with no INFO key – is refused through score_configuration_error(), so the refusal names the resource and the score in the shape every other definition refusal shares and falls inside cli_errors.RESOURCE_ERRORS by type. Every check is an explicit test, never an assert (python -O strips those) and never a lookup left to raise for itself: a resource config is data, and bad data is reported. For a tabular table, validate_scoredefs holds the CONFIG to the address rules before this runs, but for a headerless table it checks only that no name is stated – so a definition with no address at all reaches this from a real config, while the other refusals are reached through open() only by a definition edited after its config passed.

select_value_extractor(*, score_definitions: dict[str, GenomicScoreDef], table: GenomicPositionTable) → ValueExtractor[source]

Pick the per-record value read for this table’s payload.

ONE decision, per table, taken at open rather than per line. What it turns on is what a record’s PAYLOAD is, which the backend declares in its PayloadKind:

a VARIANT payload carries the variant and the pysam INFO proxies, and its score is an INFO field addressed by name – extract_vcf_value();
a VALUE payload IS the interval’s value, so the read is an identity (extract_bigwig_value()) – or, for the rare resource that configures NA sentinels, an identity plus one membership test (extract_bigwig_value_na()). Which of the two is settled here, from the score definitions, and never per record: the sentinel set is fixed for the life of the open score. A bigWig declares exactly one score (validate_bigwig_scoredefs refuses more), so there is a single answer to give; any states that without depending on it;
a ROW payload is a raw row, read by integer column – extract_column_value().

The declaration is the ONE claim read, and it is read first, because every later decision depends on it: a backend that has not made one is refused with the AttributeError the undefaulted ClassVar raises, before any other check. That a backend’s records are what its kind says – a ROW backend really yields six-slot records with a raw row in the payload slot – is pinned statically, over every backend in the tree, by test_backend_record_contract.py, so the fetch path pays nothing for it.

Module contents

Genomic score resources: the base class, its three kinds, and the algebra.

This package was one 3128-line module until gain#902 split it along the seams the code already had:

records – the batch array types and the region/record algebra over them; imports no score class, so the scan and the statistics layer can use it without pulling GenomicScore in behind it
base – GenomicScore, everything the kinds share
position, allele, fragment – one module per kind
builders – the eight factories, and the dispatch between kinds

and decomposing the class itself (gain#1027) has since added two more:

aggregation (gain#1074) – the machinery the per-kind aggregating reads share (get_scores_in_region_agg(), get_allele_scores_in_region_agg() and get_fragment_scores_overlapping_region_agg()); knows no score class, and is handed the per-kind weight rule rather than reading it
value_extraction (gain#1114) – the two decisions open() takes about how a record’s cell becomes a value: which extractor reads the payload, and which payload column each score def is addressed to
chrom_lengths (gain#1413) – the chromosome-length resolver a caller with a genome hands it to: per contig, every source’s length with the best by rank, and the ContigExtent reason when the table has none; the stored statistics/chrom_lengths.json a repair writes (gain#1576), and the load-on-open, source normalisation and refusal wording behind the score’s own length reads – get_chrom_length(), get_all_chrom_lengths(), get_chrom_length_source() and chrom_length_sources (gain#1577)

This module is a permanent facade, not a deprecation shim. It re-exports every name the pre-split module DEFINED, so each from gain.genomic_resources.genomic_scores import <name> written before the split keeps working, verbatim and indefinitely. Deep imports from the submodules are allowed and equivalent, but no caller is expected to migrate to them: gpf imports through here, and so do out-of-tree callers this repository cannot see or fix (grr_bench, demo repositories). The promise is pinned by test_genomic_scores_facade.py, which spells the surface out rather than deriving it from __all__.

What it deliberately does not carry is the ~70 names the pre-split module merely IMPORTED and so leaked as incidental re-exports – np, copy, Record, GenomicScoreDef, ScoreValue and the like. Reaching through this module for one of those worked by accident, never by intention, and an AST scan of gain, gpf and grr_bench (2026-08-31) found no caller that did. Import them from the module that defines them.

What has since moved OUT is the pair of scan validation rules. validate_records and validate_record_arrays were @abstractmethod on GenomicScore with one body per kind here; they are now singledispatch functions in gain.genomic_resources.statistics.record_validation, registered per kind, because the statistics scan is their only caller and a read class should not carry a rule only its consumer applies (gain#1269, ADR 0027). They were never part of the pre-split module’s re-exported surface – they were instance methods – so the facade promise above is untouched.

What did NOT move here is the resource implementation – genomic_scores_impl, which gain#1007 split into scan and classes and gain#1210 then laid out as this package is, one module per kind – nor the decomposition of the GenomicScore class itself, which is gain#1027 and is deliberately sequenced after this split.