gain.genomic_resources.genomic_scores package
Submodules
gain.genomic_resources.genomic_scores.aggregation module
Reducing a region to one value per requested score.
The machinery aggregate_region() orchestrates,
apart from any score class: resolving a caller’s request list to
(score_id, aggregator) pairs, building a fresh aggregator for each, and
folding a stream of fetched segments into one value per request.
Resolving and building are separate steps here because they are separate
questions. score_def_for() and resolve_aggregator_name()
answer “is this request answerable, and by what”, which a caller may ask
without meaning to read – since gain#1131
resolve_aggregation_queries() asks exactly
that when an annotation pipeline loads, so a misconfigured attribute is
refused before any annotatable arrives. build_region_aggregator()
answers “give me somewhere to accumulate”, which only a read needs, and
which a read needs freshly every time.
Nothing here knows what a score KIND is. The one thing that differs
between kinds – how many times a record’s value counts – reaches
fold_region_segments() as weigh, the kind’s own
record_weight(). The fold applies what it is
handed and does not re-derive or cross-check it: the rule is stated once
per kind, on the kind (see
test_the_weight_rule_is_stated_once_per_kind). Whether the segments
were clipped to the query window before they got here is settled the same
way, by the kind’s _aggregation_segments(), so
the fold carries no flag saying which kind it serves.
Every BACKEND feeds one path here – this generic weighted stream over
fetched records – and a backend-specific fast path was considered and
rejected to keep it that way; see
.out-of-scope/bigwig-stats-pushdown.md. That is a claim about
backends, not about the package: PositionScore folds
its own segments for the aggregated plane, and converging the two is
gain#1027’s remaining work, not this module’s promise.
- gain.genomic_resources.genomic_scores.aggregation.build_region_aggregator(score_id: str, aggregator: str, *, resource_id: str) Aggregator[source]
Build a FRESH aggregator, naming the resource if it cannot.
Fresh per call, not reused: an aggregator is a mutable accumulator and explicitly not thread-safe (see
Aggregator). Reuse is an annotator optimisation resting on being single-threaded; a score may be read from several threads (the web api’s thread pool), so this cannot assume the same.Aggregator.buildraises a bareKeyError('mediann')for an unknown name, saying nothing about which score asked for it.
- gain.genomic_resources.genomic_scores.aggregation.build_region_aggregators(requests: Sequence[tuple[str, str]], *, resource_id: str) list[Aggregator][source]
One FRESH aggregator per request, parallel to the request list.
build_region_aggregator()over a resolved request list – the shape every folding read handsfold_region_segments(), built once per READ and never held on the score, for the reason that function gives. Fresh accumulators remove one hazard, not the class of them: two concurrent region reads of one open score still share the table’s line iterator, which is each read’s “one live region read at a time” rule and not this function’s promise.
- gain.genomic_resources.genomic_scores.aggregation.distinct_score_ids(score_ids: Iterable[str]) list[str][source]
The DISTINCT ids among
score_ids, in the order asked for.One fetch serves every aggregation request, so the same list must both name what is fetched and index the values that come back – which is why the ORDER is part of the answer, and why every aggregating read in this package derives it here. A second spelling that ordered the scores differently would not fail; it would have every aggregator quietly reading its neighbour’s column.
Takes the ids rather than the requests they came off, because a request is shaped differently on each surface – a pair here, a
PositionScoreAggregationQueryresolved to a triple on the position score’s plane – and none of that is what the derivation is about. Each surface projects its own shape at its call site; the request list gets a named projection,request_score_ids(), because it is the one shape TWO readers (aggregate_region()and the fragment kind’s folding read) project, and each hands the result on to the fold.Note this is the aggregating reads’ derivation, not a package-wide one: nothing else in the package fetches one list and indexes it twice. The position annotator’s point read asks one score per attribute, a score named twice included, and pairs the answers back by position, so it has nothing to dedupe.
- gain.genomic_resources.genomic_scores.aggregation.fold_region_segments(segments: Iterable[tuple[int, int, Sequence[str | int | float | bool | None]]], aggregators: list[Aggregator], requests: Sequence[tuple[str, str]], *, score_ids: Sequence[str], weigh: Callable[[int, int], int]) list[str | int | float | bool | None][source]
Fold one region read into one value per request.
segmentsis a stream of(left, right, values)asfetch_region_segments_scores()yields it, withvaluespositional and parallel toscore_ids– the list the caller derived (request_score_ids()) to name what it fetched, handed on here so the fold indexes exactly the columns the fetch carried; one derivation cannot disagree with itself, where two (the fold once derived its own) had to be kept agreeing. That is how a request finds its column: two requests for one score share the fetch and keep separate accumulators. AnySequenceof values will do – the fold only ever indexesvalues[column]– so a kind that hands over tuples folds exactly as one that hands over lists.aggregatorsis parallel torequestsand built by the CALLER, which is what lets an invalid aggregator name be refused before the region is read at all (seeaggregate_region(), which explains why that ordering matters).weighis the caller’s per-kind weight rule, applied as handed: it turns a segment’s span into the number of times that record’s value counts. It is not second-guessed here, and neither is the stream – whether a record was first cut down to the query window is the caller’s business, settled before the segments arrive (see_aggregation_segments()). A kind that counts a record once counts it wherever the point it collapses to falls, window or not.
- gain.genomic_resources.genomic_scores.aggregation.request_score_ids(requests: list[tuple[str, str]]) list[str][source]
distinct_score_ids()of a request list’s scores.Derived ONCE per aggregating read and handed to both ends of it: the reader names the scores to fetch with it and passes the same list to
fold_region_segments()asscore_ids, which says why one list serves both. The position score’s plane projects its own shape inline instead – one reader, nothing to hand on.
- gain.genomic_resources.genomic_scores.aggregation.resolve_aggregation_queries(queries: Sequence[ScoreAggregationQuery] | None, *, score_definitions: dict[str, GenomicScoreDef], all_scores: list[str], resource_id: str) list[tuple[str, str]][source]
Resolve kind-neutral queries to
(score_id, aggregator NAME)pairs.resolve_aggregator_requests()for the query surface: the same two questions (score_def_for(),resolve_aggregator_name()), the query’s remedy, andNoneexpanded to every score with its own default. Stops at the NAME, asresolve_aggregation_queries()does, so a caller can ask whether a query list is answerable without reading – an annotator asks exactly that as its pipeline loads. Building the accumulators is the READ’s business, per call (build_region_aggregator()).Shared by the fragment and allele folding reads, which is what promoted it here from the fragment kind: this package keeps a derivation on the one kind that needs it and moves it here when a second does. The position kind keeps its own resolver because its query carries a third field,
none_value_replacement, that has to be judged between the two questions – see that method for why the two must not be merged. A pair has nowhere to put that field, and the signature keeps its query out: aPositionScoreAggregationQueryis a sibling of the neutral query, not a subclass, so it does not type-check here – seeScoreAggregationQueryfor why (gain#1302).Not routed through
resolve_aggregator_requests(), though that returns exactly these pairs and already expandsNone: it hardcodesPAIR_AGGREGATOR_REMEDY, and a query surface must sayQUERY_AGGREGATOR_REMEDY. Giving it aremedyparameter would change a functionaggregate_region()also calls, for the sake of one sentence.
- gain.genomic_resources.genomic_scores.aggregation.resolve_aggregator_name(aggregator: str | None, score_def: GenomicScoreDef, *, resource_id: str, remedy: str) str[source]
The aggregator to reduce a score with: the caller’s, else its own.
The second question, and the one statement of the rule that a score with neither is refused.
remedyis the only part that differs between surfaces, because it tells the caller what to write and the two surfaces take an aggregator in different places – a(score_id, aggregator)pair forresolve_aggregator_requests(), a field on the query forget_scores_in_region_agg(). Pass one ofPAIR_AGGREGATOR_REMEDY/QUERY_AGGREGATOR_REMEDY, which is why they live here and not at the call sites. Everything ahead of the remedy is shared, and pinned so bytest_both_surfaces_state_the_missing_default_rule_identically.The score is named by
score_defrather than beside it:score_definitionsis keyed byscore_idat every construction path, so a separate argument would be one the caller could contradict.
- gain.genomic_resources.genomic_scores.aggregation.resolve_aggregator_requests(scores: list[str | tuple[str, str]] | None, *, score_definitions: dict[str, GenomicScoreDef], all_scores: list[str], resource_id: str) list[tuple[str, str]][source]
Normalize the request list to
(score_id, aggregator)pairs.Two arguments, because they answer two questions:
all_scoressays which scores “all of them” means and in what order, for ascoresofNone;score_definitionssays what each one IS. Today a score class answers the first withlist(self.score_definitions), so the two cannot disagree – but which scores a resource OFFERS is the class’s decision to change, and this asks for it rather than assuming the answer stays derivable.
- gain.genomic_resources.genomic_scores.aggregation.score_def_for(score_id: str, *, score_definitions: dict[str, GenomicScoreDef], resource_id: str) GenomicScoreDef[source]
The definition an aggregation request names, refusing an unknown one.
The first of the two questions every aggregation request asks, and the one statement of the refusal when the answer is no. Whether the request arrived as a bare score id, as a
(score_id, aggregator)pair, as aScoreAggregationQueryor as aPositionScoreAggregationQuerychanges nothing about it: the resource either defines that score or it does not, and the caller is told which ones it has either way.
gain.genomic_resources.genomic_scores.allele module
AlleleScore – one value per (position, ref, alt) allele.
The kind keyed by a variant rather than a position, in either of two modes
(substitutions and alleles). Its reads widen the shared batch with
the two key columns a position row does not have – see
AlleleRecordArrays for why that widening is a slice of
the shared type rather than a separate one.
- class gain.genomic_resources.genomic_scores.allele.AlleleAggregate(values: tuple[str | int | float | bool | None, ...], allele_keys: tuple[str, ...] | None)[source]
Bases:
objectWhat one folding read reduced a region to, off a single walk.
valuesis parallel to the QUERIES asked, never keyed by score id: one score asked twice with two aggregators is two queries and therefore two values, which a mapping keyed by score id would silently collapse to one.allele_keysisNoneunless the read was asked for them; when asked, the distinct keys in first-seen order (D1, D2 of the allele folding-read design, gain#1132). Built off the same walk the values were folded from, so nothing a caller can do makes the two disagree about which records were seen.- allele_keys: tuple[str, ...] | None
- values: tuple[str | int | float | bool | None, ...]
- class gain.genomic_resources.genomic_scores.allele.AlleleScore(resource: GenomicResource)[source]
Bases:
GenomicScoreAllele-specific genomic score resource.
An AlleleScore provides scores that depend on specific alleles at genomic positions. Unlike PositionScore, AlleleScore considers both the reference and alternative alleles when computing scores. This makes it suitable for variant-specific predictions and annotations.
AlleleScore supports two operational modes:
SUBSTITUTIONS mode: Scores are specific to nucleotide substitutions (e.g., A>T, C>G). This mode is optimized for single nucleotide variants and considers the directionality of the change. Used by resources like CADD, which provide substitution-specific scores.
ALLELES mode: Scores are associated with specific alleles at positions, without considering the reference allele. This mode supports insertions, deletions, and more complex variants. The score depends on the alternative allele itself rather than the substitution pattern.
Typical use cases include: - Variant pathogenicity scores (e.g., CADD, DANN) - Functional impact predictions (e.g., PolyPhen, SIFT scores) - Splice site predictions - Regulatory variant scores - Any metric that depends on specific alleles
The score data is typically stored in VCF files or tabix-indexed tables with reference and alternative allele columns.
Example
>>> from gain.genomic_resources.repository_factory import ( ... build_genomic_resource_repository ... ) >>> repo = build_genomic_resource_repository() >>> resource = repo.get_resource("cadd_v1_6") >>> score = build_score_from_resource(resource) >>> with score.open() as score: ... # Fetch scores for a specific variant ... values = score.fetch_allele_scores( ... "chr1", 12345, "A", "T" ... ) ... # Iterate over the alleles in a region. The nucleotides ... # come off the record; the values come off the score. ... for record in score.fetch_records("chr1", 10000, 20000): ... values = score.get_score_values_from_record( ... record, score_defs ... ) ... print(f"{record[POS_BEGIN]} " ... f"{record[REF]}>{record[ALT]}: {values}")
Reducing those values over a region is the resource’s own job:
get_allele_scores_in_region_agg()folds a region in one streaming walk, one value perScoreAggregationQuery, with the allele keys beside them when asked, and the allele annotator’s region mode reads through it (gain.annotation.allele_score_annotator).- Variables:
resource (gain.genomic_resources.repository.GenomicResource) – The underlying GenomicResource object
resource_id – Unique identifier for the resource
config – Configuration dictionary for the score
table – GenomicPositionTable for data access (typically VCF)
score_definitions (dict[str, ScoreDefT]) – Dictionary mapping score IDs to their definitions
mode – Operating mode (SUBSTITUTIONS or ALLELES)
- Key Methods:
fetch_allele_scores: Get score values for a specific variant fetch_allele_records: Get the records of a region, filtered, telling a region holding no allele apart from one whose alleles were all rejected get_allele_scores_in_region_agg: Reduce the alleles of a region to one value per query – and their keys – in one walk, telling the same two answers apart fetch_region_segments_scores: Iterate over allele scores in a genomic region substitutions_mode: Check if operating in SUBSTITUTIONS mode alleles_mode: Check if operating in ALLELES mode
- Configuration:
The resource configuration should specify:
table.filename: Path to the data file (usually VCF)
table.reference: Column/field containing reference alleles
table.alternative: Column/field containing alternative alleles
allele_score_mode: Either “substitutions” or “alleles” (optional)
scores: List of score definitions with an optional aggregator specification
- DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'max', 'int': 'max', 'str': 'list'}
- class Mode(*values)[source]
Bases:
EnumAllele score mode.
- ALLELES = 2
- SUBSTITUTIONS = 1
- alleles_mode() bool[source]
Return True if the score is in alleles mode.
- fetch_allele_records(chrom: str, pos_begin: int | None, pos_end: int | None, *, score_filter: ScoreFilter | None = None) list[tuple[Any, ...]] | None[source]
Return the allele records overlapping a region, or
None.Nonemeans no record overlaps the region at all – absent data. A list means records were there, and holds the onesscore_filteraccepted, which may be none of them:[]is an empty selection. The two are different answers and a caller may well report them differently, which is the whole reason this read exists rather thanGenomicScore.fetch_records()serving the same purpose – an iterator makes both an empty stream.FragmentScore.fetch_fragment_scores()deliberately has noNone, and the difference is in the data rather than in taste: a region is spanned by fragments as a matter of course, so “no fragment covers it” is a count of zero. Allele records sit at points, most of a genome carries none, and a region holding no allele is the same absent data thatfetch_allele_scores()already answersNonefor.score_filter– fromGenomicScore.compile_filter()– is applied inside the read, so a rejected record costs no value extraction and the ownership check covers this path too.Records, not values: a caller wants the nucleotides and the position as well as the scores, reads several scores off one record, and may read scores this method was never told about. Handing back dicts would settle all three for it, and wrongly. Values come off a record through
get_score_value_from_record().A contig the resource does not have is refused, as the other allele reads refuse it, and refused from the call: answering
Nonewould make a caller’s typo indistinguishable from real absent data. This read materialises, so there is no generator body to defer the refusal into – unlikeGenomicScore.fetch_records(), which reports it from the first record read.Materialising is what the
list/Noneanswer costs: a caller reading a region far larger than it can hold wants the streaming read instead – or, to reduce the region rather than hold it,get_allele_scores_in_region_agg(), which shares this read’s two answers and its peek. Records the filter rejects are never held, though – only the accepted ones accumulate.
- fetch_allele_scores(chrom: str, position: int, reference: str, alternative: str, scores: list[str] | None = None, *, score_filter: ScoreFilter | None = None) dict[str, str | int | float | bool | None] | None[source]
Fetch score values at specified genomic position and nucleotide.
score_filterselects whether this allele is reported at all; an allele it rejects reads as absent, exactly as an unmatched one does.
- fetch_region_allele_arrays(chrom: str, pos_begin: int | None, pos_end: int | None, scores: list[str], *, batch_size: int = 100000) Generator[AlleleRecordArrays, None, None][source]
Fetch a region as column arrays, nucleotides included.
GenomicScore.fetch_region_value_arrays()widened by the two columns an allele row has and a position row does not, for a caller scanning a whole region for allele content rather than values – the allele statistics, above all. Each batch is that method’s(pos_begin, pos_end, {score_id: values})followed by thereferenceandalternativearrays, asAlleleRecordArrays.The nucleotides are RAW; the scores beside them are parsed. That asymmetry is deliberate and is the whole contract. A score column goes through its definition’s column parse, so an NA sentinel arrives as that score’s non-value; these two columns go through nothing at all. Whatever the row held is what the array holds – no upper-casing, no stripping, no sentinel handling – because
build_tabular_parser()reads them equally verbatim, and a consumer reading a region through this method and a region throughGenomicScore.fetch_records()must be handed the same strings rather than two dialects of them. Whoever wants them normalised normalises them, once, where the meaning of the normalisation is known.Refused, rather than emulated, for a score this facade cannot serve it for – ask
supports_region_allele_arrays()first. The guards run when this method is CALLED, not on the firstnext(), which is why the streaming half lives in_allele_array_batchesrather than ayieldhere.
- get_allele_scores_in_region_agg(chrom: str, start: int, end: int, *, queries: Sequence[ScoreAggregationQuery] | None = None, allele_keys: Sequence[str] | None = None, score_filter: ScoreFilter | None = None) AlleleAggregate | None[source]
Reduce the alleles in a region to one value per query, or
None.The kind’s folding read (gain#1132): what
fetch_allele_records()would hand back, already reduced, in ONE walk that holds no record.queriesofNonemeans every score the resource defines, each with its own default aggregator; a query’s own aggregator wins over the default. An allele line is weighed byrecord_weight(), which counts it once.Noneis absent data, judged beforescore_filterand with ownership checked first, exactly asfetch_allele_records()answers it – both through_selected_allele_records(), which states the contract. AnAlleleAggregatewhose fold saw nothing is the other answer: records were there and the filter rejected every one, so each aggregator answers for an empty selection (listgives[],maxgivesNone, …). That asymmetry with the fragment kind’s folding read is a property of the data, and ADR 0017’s Consequences say why.allele_keysofNone– the default – builds no keys, so a caller that wants none pays nothing per record for them. A sequence, possibly empty, asks for the keys and names the score ids to suffix each with:()is the barechrom:pos:ref:alt.allele_key()states the format; the keys come back distinct and in first-seen order, off the same walk the values were folded from.The REQUEST is checked when this is called: an unknown score id – in a query or in
allele_keys– a query with no aggregator to resolve to, an unknown contig and a foreign filter are all refused before a record is read. Aggregators are built fresh per call (build_region_aggregators()).
- static get_schema() dict[str, Any][source]
The
GenomicScoreschema plus the allele-specific keys.allele_score_mode,merge_vcf_scores, and the table’sreference,alternativeandvariantcolumn definitions.
- classmethod record_weight(left: int, right: int) int[source]
An allele line counts once.
Several records share a position – one per ref/alt pair – and each weighs 1. Structurally so:
fetch_region_segments_scores()yields(pos, pos, values), collapsing the record to a point however wide an optionalpos_endcolumn reaches, so a span weight would not merely be a different choice, it would disagree with the per-record read.A constant, which is elementwise: the base’s
record_weights()fills it out to a batch’s shape.
- resolve_aggregation_queries(queries: Sequence[ScoreAggregationQuery] | None) list[tuple[str, str]][source]
Resolve each query to its
(score_id, aggregator NAME)pair.The shared
resolve_aggregation_queries()on this score, made public soAlleleScoreAnnotatorcan refuse a bad attribute as the pipeline loads (D6 of the allele folding-read design) – see that function for why it stops at the name.
- resolve_allele_key_scores(allele_keys: Sequence[str]) list[GenomicScoreDef][source]
The definitions of the scores an allele-key request suffixes.
The other half of what
get_allele_scores_in_region_agg()checks on the call, made askable without reading: the same refusal every allele read gives an unknown score id, with the valid names listed, which is how the annotator refuses a badinclude_attributesas the pipeline loads rather than per record.
- substitutions_mode() bool[source]
Return True if the score is in substitutions mode.
- supports_region_allele_arrays(scores: list[str]) bool[source]
Whether
fetch_region_allele_arrays()will serve these scores.GenomicScore.supports_region_value_arrays()– the backend and the score value types – plus the one condition that is this read’s alone: the table must declare at least one of the two key columns, or there is nothing for it to carry that the shared read does not already give.The columns are configured independently, and one of them is enough. A table declaring only
alternativeis served, with the missing side yielded as theNonethe record carries for it; that is what keeps this read andGenomicScore.fetch_records()the same answer rather than two. A bigWig-backed score is turned away here without being named: it has no such columns to declare.Answerable on an UNOPENED score, as its counterpart is – and, in one case, conservative there rather than exact. A table’s key columns are resolved when it opens (
_set_core_column_keys), from the config and, failing that, from the header. So this asks the same two questions in the same order, using whichever of them can be answered yet: the declaration always, and the header when the table already has one (header_mode: listnames it in the config, and an opened table has read it).That leaves exactly one gap: a
header_mode: filetable that names its key columns nowhere but inside its own data file answersFalseuntil it is opened andTrueafter. The asymmetry is the file’s, not this method’s – a header cannot be known without reading it – and it errs the safe way, because a caller toldFalsereads per-record and gets the same rows.
- gain.genomic_resources.genomic_scores.allele.allele_key(chrom: str, pos: int, ref: str | None, alt: str | None, suffix: Sequence[str | int | float | bool | None] = ()) str[source]
The allele key:
chrom:pos[:ref:alt][:v1,v2].An allele’s identity as annotation output spells it, and the ONE statement of that spelling: the folding read builds it per record and the annotator’s exact-match path builds it from the annotatable, and the two must not drift. It lives on the score rather than in the annotator because the key is the record’s identity, which is score-layer knowledge.
The nucleotides are omitted when EITHER is absent: a table may declare only one of the two key columns, and
1:10:None:Cwould name an allele that does not exist.suffixis the values of the scores a caller asked to append, in the order asked, each rendered asstringify()renders it for output and joined with,; it is part of the key’s identity, so two records at one allele that differ in a suffixed score are two keys.
gain.genomic_resources.genomic_scores.base module
The GenomicScore base class.
Everything the three score kinds share: config parsing and score-def
construction, the open/close lifecycle over the position table, and the
record and array reads. The kinds themselves live in position,
allele and fragment, and the factories that dispatch
between them in builders.
Decomposing this class – so that a kind’s author reads the handful of hooks
their kind overrides rather than the whole base – is gain#1027. Its first
extraction (gain#1044) moved the scoredef lifecycle to
score_def and took this module under the
1500-line cap, so the file-scoped too-many-lines pragma gain#1007 added
here when it restored that cap is gone; its second (gain#1074) moved the
region-aggregation machinery to aggregation, leaving
GenomicScore.aggregate_region() here as the orchestrator that hands
it the per-kind weight rule; its third (gain#1114) moved the
value-extraction seam – picking the per-record read, and addressing each
score def to a payload column – to value_extraction, leaving
GenomicScore.open() calling both in the order that seam requires and
the two public per-record getters here. The remaining seams are #1027’s
other children.
A score’s defs are finished in place at open:
resolve_score_indices() writes score_index onto
the definitions this class already holds rather than handing back new ones
(it says there who reads them). The earlier half of the lifecycle differs:
finish_scoredefs runs inside GenomicScore._build_scoredefs(),
before there is a score_definitions to write onto, and so returns the
mapping.
- class gain.genomic_resources.genomic_scores.base.GenomicScore(resource: GenomicResource)[source]
Bases:
ScoreResource[GenomicScoreDef]Base class for genomic score resources.
GenomicScore provides a unified interface for accessing and managing genomic annotation scores stored in various formats. It serves as the foundation for specialized score types including PositionScore (position- based scores) and AlleleScore (variant-specific scores).
This abstract base class handles: - Resource configuration validation and normalization - Score definition management and parsing - File format abstraction through GenomicPositionTable - Histogram and statistics management - Default annotation attribute configuration - Context manager protocol for resource lifecycle
Score resources can be stored in multiple formats: - Tabix-indexed files (TSV, BED) - VCF files (particularly for allele scores) - BigWig files (for position scores) - In-memory tables (for testing)
- Configuration Structure:
A genomic score resource requires a YAML configuration file (genomic_resource.yaml) specifying:
type: Resource type (position_score, allele_score)
table: Table configuration with filename, format, and column mappings for chrom, pos_begin, pos_end (and ref/alt for allele scores)
scores: List of score definitions with id, type, name/index, description, and optional aggregators
default_annotation: Optional list specifying which scores to include in default annotations with optional name mappings
histograms: Optional histogram configurations for statistics
- Score Definition:
Each score in the resource is defined with:
id: Unique identifier for the score
type: Data type (int, float, str, bool)
name/index: Column name or index in the data file
desc: Human-readable description
na_values: Values to treat as missing/NA (optional)
hist_conf: Histogram configuration for statistics (optional)
aggregator: Default aggregator (optional). How several values for one annotatable are reduced to one; the default depends on the resource type and the score’s value type.
- Usage Pattern:
Genomic scores follow a resource lifecycle pattern:
Build/retrieve the resource from a repository
Create a score object from the resource
Open the score to initialize data access
Query scores using fetch methods
Close the score to release resources
- Example using context manager:
>>> from gain.genomic_resources.genomic_scores import ( ... build_score_from_resource_id ... ) >>> score = build_score_from_resource_id("phastCons100way") >>> with score.open(): ... # Score is open and ready to use ... chromosomes = score.get_all_chromosomes() ... scores = score.get_all_scores() ... # Query data... >>> # Score is automatically closed
- Statistics and Histograms:
GenomicScore supports automatic statistics generation including:
Value distribution histograms
Min/max ranges for numeric scores
Category frequencies for categorical scores
Custom histogram configurations per score
- Variables:
resource (GenomicResource) – The underlying genomic resource object
resource_id (str) – Unique identifier for the resource
config (dict) – Validated and normalized configuration dictionary
table (GenomicPositionTable) – Data access abstraction layer
score_definitions (dict[str, GenomicScoreDef]) – Mapping of score IDs to their internal definitions including parsers and metadata
table_loaded (bool) – Flag indicating if the table is currently open
- Key Methods:
open(): Initialize the score resource for data access close(): Release resources and close the data table get_all_scores(): Get list of all available score IDs get_all_chromosomes(): Get list of all available chromosomes has_chromosome(): Whether one chromosome is available get_score_definition(): Get metadata for a specific score get_default_annotation_attributes(): Get default annotation config get_histogram(): Load histogram for a score (if available) get_score_range(): Get value range for a numeric scores
- Per-kind Methods:
A kind whose records read as something other than the span they cover states that ONCE, by overriding:
_score_segments(): what a region’s raw records mean for this kind.
region_values_from_recordsis the request resolution followed by it,fetch_region_segments_scoresis THAT applied tofetch_records, and the statistics scan is it applied tovalidate_records(score, fetch_records(...))– so a kind states its reading once and every consumer gets it (ADR 0008). Override this and notregion_values_from_records: the resolving entry is shared by every kind, and a read holding an already-resolved request composes this body without going through it (gain#1282).record_weight(): how many times one record’s value counts when a region is aggregated. Every reader goes through it – the annotators’
aggregate_region, the per-record scan, and the bulk scan viarecord_weights(gain#1095)._aggregation_segments(): whether those records are cut down to the queried window before they are weighed. Unlike the others this HAS a default – not clipping – because it is a consequence of the weight rule rather than a rule of its own: a kind that counts a record once counts it wherever it falls.
All but the last have no default. A kind that inherited one would be weighed by a rule nobody chose for it, which is the failure ADR 0008 exists to undo.
A kind’s two validation rules are not on this list, and not on this class: they are registered per kind in
gain.genomic_resources.statistics.record_validation, whose default refuses a kind nobody wrote one for (ADR 0027).
See also
PositionScore: For position-based genomic scores
AlleleScore: For variant-specific genomic scores
GenomicResource: Base resource abstraction
GenomicPositionTable: Table format abstraction
- DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {}
- aggregate_region(chrom: str, pos_begin: int | None = None, pos_end: int | None = None, scores: list[str | tuple[str, str]] | None = None) list[str | int | float | bool | None][source]
Reduce a region to one value per requested score.
The aggregating counterpart of
fetch_region_segments_scores(), which it is built on: that method yields one entry per record, this one folds those entries into a single value per request.Each request is either a score id – aggregated with the resource’s own default, which
score_def.finish_scoredefsresolved from this class’sDEFAULT_AGGREGATORS– or a(score_id, aggregator)pair naming one explicitly. The aggregator string is whatever the config accepts, parametrized forms (join(,)) included.Returns a list parallel to ``scores``, not a dict. One score may legitimately be requested twice with different aggregators –
["s", ("s", "max")], which is what an annotation config does when it exposes one source as both a min and a max attribute – and a dict keyed by score id would silently drop one of them.An empty region is not an error: each aggregator answers for itself.
listreturns[];maxreturnsNone; and so doescount, which chooses to report nothing rather than 0 for an empty region (seeCountAggregator.get_final). This method does not second-guess any of them. That is deliberately unlike the per-position reads (get_scores_at_position,fetch_allele_scores), which answerNonewhere there is no data – aggregating nothing is a well-defined question, reading a value where there is none is not.Values reach the aggregator exactly as the record carried them,
Noneincluded, because that is what the annotators do (each aggregator decides what a null means for it) and the point of this method is to give the answer they would.
- property chrom_length_source: ChromLengthSource
What the score’s own file can say a contig’s length is.
Declared on the backend class, so answered on a closed score – which is what a caller weighing whether to open the table for its lengths at all (coverage’s second rung, gain#1448) needs. The lengths themselves come from
derive_chrom_lengths().
- close() None[source]
Close the underlying table and mark the score not open.
open()may be called again afterwards.
- compile_filter(expression: str) ScoreFilter[source]
Compile a boolean expression into a filter over this score.
The expression names this resource’s own scores and relates them with
>,>=,<,<=,==,!=andin, combined withnot,andandor; the result is passed back to any of the record reads asscore_filter.Raises
ScoreFilterErroron an expression that does not parse or that names a score this resource does not define. Seecompile_score_filter()for what compiling settles,docs/adr/0017-score-filtering-is-a-score-capability.mdfor why the capability sits on the score, anddocs/adr/0018-score-filter-grammar-extension.mdfor the language’s precedence and what a name may contain.
- config: dict
- fetch_records(chrom: str, pos_begin: int | None, pos_end: int | None, *, score_filter: ScoreFilter | None = None) Generator[tuple[Any, ...], None, None][source]
Yield the records of a region, optionally filtered.
A caller reads a record’s positional fields from its slots (
record[CHROM],record[POS_BEGIN], …) and a score value throughget_score_value_from_record()orget_score_values_from_record()on this score.score_filteris a predicate fromcompile_filter(), applied to each record; only records it accepts are yielded.None– the default – yields what the table yields, and is the whole of what this method did before filtering became a score capability.chromis required, here and throughout the region-read family. A caller that wants every record of a table asks the table:score.table.get_all_records().Nothing here is checked before the first
next(), the filter’s ownership included: every backend’sget_records_in_regionis itself a generator function, so an unknown contig has always been reported from the first record read rather than from the call, and there is no eagerness left to preserve by structuring this any other way. That is a property of this read rather than a rule for the family: a read that materialises has no generator body to defer a refusal into, andAlleleScore.fetch_allele_records()accordingly refuses from the call.
- fetch_region_segments_scores(chrom: str, pos_begin: int | None = None, pos_end: int | None = None, scores: Sequence[str] | None = None, *, score_filter: ScoreFilter | None = None) Generator[tuple[int, int, list[str | int | float | bool | None]], None, None][source]
Yield
(begin, end, values)per record touching the region.One tuple per underlying RECORD – a segment, at that record’s own extent – not one value per position. The region’s records, read as this kind means them. A record straddling the region’s edge is reported whole: what a partial overlap means depends on what the caller is computing, so a caller answering a question about the window composes
clip_to_region()over this stream, or callsclip_span()per segment (ADR 0008).score_filter– fromcompile_filter()– travels tofetch_records()and nowhere else: the records it rejects never reach the transform, so a rejected record costs no value extraction. It reads the RECORD, so it may name any score the resource defines, including one outsidescores. It is not a second kind of read and it selects nothing the record read would not: the composition below is the same one either way, with a filtered stream in place of an unfiltered one.Its ownership check is the one refusal here that does NOT land on the call. It rides
fetch_records(), whose generator body defers it and which says so, where the request checksregion_values_from_records()runs are eager – so a filter compiled against a different score is refused on the firstnext(), not from the call that a closed score, an unknown contig and an unknown score id are refused from.A plain read: it checks nothing. The statistics scan reads the same records through the same transform with
validate_records()composed in front, and that extra link – visible at the consumer, ingenomic_scores_impl/scan.py– is the whole of the difference between the two (ADR 0008).One body per kind, in
_score_segments(), rather than one per kind per consumer: two that had to agree is how the paths drift. The fragment plane reads through this method for that reason (gain#1272), and the position plane reaches the same body with an already-resolved request (gain#1282) – different entries, one reading.
- fetch_region_value_arrays(chrom: str, pos_begin: int | None, pos_end: int | None, scores: list[str], *, batch_size: int = 100000) Generator[tuple[ndarray, ndarray, dict[str, ndarray]], None, None][source]
Fetch a region as column arrays, without building a record per row.
The bulk counterpart of
fetch_records(), for a caller that scans a whole region and wants columns rather than rows – statistics, above all. Each batch is(pos_begin, pos_end, {score_id: values}): the one-based position arrays, plus one array of parsed values per requested score.Values are parsed, by the same contract the per-record read uses. Each column goes through
GenomicScoreDef.parse_array(), whose agreement with the per-valueGenomicScoreDef.parse_value()is pinned by test_parse_array_agrees_with_parse_value_fuzz. So NA sentinels and unparseable cells arrive as that score’s non-value, whatever the backend stores underneath: afloatorintscore yieldsfloat64withnanfor no value, astrscore anobjectarray withNone. The array’s dtype follows the score’s declared type, not the backend’s – a caller reading several scores in one batch can be handed both shapes.That parse is why a value type the definition cannot parse as a column is refused, and why
supports_region_value_arrays()asks about the scores and not only about the backend.It does NOT clip. A record overlapping the region’s start is yielded whole, exactly as
fetch_records()yields it; trimming to[pos_begin, pos_end]is the caller’s, because what a partial overlap means depends on what the caller is computing.batch_sizeis a HINT. A backend whose read granularity is fixed by its own windowing –BigWigTable, whose batches are sized by its adaptive fetch window – ignores it.Each score id gets an array of its own – the parse builds one per id, so two ids sharing a payload column do not alias.
The guards below run when this method is CALLED, not on the first
next()– which is why the streaming half lives in_value_array_batchesrather than ayieldhere.
- get_all_chromosomes() list[str][source]
The chromosome names the score’s table holds, in table order.
Raises
ValueErroron a score that is not open.
- get_config() dict[str, Any][source]
The configuration, validated and normalized at construction.
- get_default_annotation_attribute(score_id: str) str | None[source]
Return default annotation attribute for a score.
Returns None if the score is not included in the default annotation. Returns the name of the attribute if present or the score if not.
- get_default_annotation_attributes() list[Any][source]
Collect default annotation attributes.
- static get_schema() dict[str, Any][source]
The config this kind accepts; each kind extends it.
- get_score_value_from_record(record: tuple[Any, ...], score_id: str) str | int | float | bool | None[source]
Read one configured score off a record of this score’s table.
- get_score_values_from_record(record: tuple[Any, ...], score_defs: list[GenomicScoreDef]) list[str | int | float | bool | None][source]
Read several scores off one record, for ALREADY-resolved defs.
The bulk counterpart of
get_score_value_from_record(): a caller resolves score names to definitions once per fetch and passes them per record, so the name->definition lookup stays out of the per-record loop.
- has_chromosome(chrom: str) bool[source]
Answer whether this score’s table carries
chrom.The yes/no half of
get_all_chromosomes(), and what every contig screen in the read path asks – the annotators’ pre-read screens, the shared region-read refusal, the position kind’s absent-contig branch and both allele reads’ refusals. All of them spelledchrom not in self.get_all_chromosomes(), a walk of the ordered list whose cost grew with the resource’s contig count and with the contig’s index in it, and grew WORST for a contig the resource does not carry, since that walks the whole list before it can say no (gain#1304).Raises
ValueErroron a score that is not open, exactly asget_all_chromosomes()does and for the same reason: those screens got that refusal for free from the accessor they used, and a predicate that answeredFalseinstead would turn “this score was never opened” into “this score does not carry that contig” at every one of them.The ordered list keeps its meaning, its order and its aliasing for the callers that genuinely need the collection – a statistics scan splitting a genome into region tasks, the resource implementation’s contig report. This is for the ones that only ever asked a question.
- is_open() bool[source]
Whether
open()has run andclose()has not since.
- open() Self[source]
Open genomic score resource and returns it.
Validate and route BEFORE opening, and so before publishing. Every input to both steps is known at construction – the table’s class, its
yields_recordsClassVar, and the score definitions – so neither needs the open handle, and two things fall out of that order:a refusal costs no handle. Routing after
table.open()would leave a caller that is not using thewithform holding an opened pysam handle it cannot reach:table_loadedwould still be False, soclose()would not have been reached. Raising first means there is nothing to leak. The bigWig config validation sits here for exactly that reason.table_loaded = Trueis what makes this score look open to everyone else: from that write on, another caller’s open() takes the is_open() early return above and reads_extract_valuestraight away. Routed last, that caller could catch the score published-but-unrouted, and since the routing has no default at all, that caller reads an AttributeError. Scores are shared across threads (the process-wide in-memory fragment-score cache; gain-web-api’s thread pool), so the window is reachable; this ordering keeps the ROUTING out of it. Pinned by test_the_score_is_routed_before_it_reports_itself_open.
It does not make open() as a whole safe to race, and does not claim to:
resolve_score_indicesstill runs after the score has published itself open, so a caller that catches that window reads a score def with noscore_indexyet. That window is older than this ordering and untouched by it – open() is not synchronised, and making it so is a separate change.
- abstractmethod classmethod record_weight(left: int, right: int) int[source]
How many times one record’s value counts when aggregating.
The rule is a property of the resource TYPE: a position-score record counts once per base pair of the queried region it covers, an allele line counts once, a fragment counts once however long it is. One record, one count, is the answer for everything except a position score.
The kind’s single statement of that rule, and every reader that weighs a RECORD goes through it:
aggregate_region()folds with it, the per-record statistics scan calls it, and the bulk scan broadcasts it over a whole batch throughrecord_weights(). One statement, so a kind cannot weigh its records one way in one of those and another way in the next. Pinned by test_the_weight_rule_is_stated_once_per_kind.A position score’s logical plane is deliberately NOT among them. It tiles POSITIONS rather than folding records –
get_scores_in_region_aggweighs each run by its length – and since gain#1131 that is the path annotation takes. The two agree for this kind, a clipped record’s width being exactly the length of the run it covers, but they agree by arithmetic rather than by reading one statement, so a change here is not automatically a change there.An implementation must be numpy-elementwise – an arithmetic expression over
leftandright, or a constant. It is declared over scalars because that is what its per-record callers hand it, butrecord_weights()answers a whole batch by handing it the position COLUMNS instead, and only an elementwise body gives the same answers that way.Most ways of breaking that break loudly – a body branching on
leftraisesValueErroron an array’s ambiguous truth, one callingint()aTypeError. The contract is written down for the ones that do not: a body REDUCING its arguments (int(np.mean(right - left + 1))) hands back a plain number, which is then broadcast as though it were every record’s weight, and the two scan paths disagree with nothing raised. That is what test_the_weight_rule_is_stated_once_per_kind’s broadcast-agreement assertion is there to catch.Deriving a weight from the span unconditionally is what this hook exists to prevent: it would give a fragment its length as a weight and disagree with the fragment score annotator for every fragment longer than one base pair.
- classmethod record_weights(begins: ndarray, ends: ndarray) ndarray[source]
record_weight()over a whole batch’s position columns.The bulk statistics scan has no record to hand the scalar hook, so it weighs a batch here instead. This does not restate the rule – it broadcasts the ONE statement of it, which is why the scan may not read the weight anywhere else.
The widening is the elementwise contract being spent: the hook is declared over scalars and its bodies are arithmetic, so the same expression answers a column. A kind whose weight is a CONSTANT answers with that constant however it was called, so a 0-d result is filled out to the batch’s shape rather than treated as an error.
- region_values_from_records(records: Iterator[tuple[Any, ...]], chrom: str, pos_begin: int | None = None, pos_end: int | None = None, scores: Sequence[str] | None = None) Generator[tuple[int, int, list[str | int | float | bool | None]], None, None][source]
Extract this kind’s
(begin, end, values)from raw records.The region read expressed as a function OF a record stream, which is what lets the two consumers of a region differ by what they COMPOSE rather than by a flag:
fetch_region_segments_scores()is this applied tofetch_records(), and the statistics scan is this applied tovalidate_records(score, fetch_records(...)). Neither can quietly acquire the other’s behaviour, and no argument travels down to say which of the two is reading (ADR 0008).chrom,pos_beginandpos_endname the region the records were asked for. Nothing is fetched here, and nothing is reshaped to the window either – what a partial overlap means belongs to the caller (ADR 0008); a consumer answering a question about the window clips withclip_span(). The positions are what the guards below are about.The guards run when this is CALLED rather than on the first
next()– the patternfetch_records()documents – which is why the streaming half lives in_score_segments. They stay here rather than moving down intofetch_recordsbecause that is not where the request is: half of what they resolve is the score ids, whichfetch_recordsis not even given, and the contig half would be a THIRD screen of the same contig in one read – after this one and before the backend’s own, which refuses an unknown contig fromget_records_in_regionregardless. This used to be argued on cost, from a time when each screen walked the ordered contig list; since gain#1304 the screen is a set lookup and the argument is only about where a request is resolved, which is here.What a kind yields is
_score_segments(), and not this method: the resolution above is the same for every kind, the reading below is not, and since gain#1282 they are split so that a kind states only the half that is its own. Override THAT to say what a record means here; overriding this one would take the resolution with it, and would be skipped by a read that enters below it holding definitions it has already resolved.
- resource_files() set[str][source]
The resource’s files the score reads: the data file, plus the index on a backend that reads one.
Answered by the table, which knows how it opens, on a closed score – the file set has to be known before any of the files is.
- supports_region_value_arrays(scores: list[str]) bool[source]
Whether
fetch_region_value_arrays()will serve these scores.Answers the two things a caller can be wrong about: the backend serves the bulk column-array read, AND every named score is one this facade can parse. A predicate that answered only the first would say True for a call that then refuses – not a capability query but a trap.
It is not a promise the call cannot fail for some OTHER reason. A score whose configured column index does not exist in its backend’s payload still raises (deliberately – see
BigWigTable), and so does a closed score or an unknown contig. This answers “is this score the kind this method serves”, not “is every argument valid”.The value-type half is not a consumer’s condition leaking in: the facade parses, so it serves the value types
GenomicScoreDef.parse_array()defines a column parse for (BULK_PARSEABLE_VALUE_TYPES) and no others. What a consumer additionally needs stays with the consumer: the statistics scan also requires a bounded region, and each of its two entry points asks for its own histogram or min/max pairing (seegenomic_scores_impl.scan.can_bulk_histogramandcan_bulk_min_max). The scan does not re-test the resource KIND; ADR 0001 records why. What it does NOT require is a particular record shape: the accumulator reads the kind’s ownrecord_weightand the scan’s door reads the rule registered for the kind ingain.genomic_resources.statistics.record_validation, so a position, allele and fragment score are all served.Answerable on an UNOPENED score: the table and the score definitions are both built in
__init__, so nothing here touches the file.
gain.genomic_resources.genomic_scores.builders module
The factories that build a score from a resource.
One pair per kind (build_<kind>_score_from_resource and its
_from_resource_id sibling) plus the dispatching pair that reads the
resource’s type and picks the kind for you. The _from_resource_id half
falls back to the default GRR when handed no repository.
- gain.genomic_resources.genomic_scores.builders.build_allele_score_from_resource(resource: GenomicResource) AlleleScore[source]
Build an allele score from an allele_score resource.
Defaults to alleles mode unless the resource configures allele_score_mode explicitly.
The deprecated np_score type was accepted here until 2026.8.5, and defaulted to substitutions mode instead (gain#920). A resource still declaring it is refused with a message naming both the replacement type and the mode key needed to keep the old reading.
- gain.genomic_resources.genomic_scores.builders.build_allele_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) AlleleScore[source]
Build an allele score from an allele_score resource id.
- gain.genomic_resources.genomic_scores.builders.build_fragment_score_from_resource(resource: GenomicResource) FragmentScore[source]
Build a fragment score from a fragment-score resource.
A fresh score every call, as the position and allele factories give: the caller owns what it gets back and may close it without affecting anyone else.
- gain.genomic_resources.genomic_scores.builders.build_fragment_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) FragmentScore[source]
Build a fragment score from a cnv_collection resource id.
- gain.genomic_resources.genomic_scores.builders.build_position_score_from_resource(resource: GenomicResource) PositionScore[source]
Build a position score from a position_score resource.
- gain.genomic_resources.genomic_scores.builders.build_position_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) PositionScore[source]
Build a position score from a position_score resource id.
- gain.genomic_resources.genomic_scores.builders.build_score_from_resource(resource: GenomicResource) GenomicScore[source]
Build a genomic score resource and return the coresponding score.
Dispatches on the resource type to the corresponding typed factory. Use the typed factories directly when the resource type is known statically; this one exists for callers handed a resource of unknown type.
Every kind yields a fresh instance per call, so the caller owns the score it gets back and closing it affects nothing else.
- gain.genomic_resources.genomic_scores.builders.build_score_from_resource_id(resource_id: str, grr: GenomicResourceRepo | None = None) GenomicScore[source]
gain.genomic_resources.genomic_scores.chrom_lengths module
Chromosome lengths of a genomic score, and where each one came from.
A score’s table can usually say how long a contig is, but the backends
answer with different confidence – a bigWig header is exact, the tabix
probe brackets an upper bound, the in-memory backend only knows how far its
rows reach – and the callers that split a contig into regions also need to
know WHY there is no length when there is none (gain#509). So the record
here keeps three things apart: the number, its provenance, and the
ContigExtent reason
when the number is absent.
The ladder the epic (gain#1412) settles is genome label → bigWig header →
tabix estimate, applied per contig of the score. The resolver here answers
every rung live: the genome rung from the ReferenceGenome the caller
hands in, the rest through the table. It holds no repository, so
resolving the score’s reference_genome label into that genome is the
caller’s job.
- class gain.genomic_resources.genomic_scores.chrom_lengths.ChromLength(length: int | None, source: ChromLengthSource | None, extent: ContigExtent | None)[source]
Bases:
objectOne contig’s length, its source, or the reason there is none.
Exactly one of the two shapes:
lengthandsourceset withextentNone, or bothNonewithextentsaying why –EMPTYwhen the table proved the contig holds no records,UNDETERMINEDwhen the probe could not answer for a contig that may well hold some.- extent: ContigExtent | None
- length: int | None
- source: ChromLengthSource | None
- gain.genomic_resources.genomic_scores.chrom_lengths.derive_chrom_length(score: GenomicScore, chrom: str, ref_genome: ReferenceGenome | None = None) ChromLength[source]
Resolve one contig of
scorethrough the ladder.ref_genomeis the top rung: a contig it lists is answered with its true length, taggedREFERENCE_GENOME, and the table is never asked. A contig it does not list – or no genome at all – falls through to the table, whose source is whatever the backend declares its lengths to be. RaisesValueErrorwhen the score is not open or does not carrychrom– a bad question, as opposed to an absent answer – in the TABLE’s words, since it is the table that refuses; the genome rung sits behind the table’s own screen so that a contig only the genome knows is refused the same way.
- gain.genomic_resources.genomic_scores.chrom_lengths.derive_chrom_lengths(score: GenomicScore, ref_genome: ReferenceGenome | None = None) dict[str, ChromLength][source]
Resolve every contig of
score, in the table’s order.The universe is the score’s contigs –
get_all_chromosomes()– never the whole genome; a whole-reference denominator is a property of the genome and belongs to coverage (gain#1041). Keyed by contig so the consumer that splits regions (the statistics build) looks up by name; the dict keeps table order.Not memoised: on a tabix score every call re-runs the probe per contig. Raises
ValueErroron a score that is not open.
gain.genomic_resources.genomic_scores.fragment module
FragmentScore – one value per genomic interval.
The kind whose records span a region rather than a point, and the one that
still answers to a legacy resource-type spelling; recognising that spelling
announces it through
warn_deprecated_spelling().
- class gain.genomic_resources.genomic_scores.fragment.FragmentAggregate(count: int, values: tuple[str | int | float | bool | None, ...])[source]
Bases:
objectWhat one folding read saw, and what it reduced to.
Both halves come off ONE walk of the region, which is the reason they are answered together rather than by two reads a caller would have to trust to agree: nothing a caller can do makes
countdisagree with the fragmentsvalueswas folded from.valuesis parallel to the QUERIES asked, not keyed by score id. One score requested twice with two aggregators – a source exposed as both a min and a max – is two queries and therefore two values, which a mapping keyed by score id would silently collapse to one.countis the number of fragments the walk SAW: those overlapping the region, that the overlap fractions admitted, and thatscore_filterkept. An empty region and a filter that rejected every fragment are both0– the distinction gain#820 built for alleles is deliberately not drawn here, keeping ADR 0017’s reasoning that a region is spanned by fragments as a matter of course, so “none cover it” is a count of zero rather than an absence.- count: int
- values: tuple[str | int | float | bool | None, ...]
- class gain.genomic_resources.genomic_scores.fragment.FragmentScore(resource: GenomicResource)[source]
Bases:
GenomicScoreA genomic score over fragments – intervals carrying attributes.
Nothing here is copy-number specific; a CNV collection is one application of it. Accepts either resource type in
FRAGMENT_SCORE_TYPES, warning once per resource on the deprecated one.- DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'max', 'int': 'max', 'str': 'join(,)'}
- fetch_fragment_scores(chrom: str, start: int, stop: int, scores: list[str] | None = None, *, score_filter: ScoreFilter | None = None) Generator[tuple[int, int, tuple[str | int | float | bool | None, ...]], None, None][source]
Stream
(begin, end, values)for the fragments over a region.Private to the fragment plane.
fetch_region_segments_scores()through_tupled(), and not a read to reach for directly; it keeps its name because it had one, not because the name is an invitation. It diverges from the internals beside it (_score_segments,_region_read_defs) in spelling only.What it adds to the base read is the tuple, a locus that is required rather than defaulted, and a
listof score ids. What the fragment plane once had a private TWIN of that method for wasscore_filter, which the base method takes now (gain#1272).One entry per overlapping fragment, in table order, each reporting the fragment’s OWN extent – unclipped, even where it runs past the region asked for. What a partial overlap means depends on what the caller is computing, so ADR 0008 leaves it to them; a caller that wants the window intersected composes
clip_span().valuesis positional, parallel toscoresas requested (toget_all_scores()when that isNone), rather than a mapping: the caller already knows what it asked for and in what order. A value may beNonewhere the record carries no value for that score – unlike the per-position reads, that is the onlyNonehere, because a fragment score has no notion of an uncovered position.score_filter– fromGenomicScore.compile_filter()– drops the fragments it rejects, which are then simply not yielded. It reads the RECORD, so it may name any score the resource defines, including one outsidescores, and a rejected fragment costs no extraction.The REQUEST is checked when this is called; the READING is lazy. A closed score, a contig this resource does not have and an unknown score id are refused before the first
next()rather than on it, for the reason_region_read_defs()gives. A malformed RECORD is a different matter and is refused when the record is reached: a fragment whose end precedes its begin ends the iteration then, mid-stream.One live read at a time. A score serves a single region read at once – the table’s line iterator and line buffer are the table’s, not the generator’s – so starting a second read invalidates one that is still being consumed, and on a tabix-backed table the two then answer each other’s records with no error raised. Materialising is what makes a held answer safe to keep:
kept = list(score.fetch_fragment_scores(chrom, beg, end))
Abandoning a read mid-stream is safe and costs only a
TabixGenomicPositionTablebuffer prune, which gain#1120 moved into afinally– though that runs when the generator is released, so a caller holding a reference to aclose()-ed generator still holds the read open.
- get_fragment_score_at_position(chrom: str, pos: int, *, score: str | None = None, score_filter: ScoreFilter | None = None) Sequence[tuple[int, int, str | int | float | bool | None]][source]
Return
(begin, end, value)per fragment covering a position.The singular form of
get_fragment_scores_at_position(), which says why it materialises;scoreofNoneis honoured only when the resource declares exactly one.
- get_fragment_score_overlapping_region(chrom: str, start: int, end: int, *, score: str | None = None, score_filter: ScoreFilter | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None) Generator[tuple[int, int, str | int | float | bool | None], None, None][source]
Yield
(begin, end, value)per fragment overlapping a region.The singular form of
get_fragment_scores_overlapping_region(), which documents the overlap fractions and the one-live-read limit this inherits;scoreofNoneis honoured only when the resource declares exactly one.
- get_fragment_score_overlapping_region_agg(chrom: str, start: int, end: int, *, score: str | None = None, aggregator: str | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None, score_filter: ScoreFilter | None = None) FragmentAggregate[source]
Reduce the fragments overlapping a region for ONE score.
The singular form of
get_fragment_scores_overlapping_region_agg(), which documents the selection and the reduction;scoreofNoneis honoured only when the resource declares exactly one.Alone among this plane’s singular reads it does NOT unwrap: it answers the same
FragmentAggregate, whosevaluesis a one-element tuple. The others have a bare value to answer with; this one’s answer is a count and a reduction together, and the count is a property of the QUERY rather than of the score named – so there is nothing for a bare value to be.
- get_fragment_score_starting_in_region(chrom: str, start: int, end: int, *, score: str | None = None, score_filter: ScoreFilter | None = None) Generator[tuple[int, int, str | int | float | bool | None], None, None][source]
Yield
(begin, end, value)per fragment BEGINNING in a region.The singular form of
get_fragment_scores_starting_in_region(), which documents the partition it answers and the one-live-read limit this inherits;scoreofNoneis honoured only when the resource declares exactly one.
- get_fragment_scores_at_position(chrom: str, pos: int, *, scores: list[str] | None = None, score_filter: ScoreFilter | None = None) Sequence[tuple[int, int, tuple[str | int | float | bool | None, ...]]][source]
Return
(begin, end, values)per fragment covering a position.A one-position region read of
get_fragment_scores_overlapping_region(), which documents what an entry is; spans are unclipped here too, so a fragment answering a position is reported at its full extent.Materialised, for the caller’s convenience. A point query returns a handful of fragments, callers want all of them, and a materialised answer can be measured with
len(), iterated twice and kept across a later read. It is NOT the drain hazardget_scores_at_position()documents: that was gain#1120’s to fix, and abandoning a region generator has been safe since.The overlap fractions are deliberately absent. Over a one-base region
overlap / region_lengthis always 1, so the region fraction could only ever be vacuous, and the fragment fraction of a single base is a ratio no caller has been found to want.posis refused below 1, through the same_guard_region_span()the region reads use: a backend that reads0as “unbounded” would otherwise answer a caller error with the whole contig.
- get_fragment_scores_overlapping_region(chrom: str, start: int, end: int, *, scores: list[str] | None = None, score_filter: ScoreFilter | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None) Generator[tuple[int, int, tuple[str | int | float | bool | None, ...]], None, None][source]
Yield
(begin, end, values)per fragment overlapping a region.The plane’s workhorse. Entries are shaped as
fetch_fragment_scores()shapes them – one per overlapping fragment, in table order, at the fragment’s OWN unclipped extent, withvaluespositional and parallel toscores– andscore_filterbehaves as it documents there. What this adds is the two thresholds below.The two overlap fractions are
overlap_fractions_admit(), applied with this region as[start, end]and each fragment as the record; that function defines them. In this plane’s vocabularymin_region_overlap_fractionis “the fragment must cover at least this much of MY region” andmin_fragment_overlap_fractionis “at least this much of the FRAGMENT must fall in my region”. Both unset filters nothing – which is what this read did before the thresholds existed – and hands the stream through without consulting the predicate.They SELECT, they do not RESHAPE: a fragment that passes is still reported at its own unclipped span. That is this plane’s rule and it has no decision record of its own – ADR 0008 is about who validates, not about what a read may do to a span, so it is not the authority for it.
Everything after the locus is keyword-only, and that is not cosmetic:
fetch_fragment_scores()takes its score list positionally, so a caller migrating fromfetch_fragment_scores(chrom, start, stop, scores)would otherwise bind that list to whatever this signature happens to put fourth – no error, just a plausible-looking filtered result.The REQUEST is checked when this is called; the READING is lazy. A closed score, an unknown contig, an unknown score id, a region no genomic span can mean and an out-of-range fraction are all refused before the first
next().One live region read per score at a time. The table’s line iterator and line buffer belong to the table, not to the generator, so starting a second read invalidates one that is still being consumed – on a tabix-backed table the two then answer each other’s records with no error raised. A held generator may be closed across another query, never resumed across one. Materialise (
list(...)) whatever has to outlive the next read.
- get_fragment_scores_overlapping_region_agg(chrom: str, start: int, end: int, *, queries: Sequence[ScoreAggregationQuery] | None = None, min_region_overlap_fraction: float | None = None, min_fragment_overlap_fraction: float | None = None, score_filter: ScoreFilter | None = None) FragmentAggregate[source]
Reduce the fragments overlapping a region to one value per query.
The plane’s folding read: what
get_fragment_scores_overlapping_region()yields, already reduced, in ONE pass that also counts what it saw. The selection is that read’s exactly – the two overlap fractions andscore_filtermean what they mean there, and a fragment is weighed byrecord_weight(), which counts it once however long it is. Exactly, because it is the same code: both consume_segments_overlapping_region(), and only the public read tuples what comes out.Answers a
FragmentAggregate, which documents why the count and the values travel together and whatcountcounts.Deliberately NOT built on
aggregate_region(), despite reducing the same way: that surface takes noscore_filter, and itsCountAggregatorhas the wrong count semantics here – it skipsNonevalues, so it counts non-null VALUES rather than fragments, and answersNonerather than0for a region no fragment overlaps.THIS READ holds nothing per fragment: the stream is folded as it arrives and never materialised. Whether the CALL is constant in the number of fragments is then the AGGREGATORS’ business, and they divide three ways:
constant –
max,min,mean,count,bool;one entry per DISTINCT value –
mode,value_count, so bounded by how many values a resource has rather than by how many fragments a region holds;one entry per FRAGMENT –
list,median,concatenateandjoin.join(,)is the DEFAULT for astrscore, which makes this the ordinary case for a CNV collection rather than an exotic one.
Under the last group the fold still allocates per fragment. What this read removes is the SECOND copy the annotator used to build beside it, which is a halving there and a flattening everywhere else.
- get_fragment_scores_starting_in_region(chrom: str, start: int, end: int, *, scores: list[str] | None = None, score_filter: ScoreFilter | None = None) Generator[tuple[int, int, tuple[str | int | float | bool | None, ...]], None, None][source]
Yield
(begin, end, values)per fragment BEGINNING in a region.Exactly the fragments whose begin lies in
[start, end], so a set of adjacent windows answers each fragment from exactly ONE of them: no duplicates and no gaps. That is the property chunked and parallel work depends on, and it is the only predicate on this plane that guarantees it –get_fragment_scores_overlapping_region()answers a fragment from every window it reaches into. The rule isowns_record().The allele statistics scan makes the same ownership claim inline, as
_owns, but spells itclip_span(pos, pos, start, end): an allele row sits AT one position, so for it the record partition and the position one coincide. For a fragment they emphatically do not, which is why this read names the record partition rather than reusing that spelling. That scan is left as it is.There is no caller yet. It is kept for that meaning, so the partition has a name before something needs it.
Entries are shaped as
get_fragment_scores_overlapping_region()shapes them, spans unclipped, and the one-live-read limit it documents applies here too.The overlap fractions are deliberately absent: this read partitions, and a fraction filter would let a fragment fall out of every window, which is the property being partitioned FOR.
- static get_schema() dict[str, Any][source]
The
GenomicScoreschema plus a per-scoreaggregator.
- classmethod record_weight(left: int, right: int) int[source]
A fragment counts once however long it is.
The kind’s whole reason for weighing by record rather than by span: a fragment is a measured thing, not a run of per-base values, so its length says nothing about how many times its value counts.
A constant, which is elementwise: the base’s
record_weights()fills it out to a batch’s shape.
gain.genomic_resources.genomic_scores.position module
PositionScore – one value per genomic position.
The kind whose records carry no reference or alternative allele, so a position is the whole key. Adds the position-run reads and the binned and aggregated region queries built on them.
- class gain.genomic_resources.genomic_scores.position.PositionScore(resource: GenomicResource)[source]
Bases:
GenomicScorePosition-based genomic score resource.
A PositionScore provides scores associated with genomic positions, where each score value applies to a specific genomic coordinate or range. Unlike AlleleScore, PositionScore does not consider reference or alternative alleles - scores are purely position-based.
Typical use cases include: - Conservation scores (e.g., phastCons, phyloP) - Mappability scores - GC content - Recombination rates - Any metric that depends only on genomic position
The score data can be stored in various formats including tabix-indexed files, BigWig files, or in-memory tables.
Example
>>> from gain.genomic_resources.repository_factory import ( ... build_genomic_resource_repository ... ) >>> repo = build_genomic_resource_repository() >>> resource = repo.get_resource("phastCons100way") >>> score = build_score_from_resource(resource) >>> with score.open() as score: ... # Fetch scores at a specific position ... values = score.get_scores_at_position("chr1", 12345) ... # Fetch scores across a region ... region = score.fetch_region_segments_scores( ... "chr1", 10000, 20000) ... for pos_begin, pos_end, scores in region: ... print(f"{pos_begin}-{pos_end}: {scores}")
Aggregating those values over the region is the resource’s job since gain#1131:
get_scores_in_region_aggreduces a region to one value per query, andgain.annotation.position_score_annotatorasks for that rather than folding records of its own. What the kind contributes to the reduction isrecord_weight– how many queried bases a record covers, and so how many times its value counts.- Variables:
resource (gain.genomic_resources.repository.GenomicResource) – The underlying GenomicResource object
resource_id – Unique identifier for the resource
config (dict) – Configuration dictionary for the score
table – GenomicPositionTable for data access
score_definitions (dict[str, ScoreDefT]) – Dictionary mapping score IDs to their definitions
- Key Methods:
get_scores_at_position: Get score values at a specific position fetch_region_segments_scores: Iterate over score segments in a genomic region, each at its record’s own extent get_scores_in_region_agg: Reduce a genomic region to one value per aggregation query, weighing each record by the bases it covers
- DEFAULT_AGGREGATORS: ClassVar[dict[str, str | None]] = {'bool': None, 'float': 'mean', 'int': 'mean', 'str': 'list'}
- static get_schema() dict[str, Any][source]
The
GenomicScoreschema plus a per-scoreaggregator.
- get_score_at_position(chrom: str, pos: int, score: str | None = None) str | int | float | bool | None[source]
Return one score’s value at one position,
Noneif uncovered.The singular form of
get_scores_at_position();scoreofNoneis honoured only when the resource declares exactly one.
- get_score_in_bins(chrom: str, start: int, end: int, bin_size: int, score: str | None = None, aggregator: str | None = None, none_value_replacement: str | int | float | bool | None = None) Generator[tuple[int, int, str | int | float | bool | None], None, None][source]
Yield
(bin_start, bin_end, value)per bin of[start, end].The singular form of
get_scores_in_bins();scoreofNoneis honoured only when the resource declares exactly one.
- get_score_in_region(chrom: str, start: int, end: int, score: str | None = None) Generator[str | int | float | bool | None, None, None][source]
Yield one value per position of
[start, end]for one score.The singular form of
get_scores_in_region();scoreofNoneis honoured only when the resource declares exactly one.
- get_score_in_region_agg(chrom: str, start: int, end: int, score: str | None = None, aggregator: str | None = None, none_value_replacement: str | int | float | bool | None = None) str | int | float | bool | None[source]
Reduce
[start, end]to one value for one score.The singular form of
get_scores_in_region_agg();scoreofNoneis honoured only when the resource declares exactly one.
- get_scores_at_position(chrom: str, pos: int, scores: Sequence[str] | None = None) tuple[str | int | float | bool | None, ...][source]
Return the score values at one position,
Nonewhere uncovered.Read straight off
_position_runs()rather than through the region read’s per-position expansion: a one-position region is exactly ONE run – no run is ever yielded empty, and the run lengths sum to the region width – so there is nothing to expand and nothing to index a position out of.The run is DRAINED rather than abandoned; that is simpler to write, not load-bearing, since gain#1120 moved the tabix buffer prune into a
finally(fragment.pycross-references this paragraph, andtest_a_walk_of_point_reads_leaves_the_tabix_ buffer_prunedpins the drained half).Going straight to the runs rather than delegating to
get_scores_in_region()is worth ~4% on this read, which the position annotator pays once per substitution.
- get_scores_in_bins(chrom: str, start: int, end: int, bin_size: int, queries: Sequence[PositionScoreAggregationQuery]) Generator[tuple[int, int, tuple[str | int | float | bool | None, ...]], None, None][source]
Yield one aggregated tuple per grid bin of
[start, end].Bins follow the GLOBAL grid anchored at position 1 (
calc_bin_index/calc_bin_begin/calc_bin_end), so adjacent queries tile and results are comparable across calls. Edge bins are clipped to the query, so the yielded bounds name exactly what was aggregated. Every bin in range is emitted, including bins no record touches; a segment straddling a bin boundary contributes its weight to each bin it touches, split at the boundary.A contig this score never mentions is uncovered rather than refused (gain#1211): every bin of it is emitted, with the bounds a covered contig would yield. That is this read and
get_scores_in_region_agg()only; a read that materialises positions still refuses an unknown contig.
- get_scores_in_region(chrom: str, start: int, end: int, scores: Sequence[str] | None = None) Generator[tuple[str | int | float | bool | None, ...], None, None][source]
Yield one tuple of score values per position of
[start, end].Exactly
end - start + 1tuples, in position order,Noneat every position no record covers.scoresofNoneasks for every score this resource defines, in definition order.
- get_scores_in_region_agg(chrom: str, start: int, end: int, queries: Sequence[PositionScoreAggregationQuery]) tuple[str | int | float | bool | None, ...][source]
Reduce
[start, end]to one value per query, over positions.Defined on the per-position expansion – an uncovered position is a
None, and with anone_value_replacementset it counts – but computed by walking segments, so cost stays proportional to record count. Where two records cover one position the first answers, so accumulated weight never exceeds the region width.A contig this score never mentions is uncovered rather than refused (gain#1211): it answers as a window on a contig the score has but does not cover. That is this read and the binned one only; a read that materialises positions still refuses an unknown contig.
- classmethod record_weight(left: int, right: int) int[source]
A record counts once per base pair it covers.
The only kind whose answer is not 1. That there is exactly one value per position – what a position score PROMISES – is not stated here but in the rules this kind is registered under in
gain.genomic_resources.statistics.record_validation, the only places that enforce it. This is a MEASURE.Elementwise, as the base requires: handed the position columns of a whole batch, the same expression answers that batch’s weights.
- resolve_aggregation_queries(queries: Sequence[PositionScoreAggregationQuery]) list[tuple[str, str, str | int | float | bool | None]][source]
Resolve each query to its (score_id, aggregator NAME, replacement).
The third element is the query’s
none_value_replacement.A query asks the same two questions a request list does – which score, and what reduces it – so they are asked where they are answered for every surface, in
aggregation(score_def_for(),resolve_aggregator_name()). Only the remedy of the missing-default refusal is this surface’s own, because a caller here names an aggregator on the query rather than in a pair.What a query asks BESIDES is the third: a
none_value_replacementmust be of a type the score can mean, followingvalidate_aggregator’s precedent. It is judged BETWEEN the other two – after the score is known, since its value type is what judges the replacement, and before an aggregator is looked for, so that a query wrong in both ways is answered about the value it named rather than the one it left out. That order is a decision and not an accident of composition; it is pinned bytest_a_query_invalid_several_ways_reports_the_first_ground.Public, and stopping at the NAME, because asking whether a query is answerable is a question a caller may have without wanting to read (gain#1131).
PositionScoreAnnotatorasks it once when the pipeline loads, so an attribute naming no aggregator for aboolscore is refused there rather than on the first region that reaches it. Building the accumulators is the READ’s business –_resolve_aggregation_queries()adds them, per call, which is what keeps a read thread-safe and an annotator stateless.It lives on this kind rather than on
GenomicScorebecause of its middle step:none_value_replacementis a field only aPositionScoreAggregationQuerycarries, a position score being the only kind with an uncovered position to speak for. Should a fragment or an allele score come to want a resolver of its own (gain#1124, gain#1132), thescore_def_for/resolve_aggregator_namepair is the part that generalises – both already live inaggregation, kind-neutral, for that reason – and the replacement validation is the part that does not.
gain.genomic_resources.genomic_scores.records module
Batch array types and the region/record algebra over them.
The half of the score layer that knows nothing about score resources: the
shape of a read batch (RecordArrays, AlleleRecordArrays)
and the five functions that decide which part of a record a region gets,
or whether it gets it at all.
Two of those partition different things and are deliberately neighbours –
clip_span() partitions POSITIONS, owns_record() partitions
RECORDS by where they BEGIN – so that the two halves of the algebra have one
home and a caller picks the one it means. overlap_fractions_admit()
sits beside them without partitioning anything: it SELECTS records by how
much of the region, or of themselves, the two share, and a record it rejects
is answered by no region at all.
Nothing here imports a score class. That is a property of this module, not
yet a saving for its callers: the scan and the statistics layer still reach
these names through the package facade, which imports every submodule, so
GenomicScore is loaded either way. What it buys is that
those callers CAN be pointed at genomic_scores.records directly, one at
a time, without anything else moving – migrating them was out of scope for
the gain#902 split that created this module.
- class gain.genomic_resources.genomic_scores.records.AlleleRecordArrays(pos_begin: np.ndarray, pos_end: np.ndarray, values: dict[str, np.ndarray], reference: np.ndarray, alternative: np.ndarray)[source]
Bases:
NamedTupleOne batch as
AlleleScore.fetch_region_allele_arrays()makes it.RecordArrayswidened by the two key columns an allele row has and a position row does not. The first three fields are that tuple exactly, in the same order, sobatch[:3]is aRecordArrays.That slice is required, not decorative: every consumer of the shared read unpacks three names (the scan’s array door
validate_record_arrays()and its coverage accumulator among them), and handing one of them a batch of five raisestoo many values to unpack. A caller feeding this read into machinery written for the shared one passesbatch[:3], and mypy says so too – this type is not aRecordArrays.referenceandalternativeare the cells as stored – see the fetch method for why they are the one part of a batch that is not parsed.- alternative: ndarray
Alias for field number 4
- pos_begin: ndarray
Alias for field number 0
- pos_end: ndarray
Alias for field number 1
- reference: ndarray
Alias for field number 3
- values: dict[str, ndarray]
Alias for field number 2
- gain.genomic_resources.genomic_scores.records.RecordArrays
GenomicScore.fetch_region_value_arrays() <.base.GenomicScore.fetch_region_value_arrays> produces it: the RAW one-based begin and end columns, plus one parsed value array per requested score id. Named because the vectorized scan validators are transducers over a stream of these.
- Type:
One batch as
- Type:
meth
alias of
tuple[ndarray,ndarray,dict[str,ndarray]]
- gain.genomic_resources.genomic_scores.records.clip_span(rec_begin: int, rec_end: int, pos_begin: int | None, pos_end: int | None) tuple[int, int] | None[source]
Clip a record’s span to a queried window: skip, clip, or refuse.
Returns the part of
[rec_begin, rec_end]inside[pos_begin, pos_end], where aNonebound means unbounded on that side, orNonefor a record with no part inside the region: one ending before it (the skip) or one starting past it (which naive clipping would turn into an inverted span, whose width as a weight is negative).
- gain.genomic_resources.genomic_scores.records.clip_to_region(segments: Iterator[tuple[int, int, T]], pos_begin: int | None, pos_end: int | None) Generator[tuple[int, int, T], None, None][source]
Clip a segment stream to a region, dropping what falls outside.
- gain.genomic_resources.genomic_scores.records.overlap_fractions_admit(rec_begin: int, rec_end: int, start: int, end: int, min_region_fraction: float | None, min_record_fraction: float | None) bool[source]
Whether a record overlaps a region by enough of either side.
With overlap the length of the intersection,
min_region_fractionisoverlap / region_length– “the record covers at least this much of MY region” – andmin_record_fractionisoverlap / record_length– “at least this much of the RECORD falls in my region”. The two answer different questions: a 10 bp record inside a 1 Mb region scores ~0.00001 on the first and 1.0 on the second.Every threshold supplied must hold, and each is compared with
>=, so1.0means full containment of the side it is about. BothNoneadmits everything, which is what a region read does with no thresholds at all.0.0is not the same request asNone, though both admit every record a region query actually answers:0 / length >= 0.0holds, so0.0also admits a record with NO overlap at all. A region query answering with such a row is a backend that over-returns; one cause of it – a tabix table whose index andpos_endname different columns – ADR 0008 refuses atopen(), and no threshold here is a check for the rest.A SELECTION predicate, not a reshaping one: it says whether the record is answered, never what span is reported for it.
rec_endis assumed not to precederec_begin, so that the record length it divides by is at least 1. This does not check it: the record reads refuse an inverted span at_score_segments(), before any consumer sees the record. Called with one directly, it divides by zero or worse.
- gain.genomic_resources.genomic_scores.records.owned_records_mask(pos_begin: ndarray, start: int | None, end: int | None) ndarray[source]
owns_record()over a whole batch’s begin column.
- gain.genomic_resources.genomic_scores.records.owns_record(begin: int, start: int | None, end: int | None) bool[source]
Whether a region owns a record, by where that record BEGINS.
The scan’s partition of RECORDS, and the one statement of it (gain#816). Ownership is total, unique and reachable: the regions tile a contig contiguously from position 1, so every record’s begin falls in exactly one of them, and the owning region’s query always returns it –
begininside[start, end]is by itself an overlap. An unbounded side owns everything on that side, which is the same rule with one region.Contrast
clip_span(), which partitions POSITIONS. A statistic that sums over records wants this; one that unions positions wants that. Both live here so the two halves of the algebra have one home.
gain.genomic_resources.genomic_scores.value_extraction module
How a record’s cell becomes a value: the two decisions taken at open.
The seam between a score and its table’s payload. Both decisions are taken
once per open, from the table’s TYPE and the score definitions, and neither
needs a GenomicScore:
select_value_extractor()picks the per-record read, beforetable.open();resolve_score_indices()addresses each definition to a payload column, after it.
That order is load-bearing – each function’s docstring says what forces its
half – and it belongs to the caller, open().
The extractors themselves are not here: they live with the backends that
know what a payload IS – vcf_scores,
bigwig_scores, and
score_def for the column read, which also
owns the ValueExtractor alias. This module only chooses between
them, which is why it sits ABOVE all three rather than inside one of them:
hosting the choice in score_def, where gain#1044 put the scoredef
lifecycle, would have it import vcf_scores and bigwig_scores, both
of which import score_def – the same cycle that kept
GenomicScore._build_scoredefs on the class. resolve_score_indices
alone could have gone there; it is here so that the seam reads as one
module rather than two homes.
resolve_score_indices() mutates the definitions in place and
returns nothing; its docstring says who reads what it wrote.
- gain.genomic_resources.genomic_scores.value_extraction.resolve_score_indices(score_definitions: dict[str, GenomicScoreDef], *, is_vcf: bool, is_bigwig: bool, table: GenomicPositionTable, resource_id: str) None[source]
Resolve each score’s configured address to a payload column.
Runs after
table.open(), because the by-NAME case is the one thing here that has to consult the table’s header.Writes
score_indexonto the definitions it is handed, and returns nothing: “the defs are finished in place at open” is the contractbasestates. Two paths read what it wrote –extract_column_value(), on every record of the tabular per-record read, and so the very extractor this module’s other half binds; andfetch_region_value_arrays, which the statistics scan reaches it through. Handing back a mapping would only mean every caller writes it back.A definition it cannot resolve – no address, two addresses, a name over a table with no header, a name the header does not have, a VCF definition with no INFO key – is refused through
score_configuration_error(), so the refusal names the resource and the score in the shape every other definition refusal shares and falls insidecli_errors.RESOURCE_ERRORSby type. Every check is an explicit test, never anassert(python -Ostrips those) and never a lookup left to raise for itself: a resource config is data, and bad data is reported. For a tabular table,validate_scoredefsholds the CONFIG to the address rules before this runs, but for a headerless table it checks only that no name is stated – so a definition with no address at all reaches this from a real config, while the other refusals are reached throughopen()only by a definition edited after its config passed.
- gain.genomic_resources.genomic_scores.value_extraction.select_value_extractor(*, score_definitions: dict[str, GenomicScoreDef], table: GenomicPositionTable, is_vcf: bool, is_bigwig: bool) ValueExtractor[source]
Pick the per-record value read for this table’s payload.
ONE decision, per table, taken at open rather than per line. What it turns on is what a record’s PAYLOAD is, which is whatever the backend that built it says it is:
a VCF record’s payload carries the variant and the pysam INFO proxies, and a VCF score is an INFO field addressed by name –
extract_vcf_value();a bigWig record’s payload IS the interval’s value, so the read is an identity (
extract_bigwig_value()) – or, for the rare resource that configures NA sentinels, an identity plus one membership test (extract_bigwig_value_na()). Which of the two is settled here, from the score definitions, and never per record: the sentinel set is fixed for the life of the open score. A bigWig declares exactly one score (validate_bigwig_scoredefsrefuses more), so there is a single answer to give;anystates that without depending on it;any other record-yielding table’s payload is a raw row, read by integer column –
extract_column_value().
The table’s
yields_recordsclaim is simply believed: that every backend’s claim matches what it really yields is pinned statically, over all four of them, by test_backend_record_contract.py, so the fetch path pays nothing for it.A table that yields no records is a programming error, not a data error: there is no fallback reader, so a backend leaving the flag False has nothing that can read it and we refuse rather than guess. (Nothing in the tree reaches it: it guards a backend added later without its migration.)
Module contents
Genomic score resources: the base class, its three kinds, and the algebra.
This package was one 3128-line module until gain#902 split it along the seams the code already had:
records– the batch array types and the region/record algebra over them; imports no score class, so the scan and the statistics layer can use it without pullingGenomicScorein behind itbase–GenomicScore, everything the kinds shareposition,allele,fragment– one module per kindbuilders– the eight factories, and the dispatch between kinds
and decomposing the class itself (gain#1027) has since added two more:
aggregation(gain#1074) – the machineryaggregate_region()orchestrates; knows no score class, and is handed the per-kind weight rule rather than reading itvalue_extraction(gain#1114) – the two decisionsopen()takes about how a record’s cell becomes a value: which extractor reads the payload, and which payload column each score def is addressed tochrom_lengths(gain#1413) – the chromosome-length resolver a caller with a genome hands it to: a length per contig with its source, or theContigExtentreason there is none
This module is a permanent facade, not a deprecation shim. It
re-exports every name the pre-split module DEFINED, so each
from gain.genomic_resources.genomic_scores import <name> written before
the split keeps working, verbatim and indefinitely. Deep imports from the
submodules are allowed and equivalent, but no caller is expected to migrate
to them: gpf imports through here, and so do out-of-tree callers this
repository cannot see or fix (grr_bench, demo repositories). The
promise is pinned by test_genomic_scores_facade.py, which spells the
surface out rather than deriving it from __all__.
What it deliberately does not carry is the ~70 names the pre-split module
merely IMPORTED and so leaked as incidental re-exports – np, copy,
Record, GenomicScoreDef, ScoreValue and the like. Reaching
through this module for one of those worked by accident, never by
intention, and an AST scan of gain, gpf and grr_bench (2026-08-31)
found no caller that did. Import them from the module that defines them.
What has since moved OUT is the pair of scan validation rules.
validate_records and validate_record_arrays were
@abstractmethod on GenomicScore with one body per kind
here; they are now singledispatch functions in
gain.genomic_resources.statistics.record_validation, registered per
kind, because the statistics scan is their only caller and a read class
should not carry a rule only its consumer applies (gain#1269, ADR 0027).
They were never part of the pre-split module’s re-exported surface – they
were instance methods – so the facade promise above is untouched.
What did NOT move here is the resource implementation –
genomic_scores_impl, which gain#1007 split into scan and classes and
gain#1210 then laid out as this package is, one module per kind – nor
the decomposition of the GenomicScore class itself, which
is gain#1027 and is deliberately sequenced after this split.