gain.genomic_resources.genomic_position_table package

Submodules

gain.genomic_resources.genomic_position_table.index_columns module

The columns a tabix index was built from, read off the index itself.

pysam does not expose them – a pysam.TabixFile reports the index’s filename and nothing about its contents – so the header is decoded here, directly, from the first bytes of the (gzip-compressed) index file.

The layouts are the ones in the SAM/BCF specification. A .tbi opens with a fixed 36-byte header, of which the first 32 bytes are magic, reference count, format, and the four coordinate fields. A .csi opens with magic, min_shift, depth and an auxiliary-data length; when the index was written by tabix that auxiliary data begins with the same format and coordinate fields, which is what lets one decoder answer for both flavours.

class gain.genomic_resources.genomic_position_table.index_columns.IndexColumns(chrom: int, pos_begin: int, pos_end: int, end_is_implied: bool)

Bases: object

The coordinate columns of a tabix index, as table column keys.

Zero-based, unlike the index’s own fields, so that they compare directly against the column keys a table resolves; the index stores them one-based with 0 meaning “absent”.

pos_end is never None: an index that records no end column (col_end == 0) treats every record as ending where it begins, so its end column is its begin column. end_is_implied says that is how pos_end came to hold that value, because a reader told only the number would not be able to tell it from an index built with the two columns deliberately pointed at the same place.

chrom: int

end_is_implied: bool

pos_begin: int

pos_end: int

gain.genomic_resources.genomic_position_table.index_columns.parse_index_columns(header: bytes) → IndexColumns | None

Decode the coordinate columns out of an index’s leading bytes.

Returns None when the bytes carry no column configuration to decode: an unrecognised magic, a header cut short, or a .csi written by something other than tabix (its auxiliary data is then absent or too short to hold the tabix fields). The caller decides what to do about it – which must not be to pass the resource silently.

gain.genomic_resources.genomic_position_table.line module

class gain.genomic_resources.genomic_position_table.line.LineBuffer

Bases: object

Buffer of records read from a Tabix genome position table.

Holds records – the six-slot tuples the tabular parser builds – and reads them by slot constant (record[CHROM], record[POS_BEGIN], record[POS_END]), never by attribute. The slots this buffer indexes on are immutable – a record’s tuple cells cannot be rebound – so a buffered record can be handed out and retained here at the same time without any risk that a later read moves it out from under the positional logic below (which the Line adapter it replaces did allow: the zero-based/chrom-mapping transforms rewrote the object in place).

That promise covers the slots, not the payload: the buffer holds the same record object the caller got, and a tabix payload is a pysam.TupleProxy, which defines __setitem__ – a caller that writes record[PAYLOAD][i] = ... mutates the row this buffer is holding. The payload is shared by reference on purpose (that is what keeps it lazy); see record.py. Nothing here reads it, so the buffer’s own behaviour is unaffected either way.

The semantics are those of the adapter-era buffer: it clears on a chromosome change, clears when it observes a non-monotonic ordering (region()), evicts records it can no longer be asked about (prune()), and locates a position by binary search with a linear scan over the overlapping intervals around the hit.

The buffer is ordered by ``pos_begin`` – and by nothing else. That is the file’s order, and it is the only ordering this class may assume. Every method here used to lean on a second, unstated assumption – that pos_end is non-decreasing too – which holds for point records and for strictly-disjoint intervals and fails the moment two intervals overlap: a record can then contain the ones that follow it. Reading a warm buffer through that assumption both hid records that overlap the query and surfaced records that do not (gain#250). So:

• the right edge of region() is the maximum pos_end in the buffer (_max_end), not the last record’s – the last record is merely the one that begins last;

• region() judges the ordering by pos_begin, since a pos_end that runs backwards is ordinary nested data rather than corruption;

• find_index() bounds its scan by the widest interval buffered (_max_width) instead of stopping at the first record that fails to reach the position.

Neither maximum may ever under-estimate – that would drop records. Over-estimating is harmless in both, but by different routes, which is why they are worth keeping apart: an over-wide _max_width only widens find_index()’s scan, and fetch() filters it exactly; an over-high _max_end instead lets contains() admit a position the buffer cannot answer, and find_index() then finds no record to return – whereupon the read falls through to the file, which is a wasted look rather than a wrong record. _max_end stays exact under every eviction (see prune()), and both are rebuilt exactly whenever prune() walks the whole deque – which is what lets _max_width shrink back after a wide record dies rather than widening every subsequent search for the rest of the buffer’s life.

The maxima and _compact_size – the deque’s length as of that last walk – are the only state beyond the deque itself, and all three are reset with it.

The VCF backend feeds this buffer too, with records of its own – whose PAYLOAD is a (variant record, allele index) pair rather than a raw row. Nothing here reads the payload, so the buffer needs to know nothing about which backend built the record it is holding: it windows every record by the three slots above, and those mean the same thing in all of them.

COMPACT_FLOOR: int = 32

COMPACT_GROWTH: float = 1.5

append(record: tuple[Any, ...]) → None

Buffer a record read from the file, maintaining the maxima.

A record from another contig empties the buffer first: nothing that precedes it can answer a query on the new contig.

clear() → None

contains(chrom: str, pos: int) → bool

fetch(chrom: str, pos_begin: int, pos_end: int) → Generator[tuple[Any, ...], None, None]

Return a generator of records matching the region.

pos_end is never None here: the buffer is only consulted when the caller asked for a bounded region. get_records_in_region turns buffering off when pos_end is None, so an unbounded query never reaches the buffer at all.

find_index(chrom: str, pos: int) → int

Find the first index in the buffer relevant to pos.

Returns the leftmost record that overlaps pos or, when none does, the leftmost that begins at or after it – and -1 when the buffer does not span pos at all. fetch() scans forward from here and filters exactly, so the one thing this must never do is land to the right of a record that overlaps.

Which is what it used to do. The old back-scan walked left while the predecessor reached pos and stopped at the first one that did not – sound only if a record that fails to reach pos proves that everything before it fails too, i.e. only if pos_end is non-decreasing. With overlapping intervals a record containing pos can sit to the left of one that does not, behind that stop.

The bound that does hold: the buffer is sorted by pos_begin, and a record overlapping pos spans it, so it cannot begin before pos - _max_width. Binary-searching for that lower bound puts the scan at or left of every record that can overlap, whatever the ends do. The window is as tight as the buffered data’s widest interval; a stale _max_width only widens it, and fetch() filters regardless.

peek_first() → tuple[Any, ...]

peek_last() → tuple[Any, ...]

prune(chrom: str, pos: int) → None

Drop the records that can no longer match pos or later.

A record is dead once its pos_end falls below pos: every query from here on starts at pos or later, so nothing it could overlap will ever be asked for again.

_max_end survives either pass exactly, and not by luck. Every record dropped ends before pos while at least one survivor ends at or after it, so a dropped record can never have held the maximum unless there are no survivors at all – and that case empties the deque and resets the maxima with it. That exactness is what keeps contains() from admitting a position the buffer cannot answer. _max_width is not exact between compactions and is not required to be: it may only over-estimate, which merely widens find_index()’s search. A compaction rebuilds both exactly.

Pruning drops every record that fails that test, not just the leading run of them. Stopping at the first survivor – which is what this did – let a wide record pin the head and hold behind it every narrow record it spans, which fetch() then rescanned on every query; the buffer grew with the widest interval over a dense region rather than staying at the live set (gain#287). Dropping the dead records wherever they sit keeps the deque at the records that can actually match, and fetch()’s scan is bounded by the deque, so the two shrink together.

region() → tuple[str | None, int | None, int | None]

Return the region the buffered records span.

The right edge is the widest pos_end buffered, not the last record’s: the records are ordered by pos_begin, so the last one to begin need not be the last one to end (see the class docstring).

Ordering is judged by pos_begin for the same reason. A first record that ends after the last one is not evidence of anything – that is what a nested interval looks like – but a first record that begins after the last one contradicts the one order the buffer is built on, so the buffer discards itself rather than answer from a scrambled window.

gain.genomic_resources.genomic_position_table.record module

Record contract and pure tabular parser factory.

A genomic position table yields a record: a plain six-element tuple whose slot positions are named by the module-level integer constants below – chromosome, start position, end position, reference allele, alternative allele, and an opaque backend payload. The five core fields are decoded eagerly; the payload stays lazy (for a tabular backend it is the raw row, which decodes columns only when a caller asks for one).

The decoded slots come first and the payload is the last slot. That is part of the contract, not an accident of the current layout, and it is what lets the module say “the decoded half” as everything before the payload – which sort_key() does, and which RECORD_SLOTS derives its count from. So the payload’s index is the ONE place a record’s shape is stated: to add a decoded slot, insert it before the payload and renumber PAYLOAD up; the slot count and the ordering key then both follow, with nothing else to keep in step. Appending a decoded slot after the payload is not a legal record – it would fall outside the ordering key, and split that one statement of the record’s shape into two that can drift apart. (Pinned in test_record_parser.py.)

The tuple is immutable: six slots, none of which can be rebound. That promise stops at the payload, which is the backend’s raw row held by reference – it is deliberately neither copied nor frozen, because that is what keeps it lazy. A mutable raw row therefore stays mutable through the record’s payload slot. (Both halves are pinned in test_record_parser.py.)

A record’s hashability is its PAYLOAD’s – do not assume a record can go in a set or key a dict. A record is a plain tuple, so hash(record) walks the tuple, straight into the payload; whether that succeeds is the backend’s answer, not the contract’s, and today only ONE of the three record backends says yes. The in-memory backend’s payload is a tuple[str, ...] and hashes; the tabix backend’s is a pysam.TupleProxy and the VCF backend’s is a (pysam.VariantRecord, allele index) pair, and both of those pysam types define __eq__ without __hash__, so hashing such a record raises TypeError: unhashable type. The contract deliberately does not promise hashability: guaranteeing it would mean wrapping every payload in a hashable per-line object, which is the per-line cost records exist to remove. The five decoded slots always hash, so the portable key of a record is sort_key(record) – {sort_key(record): ...}, never {record: ...}. (Each backend’s answer is declared and pinned in test_backend_record_contract.py, so a new backend must state its own.)

Records are not orderable – always sort them through sort_key(). Because the payload sits inside the tuple, a plain sorted(records) compares payloads whenever two records tie on all five decoded slots, and the tabular payload (a pysam.TupleProxy) implements no comparison: it raises NotImplementedError: op 0 isn't implemented yet. This is data-dependent and so especially treacherous – records look sortable, and are, until two rows land on the same position, which real data does. sort_key() projects the decoded slots and stops at the payload; it is THE way to order records.

This module is deliberately pure: it imports no pysam, no file handles and no genomic resource, so build_tabular_parser() can be unit-tested against plain lists of strings.

gain.genomic_resources.genomic_position_table.record.build_tabular_parser(chrom_key: int, pos_begin_key: int, pos_end_key: int, ref_key: int | None, alt_key: int | None, rev_chrom_map: dict[str, str] | None, *, zero_based: bool) → Callable[[Sequence[str]], tuple[Any, ...] | None]

Build a pure row->record parser for a tabular backend.

The parser is a pure function of the resolved column keys, the reverse chromosome map (file contig -> reference contig) and the zero-based flag. It fuses record construction with one of four transform specialisations, selecting one once here rather than branching per line:

• identity – no chromosome mapping, one-based coordinates;

• zero-based – shift begin/end from a half-open zero-based interval;

• chromosome-mapping – remap the contig, dropping rows whose contig is absent from the map (the parser returns None);

• zero-based and chromosome-mapping – both of the above.

Reference and alternative are read from their columns when configured and are None otherwise. The returned interval is closed on both sides and one-based, exactly as today.

What the per-row body costs, precisely. Each specialisation’s body is fully inlined: no helper call and no intermediate tuple is built per row, only the record itself. The row still pays two is not None checks on ref_key/alt_key – they are loop-invariant, but folding them out would mean crossing the ref/alt presence (four combinations, since ref and alt are configured independently) with the four specialisations here, i.e. sixteen near-identical closures. Two pointer compares are not worth that, so the ref/alt reads stay inline and this docstring states the cost rather than claiming a branch-free body. The zero-based pos_begin == pos_end check is data-dependent and cannot be hoisted at all.

gain.genomic_resources.genomic_position_table.record.sort_key(record: tuple[Any, ...]) → tuple[Any, ...]

Return the ordering key of a record: its five decoded slots.

The one supported way to order records – sorted(records, key=sort_key). Never sort records as bare tuples: the payload rides in the last slot, so a bare sort compares payloads on every tie and a pysam.TupleProxy payload raises NotImplementedError when compared (see the module docstring).

The key is a slice, record[:PAYLOAD], and it spans every decoded slot and stops exactly at the payload because the payload is the last slot – the contract in the module docstring. That is why the slice is written in terms of PAYLOAD rather than as a hard-coded five: insert a decoded slot before the payload, renumbering PAYLOAD up, and it joins the ordering key with no edit here. No decoded slot is dropped from the ordering, and none of the opaque half enters it.

A None in the key is safe. REF and ALT are None when the table has no ref/alt column – but that is a property of the table: the parser fixes ref_key/alt_key once, so within one table every record carries None in that slot or every record carries a string, never a mix. Tuple ordering settles a None against a None by equality and moves on; it never asks None < None, which would raise. (Records are only ever sorted within one table – the in-memory backend sorts each contig’s own records.)

gain.genomic_resources.genomic_position_table.table module

class gain.genomic_resources.genomic_position_table.table.ChromLengthSource(*values)

Bases: Enum

Where a contig’s length was read from, and so how far to trust it.

Three of the members are what a backend’s find_chromosome_length() measures, and every backend names its own in chrom_length_source – a fact about the FORMAT, declared on the class the way yields_records and supports_value_arrays are, so no caller has to know which backends exist. The fourth, REFERENCE_GENOME, no table produces: it is the genome rung of the score layer’s ladder (gain#1412), and lives here beside the other three so that the one vocabulary answers “where did this length come from” for every rung.

Only REFERENCE_GENOME and BIGWIG are exact. Callers ask is_exact rather than enumerating members, so a new source needs no edits at the call sites.

BIGWIG = 'bigwig'

The bigWig header, which carries an exact size for every contig.

REFERENCE_GENOME = 'reference_genome'

the contig’s true length.

Type:

A reference genome’s index

TABIX_ESTIMATE = 'tabix_estimate'

an upper bound, guaranteed LARGER than the actual length, never the length itself.

Type:

The tabix index probe

TABLE_EXTENT = 'table_extent'

how far the rows reach, which is an extent of the data rather than a length of the contig.

Type:

The in-memory backend’s max(pos_end) + 1

property is_exact: bool

Whether a length from this source is the contig’s true length.

A caller that needs a true denominator – a coverage fraction – may only trust an exact source; a bound or an extent would put the fraction off by whatever the probe over-shot or the rows fell short.

class gain.genomic_resources.genomic_position_table.table.ContigExtent(*values)

Bases: Enum

Why a backend has no length to report for a contig.

The return of GenomicPositionTable.find_chromosome_length() when there is no number to give. The two members are NOT interchangeable, and that is the whole reason this type exists: a caller that splits a contig into regions treats them oppositely (gain#509).

Which member a backend can return is a property OF THE BACKEND, not of the contig – which is what the caller used to encode as an isinstance ladder over concrete table classes:

• a backend holding the whole file (in-memory) can PROVE a contig has no records, and never has to guess a length for one that does – so it returns EMPTY and never UNDETERMINED;

• a backend reading lengths out of a header (bigWig) always has an exact length for a contig it lists, and returns neither;

• a backend probing an index (tabix, VCF) only indexes contigs that HAVE records, so it never sees an empty one, and its probe can fail on a contig that is not – so it returns UNDETERMINED and never EMPTY.

Neither member is a failure. A caller that asked wrongly – a closed table, a contig the table does not list – gets ValueError instead, and that split is the contract: an exception means the question was bad, a member means the question was fine and the answer is not a number.

EMPTY = 1

The backend PROVED the contig holds no records.

There is nothing to read, so nothing to split and nothing to validate.

UNDETERMINED = 2

No length is available, and the contig may well hold records.

Distinct from EMPTY because those records still have to be read: a length is what SPLITTING a contig needs, not what READING one needs.

refusal(chrom: str, contigs: list[str]) → str

The message for a caller that has no use for this member.

The one home of the two wordings, so that GenomicPositionTable.get_chromosome_length() and any other caller refusing the same two facts say them the same way (gain#1413). Named apart rather than collapsed into one “no length” message because an operator reading a failed statistics build acts on them differently: an empty contig is usually a chrom_mapping naming something the file does not carry, an undetermined length is a probe that could not answer for a contig that may well hold records. Both name the contig asked about and the contigs the table does have.

class gain.genomic_resources.genomic_position_table.table.GenomicPositionTable(genomic_resource: GenomicResource, table_definition: dict)

Bases: ABC

Abstraction over genomic scores table.

ALT = 'alternative'

CHROM = 'chrom'

COLUMN_KEY_SPELLINGS: ClassVar[tuple[str, ...]] = ('index', 'column_index', 'name', 'column_name')

POS_BEGIN = 'pos_begin'

POS_END = 'pos_end'

REF = 'reference'

alt_key: int | None

buffered_record_count() → int

How many records this table is holding from PREVIOUS reads.

Not the table’s contents, and not the size of the region last read: the records retained between queries, which is the quantity a lazy consumer can make grow without bound by walking away from its reads.

Zero is the honest answer for a backend that carries nothing across queries, and that is most of them – an in-memory table holds every one of its records and buffers none of them, so it reports zero while holding the lot. Only TabixGenomicPositionTable, which serves what it can from a warm LineBuffer, has anything to report.

It exists to be asserted on: without it the contract test would have to reach into a backend’s internals and know which ones have a buffer, which is the isinstance(Tabix) that the capability declarations on this class exist to replace.

chrom_key: int

chrom_length_source: ClassVar[ChromLengthSource]

chrom_map: dict[str, str] | None

chrom_order: list[str] | None

close() → None

Close the file and release everything read out of it.

THE RELEASE POLICY, for every backend: after ``close()`` a table holds only what ``open()`` does not rebuild – its resource, its definition, and its configured parameters (the header under header_mode: list, where it is configuration and not file content, and the core column keys resolved from it). Everything derived from the file is given up: the handle, the header read off it under header_mode: file, the parser built around that header and the file’s contigs, any buffered or fully-loaded records, and the chromosome state below.

Stated once, here, because the alternative is deciding it per field – and per field the answer always looks like “this one is small”. It is not about tidiness: a closed table is not necessarily a dropped one. A holder may keep a score, and so its table, long after closing it – an annotation pipeline holds its scores for a whole run – so whatever a closed table retains is retained for as long as that holder lives, and retained for nothing, since open() rebuilds all of it from the file rather than reusing it.

A closed table stays reopenable: open() re-establishes everything released here that a read depends on, and answers exactly as a table that was never closed. (The one release nothing re-reads is the VCF backend’s INFO metadata, its header: read at construction, needed only to build the score definitions there, and never consulted again.) Until it is reopened it refuses the reads that depend on what it read out of the file – that is the contract, and it is what releasing the state above amounts to at the call site. Four of those reads refuse in one stated way, ValueError, on all four backends: get_chromosomes() once chrom_order is released, and get_file_chromosomes() and find_chromosome_length() off the handle their open() establishes and this close() drops – plus get_chromosome_length(), which refuses by relaying what the hook beneath it raises. Those four are what a caller may write an except ValueError around. The hook is the one that must guard, and its stakes are the higher: an unguarded closed table would reach its no-records branch and answer ContigExtent.EMPTY, which is not an error at all (gain#509).

The record reads refuse too, but not in one way, and their exception type is not part of the contract. Neither get_all_records nor get_records_in_region carries a not-open guard of its own: measured on a closed table, some backend/method pairs raise the same ValueError on their way through get_chromosomes(), and the rest run into a pre-existing assert in the fetch path (assert self._bw_file is not None, assert isinstance( self.pysam_file, pysam.TabixFile | pysam.VariantFile), assert self.parser is not None) and hand the caller a message-less AssertionError – or, under python -O which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are there for a different case, a scan already in flight when the close lands; do not catch on them. This whole paragraph used to claim the opposite of all of it – that reading a closed table was unchanged – which was never true of the code it documents (gain#358). No in-tree caller reads a table it has not opened: every read sits behind GenomicScore.is_open().

The one read that does not refuse is chromosome mapping, and it is left that way deliberately. map_chromosome() and unmap_chromosome() return their argument unchanged when rev_chrom_map/chrom_map are None – which is how a table that configures no chrom_mapping answers, and is exactly the state this method leaves behind. So a closed mapped table passes reference-space names through as if they were the file’s, silently, and nothing left on the table can tell the two apart: _build_chrom_mapping() sets chrom_map = None on an OPEN table with no mapping configured, so the field does not distinguish closed from mapping-free, and there is no open/closed flag to consult. Adding one was considered and rejected (gain#358): it is an invariant every backend would have to maintain, bought at the price of a new way for the read path to fail – which is what the release policy above set out not to introduce. Recorded rather than fixed, here and in this package’s __init__ ledger, so that a reader who finds a closed table mapping a name through knows it is a decision and not an oversight.

Released here is the base class’s own file-derived state: the get_file_chromosomes memo and the chromosome mapping _build_chrom_mapping() derives from it, which that method rebuilds – memo included – on every open(); and the header, when header_mode says it is the file’s. A backend’s ``close()`` must call up into this one; what each backend releases on top of it is its own, and test_table_lifetime.py holds all four to the policy: it opens a table, reads through it, closes it, and then requires both that everything the open established – by rebinding, or by changing a field in place, which a by-value snapshot sees (gain#360) – was given up and that nothing the closed table still holds has anything in it; the read is what reaches the buffers a fetch establishes.

abstractmethod find_chromosome_length(chrom: str, step: int = 100000000) → int | ContigExtent

Return the length of a contig, or why there is not one.

The hook every backend implements; get_chromosome_length() is built on it. A returned length is guaranteed to be LARGER than the actual contig length – callers rely on that to split a contig into regions without dropping its tail.

Returns a ContigExtent member instead of a number when the backend has no length to give, and the member says WHY: EMPTY when the backend can prove the contig holds no records, UNDETERMINED when a length simply could not be established and the contig may hold records after all. A caller that splits contigs into regions must treat those two oppositely – skip the first, read the second whole – which is why this hook reports them apart rather than collapsing both into None (gain#509).

Raises ValueError when the QUESTION is bad rather than the answer absent: a table that is not open, or a contig not in get_chromosomes(). Implementations must guard the closed table FIRST, before any read that a closed table refuses – including get_chromosomes(), which the contig-naming diagnostics interpolate (gain#358).

abstractmethod get_all_records() → Generator[tuple[Any, ...], None, None]

Return generator of all records in the table.

get_chromosome_length(chrom: str, step: int = 100000000) → int

Return the length of a chromosome (or contig).

Returned value is guarnteed to be larget than the actual contig length.

The raising view of find_chromosome_length(), for the callers – most of them – that have nothing useful to do with a contig whose length is unavailable and want to be told rather than handed a value they must classify. Concrete here rather than per backend so the two cannot drift: every reason the hook has no number becomes a ValueError, whichever backend produced it, and a new backend gets this behaviour by implementing the hook alone.

get_chromosomes() → list[str]

Return list of contigs in the genomic position table.

get_column_key(col: str) → int | None

Find the index of a column in the table.

Reads the definition; never writes to it. The resolved index used to be memoised back as definition[col]["column_index"] (and the deprecated index/name spellings canonicalised there the same way), but a table’s definition is configuration that outlives this call and is read by more than this table: GenomicScoreImplementation.calc_statistics_hash serialises it. Writing to it made a resource’s statistics hash depend on whether its score had been opened in the current process – and repo-repair computes that hash on both sides of the rebuild it is deciding, in a process that has opened the score and in one that has not. Every fragment score in the deployed GRR was rebuilt on every run because of it (#502).

get_file_chromosomes() → list[str]

Return the chromosomes in the table file, in the file’s own order.

The result is cached for the lifetime of the open table; reopening re-reads it.

abstractmethod get_records_in_region(chrom: str, pos_begin: int | None = None, pos_end: int | None = None) → Generator[tuple[Any, ...], None, None]

Return an iterable over the records in the specified range.

The interval is closed on both sides and 1-based. pos_begin and pos_end are optional and default to the contig’s own bounds; chrom is required.

It used to be optional, and passing None meant “every record in the table” – each backend opened with if chrom is None: yield from self.get_all_records(). That made the default argument list a legal call (get_records_in_region()) that quietly scanned a whole genome, and it gave one method two jobs whose only shared code was the delegation. get_all_records() is that second job, is not going anywhere, and says what it does in its name. Callers that passed no contig call it directly instead.

A caller may stop iterating at any point. A backend that carries state between queries must therefore release it from a finally rather than after the yield loop, which an abandoned generator never reaches: what it retains may not grow with the number of abandoned reads, and the reads that follow must answer as though none had been abandoned. buffered_record_count() is how a backend reports what it is holding, and test_backend_record_contract.py holds every backend to both halves (gain#1120).

Releasing from a finally puts the release under the caller’s control, since that is who decides when a generator is closed – so a backend whose release depends on query order has to say so itself rather than assume it. See TabixGenomicPositionTable._prune_if_current for the one in-tree case and what a stale release would otherwise cost.

get_region_value_arrays(chrom: str, start: int | None, end: int | None, value_columns: Iterable[int], batch_size: int) → Generator[tuple[ndarray, ndarray, dict[int, ndarray]], None, None]

Yield a region’s rows as column arrays, without building records.

The region bounds are named start/end rather than the pos_begin/pos_end of get_records_in_region(), because an implementation of this method builds pos_begin/pos_end arrays in its body – the scalar bounds need names that do not shadow them.

An OPTIONAL fast path: a backend that serves it sets supports_value_arrays and overrides this; the base refuses. Ask before calling – do not probe by catching the exception.

Each batch is (pos_begin, pos_end, {column index: raw cells}): the parsed one-based position arrays, plus the raw cells of each requested payload column. Cells are NOT parsed and rows are NOT clipped to the region – both stay with the caller, exactly as on the record path. batch_size is a hint a backend may ignore when its read granularity is fixed by its own windowing.

has_chromosome(chrom: str) → bool

Answer whether this table carries chrom.

The yes/no half of get_chromosomes(), for the callers – most of them – that only screen a contig and never look at the order. Answered out of a set derived once per open, so the cost does not grow with the table’s contig count nor with where the contig sits in the order; chrom not in table.get_chromosomes() grew with both, and grew worst for a contig the table does NOT carry, which is exactly what a screen exists to detect (gain#1304).

Measured through GenomicScore.get_all_chromosomes() on a real TABIX table – not the in-memory test fixture, whose denominator .out-of-scope/point-read-pre-resolution.md refuses for this layer – at hg38-shaped contig counts, timeit best-of-5 over 20k calls, one screen, in microseconds:

screen             195 contigs   640 contigs   predicate
chr1, index 0            0.136         0.135       0.135
chr22, index 21          0.270         0.262       0.135
tail alt                 1.359         4.311       0.135
absent                   0.973         2.929       0.132

One annotated substitution makes THREE such screens on this tree: the annotator’s and the shared region-read refusal’s, both on the score (0.135us each, the score adding an is_open check to the table’s own 0.070us), and the tabix record read’s, on the table. So at 640 contigs a record on a tail alt paid about 12.9us and one on an absent contig about 8.8us, against a 5.3us point read on the same table; all three together now cost ~0.34us, flat in the contig count and in the contig’s index. The saving is an absolute per-record cost, so it does not shrink against the larger point reads a genome-sized file gives (31us sequential, ~500us random, measured in that same out-of-scope note); it is worth most where the file has many contigs, which is where the screens cost most.

Raises ValueError on a table that is not open, in whatever words that backend’s get_chromosomes() uses – because it is that method the set is derived from. A drop-in for the screens that used to spell the membership out over the list, refusal included.

Derived from :meth:`get_chromosomes` rather than beside it, so the two cannot answer differently. A predicate that disagreed with the list would be the quietest failure this method could have: not a wrong value handed to a caller, but a screen answering “no” for a contig the table has, so the read above it reports no data on a contig full of it and nothing raises. Deriving from the accessor makes that unrepresentable on every backend at once, including the tabix family, whose list is mapped and filtered rather than stored.

The memo’s lifetime is the open table’s, released by close() and by _build_chrom_mapping() – the same two seams as the get_file_chromosomes memo, and per-instance for the reasons set out there.

header: tuple | None

map_chromosome(chromosome: str) → str | None

Map a file contig to its reference genome chromosome.

The inverse of unmap_chromosome(). Returns None when the table configures a chrom_mapping that does not cover chromosome, and chromosome unchanged when it configures none.

abstractmethod open() → GenomicPositionTable

pos_begin_key: int

pos_end_key: int

ref_key: int | None

resource_files() → set[str]

The resource’s files this table reads: the data file, and the index on a backend that reads one.

What the resource implementation hashes and lists as its file set, answered here because it is the table that knows how it opens – the tabix backend adds the index, this base has only the data file. Answered without opening anything.

rev_chrom_map: dict[str, str] | None

supports_value_arrays: ClassVar[bool] = False

unmap_chromosome(chromosome: str) → str | None

Map a reference genome chromosome to its file contig.

The inverse of map_chromosome(). Named for what it undoes: the mapping a caller sees is reference-facing, so unmapping goes back to the file’s own name – which is why every caller spells the result fchrom. Returns None when the table configures a chrom_mapping that does not cover chromosome, and chromosome unchanged when it configures none.

would_resolve_column(col: str) → bool

Whether col will have a key once this table is open.

The question get_column_key() answers, asked of a table that may not be open yet – and asked HERE, because which spellings address a column, and that a bare header column counts as addressing one, is this class’s knowledge and not its callers’.

With a header in hand – header_mode: list names it in the config, and an opened table has read it – this IS get_column_key(), so the two cannot disagree. Without one, the config is the only evidence there is, and the answer is whether the definition addresses the column at all: a col_def carrying none of COLUMN_KEY_SPELLINGS is not an address, and get_column_key would fall past it to the header fallback and resolve None. Answering on the mere PRESENCE of the block would promise a column an empty reference: never delivers.

yields_records: ClassVar[bool] = False

gain.genomic_resources.genomic_position_table.table_bigwig module

class gain.genomic_resources.genomic_position_table.table_bigwig.AdaptiveFetchWindow(target_records: int)

Bases: object

A base-pair window retuned toward a target records-per-call budget.

Owned by a BigWigTable and kept across region fetches on purpose: density is a property of the resource, so what one fetch learns about a track is exactly what the next fetch should start from.

retune(records_fetched: int) → None

Rescale the window from the record count the last call returned.

class gain.genomic_resources.genomic_position_table.table_bigwig.BigWigTable(genomic_resource: GenomicResource, table_definition: dict)

Bases: GenomicPositionTable

bigWig format implementation of the genomic position table.

Yields records – the six-slot plain tuples of the record contract – exactly like the tabix and in-memory backends. A bigWig record’s PAYLOAD is the interval’s value, a bare float (see build_bigwig_parser()); the score layer reads it straight out of the slot, with no index and no parse.

chrom_length_source: ClassVar[ChromLengthSource] = 'bigwig'

close() → None

Close the file and release everything read out of it.

THE RELEASE POLICY, for every backend: after ``close()`` a table holds only what ``open()`` does not rebuild – its resource, its definition, and its configured parameters (the header under header_mode: list, where it is configuration and not file content, and the core column keys resolved from it). Everything derived from the file is given up: the handle, the header read off it under header_mode: file, the parser built around that header and the file’s contigs, any buffered or fully-loaded records, and the chromosome state below.

Stated once, here, because the alternative is deciding it per field – and per field the answer always looks like “this one is small”. It is not about tidiness: a closed table is not necessarily a dropped one. A holder may keep a score, and so its table, long after closing it – an annotation pipeline holds its scores for a whole run – so whatever a closed table retains is retained for as long as that holder lives, and retained for nothing, since open() rebuilds all of it from the file rather than reusing it.

A closed table stays reopenable: open() re-establishes everything released here that a read depends on, and answers exactly as a table that was never closed. (The one release nothing re-reads is the VCF backend’s INFO metadata, its header: read at construction, needed only to build the score definitions there, and never consulted again.) Until it is reopened it refuses the reads that depend on what it read out of the file – that is the contract, and it is what releasing the state above amounts to at the call site. Four of those reads refuse in one stated way, ValueError, on all four backends: get_chromosomes() once chrom_order is released, and get_file_chromosomes() and find_chromosome_length() off the handle their open() establishes and this close() drops – plus get_chromosome_length(), which refuses by relaying what the hook beneath it raises. Those four are what a caller may write an except ValueError around. The hook is the one that must guard, and its stakes are the higher: an unguarded closed table would reach its no-records branch and answer ContigExtent.EMPTY, which is not an error at all (gain#509).

The record reads refuse too, but not in one way, and their exception type is not part of the contract. Neither get_all_records nor get_records_in_region carries a not-open guard of its own: measured on a closed table, some backend/method pairs raise the same ValueError on their way through get_chromosomes(), and the rest run into a pre-existing assert in the fetch path (assert self._bw_file is not None, assert isinstance( self.pysam_file, pysam.TabixFile | pysam.VariantFile), assert self.parser is not None) and hand the caller a message-less AssertionError – or, under python -O which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are there for a different case, a scan already in flight when the close lands; do not catch on them. This whole paragraph used to claim the opposite of all of it – that reading a closed table was unchanged – which was never true of the code it documents (gain#358). No in-tree caller reads a table it has not opened: every read sits behind GenomicScore.is_open().

The one read that does not refuse is chromosome mapping, and it is left that way deliberately. map_chromosome() and unmap_chromosome() return their argument unchanged when rev_chrom_map/chrom_map are None – which is how a table that configures no chrom_mapping answers, and is exactly the state this method leaves behind. So a closed mapped table passes reference-space names through as if they were the file’s, silently, and nothing left on the table can tell the two apart: _build_chrom_mapping() sets chrom_map = None on an OPEN table with no mapping configured, so the field does not distinguish closed from mapping-free, and there is no open/closed flag to consult. Adding one was considered and rejected (gain#358): it is an invariant every backend would have to maintain, bought at the price of a new way for the read path to fail – which is what the release policy above set out not to introduce. Recorded rather than fixed, here and in this package’s __init__ ledger, so that a reader who finds a closed table mapping a name through knows it is a decision and not an oversight.

Released here is the base class’s own file-derived state: the get_file_chromosomes memo and the chromosome mapping _build_chrom_mapping() derives from it, which that method rebuilds – memo included – on every open(); and the header, when header_mode says it is the file’s. A backend’s ``close()`` must call up into this one; what each backend releases on top of it is its own, and test_table_lifetime.py holds all four to the policy: it opens a table, reads through it, closes it, and then requires both that everything the open established – by rebinding, or by changing a field in place, which a by-value snapshot sees (gain#360) – was given up and that nothing the closed table still holds has anything in it; the read is what reaches the buffers a fetch establishes.

find_chromosome_length(chrom: str, step: int = 100000000) → int | ContigExtent

Return the length of a contig, or why there is not one.

The hook every backend implements; get_chromosome_length() is built on it. A returned length is guaranteed to be LARGER than the actual contig length – callers rely on that to split a contig into regions without dropping its tail.

Returns a ContigExtent member instead of a number when the backend has no length to give, and the member says WHY: EMPTY when the backend can prove the contig holds no records, UNDETERMINED when a length simply could not be established and the contig may hold records after all. A caller that splits contigs into regions must treat those two oppositely – skip the first, read the second whole – which is why this hook reports them apart rather than collapsing both into None (gain#509).

Raises ValueError when the QUESTION is bad rather than the answer absent: a table that is not open, or a contig not in get_chromosomes(). Implementations must guard the closed table FIRST, before any read that a closed table refuses – including get_chromosomes(), which the contig-naming diagnostics interpolate (gain#358).

get_all_records() → Generator[tuple[Any, ...], None, None]

Return generator of all records in the table.

get_records_in_region(chrom: str, pos_begin: int | None = None, pos_end: int | None = None) → Generator[tuple[Any, ...], None, None]

Yield the records overlapping the region, as record tuples.

Chromosome mapping is applied on both ends, unchanged: the query contig is mapped reference->file by _map_file_chrom before the fetch, and each record’s CHROM slot carries chrom back – the reference-space contig the caller asked for – so the result stays in reference space. The intervals go to the parser RAW, in the file’s 0-based half-open coordinates, and the parser converts them to the contract’s closed one-based interval in the same expression that assembles the record (see build_bigwig_parser()).

The adaptive chunk walk is inlined here rather than sitting behind a fetch generator of its own, which would resume once per record to hand over an interval this method passes straight on (gain#823). It is the only fetch strategy; the retained-buffer one is gone (see docs/adr/0002-remove-bigwig-fetch-buffering.md).

get_region_value_arrays(chrom: str, start: int | None, end: int | None, value_columns: Iterable[int], batch_size: int) → Generator[tuple[ndarray, ndarray, dict[int, ndarray]], None, None]

Yield a region’s intervals as column arrays, without records.

The bigWig counterpart of TabixGenomicPositionTable.get_region_value_arrays(): a fast path for a full sequential scan (statistics). It reuses the adaptive _fetch_chunk() windowing – so memory stays bounded exactly as the record path’s – but turns each chunk of raw intervals into arrays in one shot rather than building a Record per interval. The coordinates match the record path’s: the raw zero-based half-open [begin, end) becomes closed one-based (begin + 1, end) – here in one vectorized shift, there in build_bigwig_parser().

A bigWig has exactly one column, and its index is 0. A record’s PAYLOAD is the interval’s value (see build_bigwig_parser()), so “the payload’s column 0” and “the value” are the same thing, and any other index names a column this backend does not have – refused with a KeyError naming the resource rather than served whatever the old four-tuple reconstruction happened to hold at that offset. That reconstruction existed to make a bad index raise the IndexError the record path raised; the record path no longer indexes anything, and a misconfigured index is now refused when the score is opened, by name, which is a better diagnostic than either. This check is the backstop for a caller that reaches the table directly.

batch_size is accepted for a uniform producer signature; the batch size here is set by the adaptive fetch window, not this argument.

open() → BigWigTable

supports_value_arrays: ClassVar[bool] = True

yields_records: ClassVar[bool] = True

gain.genomic_resources.genomic_position_table.table_bigwig.build_bigwig_parser() → Callable[[str, tuple[int, int, float]], tuple[Any, ...]]

Build a (chrom, interval) -> record parser for the bigWig backend.

Built once, at BigWigTable.open(), and called per line – the point of the record migration is that a fetched line no longer constructs a per-line BigWigLine adapter object, only a plain record tuple.

A bigWig record’s PAYLOAD is the value itself – a bare float, not a tuple. A bigWig carries one number per interval; everything else the payload used to repeat ((chrom, pos_begin, pos_end, value), inherited from the retired BigWigLine’s raw row) is already decoded into the record’s own slots, and the repetition existed only so the value was addressable at payload[3]. With the narrowing, reading a bigWig score is record[PAYLOAD] – an identity, with no index and no parse (see bigwig_scores.extract_bigwig_value). REF and ALT are always None: a bigWig carries neither.

The three reasons the earlier, wider shape was kept are recorded – and answered – in the ledger entry in this package’s __init__. The short of it: index: 3 survives as an accepted deprecated no-op rather than as a payload shape, the out-of-range IndexError is superseded by an open-time refusal that names the resource and the score, and the “no speed to be had” measurement predated the removal of the parse.

The parser owns the coordinate conversion. It takes the interval RAW, in the file’s 0-based half-open coordinates, and produces the closed one-based interval of the record contract by doing the +1 on the begin itself – the same fusion of transform and record construction the tabular parser performs for its zero-based tables. This reverses an earlier design point, which converted in the fetch methods and documented the parser as having no transform of its own: converting a layer up meant building a 3-tuple per interval whose only reader was this function (gain#823).

The parser closes over nothing. A bigWig has no configurable transform for it to specialise on – it is a binary format with a fixed layout, its conversion is fixed by the format rather than by config, and its result contig is the (already reference-mapped) query chrom threaded in per call. It is still built here, once, rather than inlined, to keep the shape of the three record backends identical: a parser is built at open() and produces the records the fetch path yields.

gain.genomic_resources.genomic_position_table.table_inmemory module

class gain.genomic_resources.genomic_position_table.table_inmemory.InmemoryGenomicPositionTable(genomic_resource: GenomicResource, table_definition: dict, file_format: str)

Bases: GenomicPositionTable

In-memory genomic position table.

Loads the whole file into memory as immutable record tuples (the record contract), keyed by their reference contig. The row->record parser is built once when the table is opened, since resolving the column keys and the chromosome map needs the header/file contigs, which are only known then.

Empty/unknown-contig policy (consistent across the four read methods). A contig can be in get_chromosomes() yet have no records – e.g. a chrom_mapping file that maps a reference contig onto a file contig with no data rows. Such a contig is known but empty:

• get_all_records() skips a known-but-empty contig (yields nothing for it, the other contigs still stream);

• get_records_in_region() raises ValueError when the contig is not in get_chromosomes(), and yields nothing for a known-but-empty one;

• find_chromosome_length() raises ValueError when the contig is unknown, and returns ContigExtent.EMPTY for a known-but-empty one. Because this backend holds the whole file, having no records PROVES the contig has none, which is a fact a caller can act on rather than an error – so the two cases this method used to conflate are now apart (gain#509);

• get_chromosome_length, inherited from the base class, is the raising view of that hook: it still raises ValueError for the unknown and the known-but-empty contig, with the same message as before, so a caller of it sees no change.

A CLOSED table is not a case of that policy and is refused ahead of it: close() empties records_by_chr and releases the contig list, so every contig would otherwise look known-but-empty and no diagnostic naming the table’s contigs could be built at all. That mattered for the message before and matters more now, because the known-but-empty answer is no longer an exception: a closed table falling through would report every contig as proven-empty, and a caller would skip the whole genome without being told. Both find_chromosome_length() and _load_file_chromosomes() therefore check str_stream first and say the table is not open, as the other three backends do (gain#358; the contract is stated on GenomicPositionTable.close()).

FORMAT_DEF: ClassVar[dict] = {'csv': (',', '\n\r', False), 'mem': (None, ' \t\n\r', True), 'tsv': ('\t', '\n\r', False)}

chrom_length_source: ClassVar[ChromLengthSource] = 'table_extent'

close() → None

Close the file and release everything read out of it.

THE RELEASE POLICY, for every backend: after ``close()`` a table holds only what ``open()`` does not rebuild – its resource, its definition, and its configured parameters (the header under header_mode: list, where it is configuration and not file content, and the core column keys resolved from it). Everything derived from the file is given up: the handle, the header read off it under header_mode: file, the parser built around that header and the file’s contigs, any buffered or fully-loaded records, and the chromosome state below.

Stated once, here, because the alternative is deciding it per field – and per field the answer always looks like “this one is small”. It is not about tidiness: a closed table is not necessarily a dropped one. A holder may keep a score, and so its table, long after closing it – an annotation pipeline holds its scores for a whole run – so whatever a closed table retains is retained for as long as that holder lives, and retained for nothing, since open() rebuilds all of it from the file rather than reusing it.

A closed table stays reopenable: open() re-establishes everything released here that a read depends on, and answers exactly as a table that was never closed. (The one release nothing re-reads is the VCF backend’s INFO metadata, its header: read at construction, needed only to build the score definitions there, and never consulted again.) Until it is reopened it refuses the reads that depend on what it read out of the file – that is the contract, and it is what releasing the state above amounts to at the call site. Four of those reads refuse in one stated way, ValueError, on all four backends: get_chromosomes() once chrom_order is released, and get_file_chromosomes() and find_chromosome_length() off the handle their open() establishes and this close() drops – plus get_chromosome_length(), which refuses by relaying what the hook beneath it raises. Those four are what a caller may write an except ValueError around. The hook is the one that must guard, and its stakes are the higher: an unguarded closed table would reach its no-records branch and answer ContigExtent.EMPTY, which is not an error at all (gain#509).

The record reads refuse too, but not in one way, and their exception type is not part of the contract. Neither get_all_records nor get_records_in_region carries a not-open guard of its own: measured on a closed table, some backend/method pairs raise the same ValueError on their way through get_chromosomes(), and the rest run into a pre-existing assert in the fetch path (assert self._bw_file is not None, assert isinstance( self.pysam_file, pysam.TabixFile | pysam.VariantFile), assert self.parser is not None) and hand the caller a message-less AssertionError – or, under python -O which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are there for a different case, a scan already in flight when the close lands; do not catch on them. This whole paragraph used to claim the opposite of all of it – that reading a closed table was unchanged – which was never true of the code it documents (gain#358). No in-tree caller reads a table it has not opened: every read sits behind GenomicScore.is_open().

The one read that does not refuse is chromosome mapping, and it is left that way deliberately. map_chromosome() and unmap_chromosome() return their argument unchanged when rev_chrom_map/chrom_map are None – which is how a table that configures no chrom_mapping answers, and is exactly the state this method leaves behind. So a closed mapped table passes reference-space names through as if they were the file’s, silently, and nothing left on the table can tell the two apart: _build_chrom_mapping() sets chrom_map = None on an OPEN table with no mapping configured, so the field does not distinguish closed from mapping-free, and there is no open/closed flag to consult. Adding one was considered and rejected (gain#358): it is an invariant every backend would have to maintain, bought at the price of a new way for the read path to fail – which is what the release policy above set out not to introduce. Recorded rather than fixed, here and in this package’s __init__ ledger, so that a reader who finds a closed table mapping a name through knows it is a decision and not an oversight.

Released here is the base class’s own file-derived state: the get_file_chromosomes memo and the chromosome mapping _build_chrom_mapping() derives from it, which that method rebuilds – memo included – on every open(); and the header, when header_mode says it is the file’s. A backend’s ``close()`` must call up into this one; what each backend releases on top of it is its own, and test_table_lifetime.py holds all four to the policy: it opens a table, reads through it, closes it, and then requires both that everything the open established – by rebinding, or by changing a field in place, which a by-value snapshot sees (gain#360) – was given up and that nothing the closed table still holds has anything in it; the read is what reaches the buffers a fetch establishes.

find_chromosome_length(chrom: str, step: int = 0) → int | ContigExtent

Return the length of a contig, or why there is not one.

The hook every backend implements; get_chromosome_length() is built on it. A returned length is guaranteed to be LARGER than the actual contig length – callers rely on that to split a contig into regions without dropping its tail.

Returns a ContigExtent member instead of a number when the backend has no length to give, and the member says WHY: EMPTY when the backend can prove the contig holds no records, UNDETERMINED when a length simply could not be established and the contig may hold records after all. A caller that splits contigs into regions must treat those two oppositely – skip the first, read the second whole – which is why this hook reports them apart rather than collapsing both into None (gain#509).

Raises ValueError when the QUESTION is bad rather than the answer absent: a table that is not open, or a contig not in get_chromosomes(). Implementations must guard the closed table FIRST, before any read that a closed table refuses – including get_chromosomes(), which the contig-naming diagnostics interpolate (gain#358).

get_all_records() → Generator[tuple[Any, ...], None, None]

Return generator of all records in the table.

get_records_in_region(chrom: str, pos_begin: int | None = None, pos_end: int | None = None) → Generator[tuple[Any, ...], None, None]

Return an iterable over the records in the specified range.

The interval is closed on both sides and 1-based. pos_begin and pos_end are optional and default to the contig’s own bounds; chrom is required.

It used to be optional, and passing None meant “every record in the table” – each backend opened with if chrom is None: yield from self.get_all_records(). That made the default argument list a legal call (get_records_in_region()) that quietly scanned a whole genome, and it gave one method two jobs whose only shared code was the delegation. get_all_records() is that second job, is not going anywhere, and says what it does in its name. Callers that passed no contig call it directly instead.

A caller may stop iterating at any point. A backend that carries state between queries must therefore release it from a finally rather than after the yield loop, which an abandoned generator never reaches: what it retains may not grow with the number of abandoned reads, and the reads that follow must answer as though none had been abandoned. buffered_record_count() is how a backend reports what it is holding, and test_backend_record_contract.py holds every backend to both halves (gain#1120).

Releasing from a finally puts the release under the caller’s control, since that is who decides when a generator is closed – so a backend whose release depends on query order has to say so itself rather than assume it. See TabixGenomicPositionTable._prune_if_current for the one in-tree case and what a stale release would otherwise cost.

open() → InmemoryGenomicPositionTable

yields_records: ClassVar[bool] = True

gain.genomic_resources.genomic_position_table.table_tabix module

class gain.genomic_resources.genomic_position_table.table_tabix.TabixGenomicPositionTable(genomic_resource: GenomicResource, table_definition: dict)

Bases: GenomicPositionTable

Represents Tabix file genome position table.

Yields records – the six-slot tuples built by the tabular parser – whose payload is the raw pysam row. The row is handed on by reference and never materialised into a tuple of columns: it decodes a column only when a caller indexes it, which is what keeps a 454-score resource from paying for 454 decodes when a caller wants one score.

The read cascade for a region query, in order:

1. the buffering decision – a query wider than BUFFER_MAXSIZE (or open-ended) is served straight from the file, unbuffered;

2. the provably-empty-gap short-circuit – the query starts after the previous query’s end and ends before the first buffered record: the records in between were already read and none of them reach it, so the answer is provably empty without touching the file;

3. the buffer hit – the query’s start is inside the buffered window;

4. the sequential seek – the query’s start is beyond the buffer but within jump_threshold of it, so reading forward beats a fresh pysam fetch (which drops the buffer and its index lookup);

5. the fresh fetch – everything else re-seeks the file.

_gen_from_tabix() buffers every record it pulls before it checks whether that record has run past the end of the query. The record that terminates a read is therefore buffered although it is never yielded – and the short-circuit in (2) and the window in (3) both depend on it being there. Do not reorder those two steps.

BUFFER_MAXSIZE: int = 20000

buffered_record_count() → int

The records held in the LineBuffer between queries.

The one backend with a non-zero answer. It over-reports by design: eviction is amortized (gain#287), so between walks the buffer knowingly holds records that are already dead. What the contract asks of this number is that it stay bounded across a scan, not that it equal the live set on any given query.

chrom_length_source: ClassVar[ChromLengthSource] = 'tabix_estimate'

close() → None

Close the file and release everything read out of it.

THE RELEASE POLICY, for every backend: after ``close()`` a table holds only what ``open()`` does not rebuild – its resource, its definition, and its configured parameters (the header under header_mode: list, where it is configuration and not file content, and the core column keys resolved from it). Everything derived from the file is given up: the handle, the header read off it under header_mode: file, the parser built around that header and the file’s contigs, any buffered or fully-loaded records, and the chromosome state below.

Stated once, here, because the alternative is deciding it per field – and per field the answer always looks like “this one is small”. It is not about tidiness: a closed table is not necessarily a dropped one. A holder may keep a score, and so its table, long after closing it – an annotation pipeline holds its scores for a whole run – so whatever a closed table retains is retained for as long as that holder lives, and retained for nothing, since open() rebuilds all of it from the file rather than reusing it.

A closed table stays reopenable: open() re-establishes everything released here that a read depends on, and answers exactly as a table that was never closed. (The one release nothing re-reads is the VCF backend’s INFO metadata, its header: read at construction, needed only to build the score definitions there, and never consulted again.) Until it is reopened it refuses the reads that depend on what it read out of the file – that is the contract, and it is what releasing the state above amounts to at the call site. Four of those reads refuse in one stated way, ValueError, on all four backends: get_chromosomes() once chrom_order is released, and get_file_chromosomes() and find_chromosome_length() off the handle their open() establishes and this close() drops – plus get_chromosome_length(), which refuses by relaying what the hook beneath it raises. Those four are what a caller may write an except ValueError around. The hook is the one that must guard, and its stakes are the higher: an unguarded closed table would reach its no-records branch and answer ContigExtent.EMPTY, which is not an error at all (gain#509).

The record reads refuse too, but not in one way, and their exception type is not part of the contract. Neither get_all_records nor get_records_in_region carries a not-open guard of its own: measured on a closed table, some backend/method pairs raise the same ValueError on their way through get_chromosomes(), and the rest run into a pre-existing assert in the fetch path (assert self._bw_file is not None, assert isinstance( self.pysam_file, pysam.TabixFile | pysam.VariantFile), assert self.parser is not None) and hand the caller a message-less AssertionError – or, under python -O which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are there for a different case, a scan already in flight when the close lands; do not catch on them. This whole paragraph used to claim the opposite of all of it – that reading a closed table was unchanged – which was never true of the code it documents (gain#358). No in-tree caller reads a table it has not opened: every read sits behind GenomicScore.is_open().

The one read that does not refuse is chromosome mapping, and it is left that way deliberately. map_chromosome() and unmap_chromosome() return their argument unchanged when rev_chrom_map/chrom_map are None – which is how a table that configures no chrom_mapping answers, and is exactly the state this method leaves behind. So a closed mapped table passes reference-space names through as if they were the file’s, silently, and nothing left on the table can tell the two apart: _build_chrom_mapping() sets chrom_map = None on an OPEN table with no mapping configured, so the field does not distinguish closed from mapping-free, and there is no open/closed flag to consult. Adding one was considered and rejected (gain#358): it is an invariant every backend would have to maintain, bought at the price of a new way for the read path to fail – which is what the release policy above set out not to introduce. Recorded rather than fixed, here and in this package’s __init__ ledger, so that a reader who finds a closed table mapping a name through knows it is a decision and not an oversight.

Released here is the base class’s own file-derived state: the get_file_chromosomes memo and the chromosome mapping _build_chrom_mapping() derives from it, which that method rebuilds – memo included – on every open(); and the header, when header_mode says it is the file’s. A backend’s ``close()`` must call up into this one; what each backend releases on top of it is its own, and test_table_lifetime.py holds all four to the policy: it opens a table, reads through it, closes it, and then requires both that everything the open established – by rebinding, or by changing a field in place, which a by-value snapshot sees (gain#360) – was given up and that nothing the closed table still holds has anything in it; the read is what reaches the buffers a fetch establishes.

find_chromosome_length(chrom: str, step: int = 100000000) → int | ContigExtent

Return the length of a contig, or why there is not one.

The hook every backend implements; get_chromosome_length() is built on it. A returned length is guaranteed to be LARGER than the actual contig length – callers rely on that to split a contig into regions without dropping its tail.

Returns a ContigExtent member instead of a number when the backend has no length to give, and the member says WHY: EMPTY when the backend can prove the contig holds no records, UNDETERMINED when a length simply could not be established and the contig may hold records after all. A caller that splits contigs into regions must treat those two oppositely – skip the first, read the second whole – which is why this hook reports them apart rather than collapsing both into None (gain#509).

Raises ValueError when the QUESTION is bad rather than the answer absent: a table that is not open, or a contig not in get_chromosomes(). Implementations must guard the closed table FIRST, before any read that a closed table refuses – including get_chromosomes(), which the contig-naming diagnostics interpolate (gain#358).

get_all_records() → Generator[tuple[Any, ...], None, None]

Return generator of all records in the table.

get_line_iterator(chrom: str | None = None, pos_begin: int | None = None) → Generator[tuple[Any, ...] | None, None, None]

Fetch the raw rows and parse them into records.

A row whose contig is absent from a configured chromosome map parses to None and is dropped by the callers, exactly as the adapter-era transform dropped it.

get_records_in_region(chrom: str, pos_begin: int | None = None, pos_end: int | None = None) → Generator[tuple[Any, ...], None, None]

Yield the records overlapping the region.

The PAYLOAD slot is backend-dependent, and the static type does not say so. This method is inherited by VCFGenomicPositionTable, whose records carry a (variant record, allele index) pair in the slot where a tabix record carries the raw tabular row. Both are tuple[Any, ...], so the type checker cannot tell them apart.

So: narrowing a table to this class with isinstance and then indexing record[PAYLOAD][i] for a column is only valid once you know the table is not a VCF one – which is a question about the table, and the score layer asks it exactly once, when it picks the score line class (GenomicScore.open). The five decoded slots (CHROM … ALT) are safe either way: they mean the same thing in every backend, which is what lets the buffer and the read cascade below treat all records alike.

get_region_value_arrays(chrom: str, start: int | None, end: int | None, value_columns: Iterable[int], batch_size: int) → Generator[tuple[ndarray, ndarray, dict[int, ndarray]], None, None]

Yield a region’s rows as column arrays, without building records.

A fast path for a full sequential scan (statistics): the rows are read straight from pysam and returned per batch as the parsed one-based pos_begin/pos_end int arrays plus the raw string cells of each requested column index – paying neither the per-row Record tuple nor the parser call. The one-based / zero-based transform matches build_tabular_parser() exactly (pos_begin += 1, and a single-base zero-based interval bumps pos_end too); the contig is fixed by the fetch, so no per-row chromosome map is needed.

The read starts and stops where get_records_in_region() would: fetch begins at start - 1 and a row whose parsed pos_begin runs past end terminates the scan (that row and everything after it are not yielded), mirroring _gen_from_tabix. Records ending before start are still yielded here and dropped by the caller’s clip, exactly as the per-record path drops them.

property index_filename: str | None

The index this table’s definition configures, if any.

None means “not configured” – the protocol then resolves the index of filename itself (manifest first, probe second; see resolve_tabix_index_filename_for_read). A table opens its file at more than one site (the VCF backend opens it a second time to read the file’s contigs off the index), and every one of them must use the SAME index, so the key is read here rather than at each open (gain#596).

open() → TabixGenomicPositionTable

resource_files() → set[str]

The data file and its index, or the data file alone with a warning.

The index is resolved the way open() resolves it: the configured index_filename when there is one, the conventional .tbi / .csi probe otherwise – but over the resource MANIFEST rather than the filesystem, and answering None where the open would raise. Only a name the manifest carries is ever returned: the statistics hash looks every file-set entry up in that same manifest and would raise on a missing key. A configured index absent from the manifest is a misconfiguration; it is reported and dropped rather than falling back to the conventional probe, which would silently hash an index the table does not read (gain#595).

supports_value_arrays: ClassVar[bool] = True

yields_records: ClassVar[bool] = True

gain.genomic_resources.genomic_position_table.table_vcf module

class gain.genomic_resources.genomic_position_table.table_vcf.VCFGenomicPositionTable(genomic_resource: GenomicResource, table_definition: dict)

Bases: TabixGenomicPositionTable

Represents a VCF file genome position table.

Yields records – the same six-slot plain tuples every other record backend yields – so it inherits its tabix parent’s read cascade and record-indexed LineBuffer as they are: the buffer windows a VCF record by the very slots (CHROM, POS_BEGIN, POS_END) it windows a tabix record by, and neither knows nor asks which backend built the one it is holding.

Its PAYLOAD is not a raw row. A VCF record carries (variant record, allele index, info, info_meta) in the slot where a tabix record carries the raw tabular row (see VARIANT/ALLELE_INDEX/INFO/ INFO_META above), because a VCF score is not a column: it is an INFO field, looked up by name against the variant’s header metadata and selected by allele. That lookup lives in one place – vcf_scores.extract_vcf_value, bound once per score when it is opened. Only the five decoded slots (CHROM … ALT) mean the same thing across every backend.

CHROM = 'CHROM'

POS_BEGIN = 'POS'

POS_END = 'POS'

close() → None

Close the file and release everything read out of it.

THE RELEASE POLICY, for every backend: after ``close()`` a table holds only what ``open()`` does not rebuild – its resource, its definition, and its configured parameters (the header under header_mode: list, where it is configuration and not file content, and the core column keys resolved from it). Everything derived from the file is given up: the handle, the header read off it under header_mode: file, the parser built around that header and the file’s contigs, any buffered or fully-loaded records, and the chromosome state below.

Stated once, here, because the alternative is deciding it per field – and per field the answer always looks like “this one is small”. It is not about tidiness: a closed table is not necessarily a dropped one. A holder may keep a score, and so its table, long after closing it – an annotation pipeline holds its scores for a whole run – so whatever a closed table retains is retained for as long as that holder lives, and retained for nothing, since open() rebuilds all of it from the file rather than reusing it.

A closed table stays reopenable: open() re-establishes everything released here that a read depends on, and answers exactly as a table that was never closed. (The one release nothing re-reads is the VCF backend’s INFO metadata, its header: read at construction, needed only to build the score definitions there, and never consulted again.) Until it is reopened it refuses the reads that depend on what it read out of the file – that is the contract, and it is what releasing the state above amounts to at the call site. Four of those reads refuse in one stated way, ValueError, on all four backends: get_chromosomes() once chrom_order is released, and get_file_chromosomes() and find_chromosome_length() off the handle their open() establishes and this close() drops – plus get_chromosome_length(), which refuses by relaying what the hook beneath it raises. Those four are what a caller may write an except ValueError around. The hook is the one that must guard, and its stakes are the higher: an unguarded closed table would reach its no-records branch and answer ContigExtent.EMPTY, which is not an error at all (gain#509).

The record reads refuse too, but not in one way, and their exception type is not part of the contract. Neither get_all_records nor get_records_in_region carries a not-open guard of its own: measured on a closed table, some backend/method pairs raise the same ValueError on their way through get_chromosomes(), and the rest run into a pre-existing assert in the fetch path (assert self._bw_file is not None, assert isinstance( self.pysam_file, pysam.TabixFile | pysam.VariantFile), assert self.parser is not None) and hand the caller a message-less AssertionError – or, under python -O which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are there for a different case, a scan already in flight when the close lands; do not catch on them. This whole paragraph used to claim the opposite of all of it – that reading a closed table was unchanged – which was never true of the code it documents (gain#358). No in-tree caller reads a table it has not opened: every read sits behind GenomicScore.is_open().

The one read that does not refuse is chromosome mapping, and it is left that way deliberately. map_chromosome() and unmap_chromosome() return their argument unchanged when rev_chrom_map/chrom_map are None – which is how a table that configures no chrom_mapping answers, and is exactly the state this method leaves behind. So a closed mapped table passes reference-space names through as if they were the file’s, silently, and nothing left on the table can tell the two apart: _build_chrom_mapping() sets chrom_map = None on an OPEN table with no mapping configured, so the field does not distinguish closed from mapping-free, and there is no open/closed flag to consult. Adding one was considered and rejected (gain#358): it is an invariant every backend would have to maintain, bought at the price of a new way for the read path to fail – which is what the release policy above set out not to introduce. Recorded rather than fixed, here and in this package’s __init__ ledger, so that a reader who finds a closed table mapping a name through knows it is a decision and not an oversight.

Released here is the base class’s own file-derived state: the get_file_chromosomes memo and the chromosome mapping _build_chrom_mapping() derives from it, which that method rebuilds – memo included – on every open(); and the header, when header_mode says it is the file’s. A backend’s ``close()`` must call up into this one; what each backend releases on top of it is its own, and test_table_lifetime.py holds all four to the policy: it opens a table, reads through it, closes it, and then requires both that everything the open established – by rebinding, or by changing a field in place, which a by-value snapshot sees (gain#360) – was given up and that nothing the closed table still holds has anything in it; the read is what reaches the buffers a fetch establishes.

get_line_iterator(chrom: str | None = None, pos_begin: int | None = None) → Generator[tuple[Any, ...] | None, None, None]

Fetch the variant records and parse them into records, per allele.

One variant record becomes one record per ALT allele – that is the VCF backend’s whole shape, and it is why its parser takes an allele index. A variant whose ALT is absent (‘.’) has no alternative allele at all, and yields a single record with a None allele index; the score layer reads its reference-allele INFO values accordingly.

open() → VCFGenomicPositionTable

supports_value_arrays: ClassVar[bool] = False

gain.genomic_resources.genomic_position_table.table_vcf.build_vcf_parser(rev_chrom_map: dict[str, str] | None) → Callable[[VariantRecord, int | None], tuple[Any, ...] | None]

Build a (variant, allele index) -> record parser for the VCF backend.

The parser is a pure function of the reverse chromosome map (file contig -> reference contig), specialised on its presence once, here, rather than branched per record – the same fusion build_tabular_parser does for the tabular backends, with the chromosome map as the only transform a VCF table can configure (a VCF is always one-based, so there is no zero-based variant, and REF/ALT are structural rather than configured columns).

A variant whose contig is absent from the map yields None and the record is dropped by the callers, exactly as the tabular parser’s rows are.

The map’s presence selects the path, not its contents. An empty map – which a well-formed chrom_mapping.filename with no body rows yields – is a map that maps nothing, so every record is dropped. It is not treated as “no mapping at all”: a table configured with such a file has no chromosomes either (get_chromosomes() comes from the mapping file), and passing the file contigs through would make it yield records on contigs it says it does not have – get_records_in_region would raise for the very contig get_all_records had just handed back. This is the same rule build_tabular_parser() follows, and it is a deliberate change from the pre-record VCF backend, which tested the map for truthiness and so identity-mapped an empty one. (Pinned in test_genomic_position_table.py by test_an_empty_chrom_mapping_file_maps_nothing_and_so_drops_every_record.)

gain.genomic_resources.genomic_position_table.utils module

gain.genomic_resources.genomic_position_table.utils.build_genomic_position_table(resource: GenomicResource, table_definition: dict) → GenomicPositionTable

Instantiate a genome position table from a genomic resource.

Module contents

Genomic position table backends.

Removed export: ``VCFLine``. It was in this package’s __all__ and so a public name of gain; #237 deleted it, because the VCF backend no longer builds a per-line object at all – it yields records (record.py), like every other record backend. This is a breaking export change, recorded here because nothing else records it: an importer of VCFLine now gets an ImportError. There is no drop-in replacement object, and none is wanted – a VCF line is a record tuple, and what used to be read off a VCFLine is read from the record’s slots (CHROM … ALT) or, for scores, through the score layer’s GenomicScore.get_score_value_from_record.

Removed exports: ``Line`` and ``BigWigLine`` (and, with them, the LineBase protocol they satisfied and the row() method all three declared). Both were in this package’s __all__, so both are public names of gain, and an importer of either now gets an ImportError – the same breaking change as VCFLine, recorded here for the same reason. With VCFLine already gone in #237, these were the last two of the three line adapters this package ever exported. #239 deleted them once #238 had migrated bigWig, the last backend that built one: every backend now yields records, so nothing constructed a line adapter and nothing consumed one. LineBase went with them because a protocol with no implementors describes nothing, and row() – which serialised an adapter back to its raw row – went because its only caller, save_as_tabix_table, was itself dead and was deleted in #235.

The score layer’s adapter-era ScoreLine was deleted by #239 too, but it was never exported from this package and was never an adapter itself – it wrapped one, asserting its line was a Line or a BigWigLine. That assert is why it could not outlive them. It has no bearing on this package’s exports; a score caller goes through GenomicScore.get_score_value_from_record (see below).

There is no replacement and no deprecation shim. A shim was considered and rejected: it costs nothing to anyone who does not call it, but hands anyone who does call it back the exact per-line allocation this whole migration exists to remove. A caller reading coordinates off a Line reads them from the record’s slots instead (record[CHROM], record[POS_BEGIN], record[POS_END], record[REF], record[ALT]); a caller using line.get(key) for a column indexes the record’s payload (record[PAYLOAD][key]); a caller wanting the whole raw row back takes tuple(record[PAYLOAD]). For scores, none of this is the intended route at all – go through the score layer’s GenomicScore.get_score_value_from_record or get_score_values_from_record, which read the same slots and additionally handle NA values, parsing and aggregation.

The ``tuple()`` around that last one is the migration, not noise. row() returned tuple(self._data) in both adapters – an immutable snapshot of the row, taken there and then. record[PAYLOAD] is not that: the payload is the backend’s row held by reference, deliberately neither copied nor frozen (record.py), and for the tabix backend it is a pysam.TupleProxy that LineBuffer may still be holding: write to it and you mutate a buffered row (see line.py).

It does NOT get reused as the fetch advances, which an earlier version of this note claimed. pysam.asTuple() hands up one proxy object PER LINE (as TabixGenomicPositionTable.get_line_iterator says), so retaining a record past its iteration keeps its own cells: materialising a region with list() and reading the payloads afterwards gives each row’s real values, measured. The mutation hazard above is real; the aliasing one was not.

tuple(record[PAYLOAD]) reproduces what row() handed back, and is what a row() caller migrates to.

``fchrom`` has no record equivalent, and is the one ``LineBase`` attribute with no slot to move to. There is no FCHROM slot, and record[CHROM] is NOT one: Line carried the file’s own contig in fchrom and the reference contig in chrom, and under a configured chrom_mapping those hold different values – the tabix backend overwrote chrom with the mapped reference contig and left fchrom at the file’s. A record’s CHROM slot is the mapped one, so migrating line.fchrom to record[CHROM] is not an error, it is wrong data, on exactly the tables that configure a map.

The file contig is still readable – but from the table, which is why it is not a slot. For the tabular backends it is record[PAYLOAD][table.chrom_key] (the raw row’s contig cell – literally the expression Line.__init__ read its own fchrom from), or table.unmap_chromosome(record[CHROM]) back through the map. Both need the table, and a caller holding records has one. Adding a sixth decoded slot to spare it that is not on the table here: a record is what every backend yields, and a file contig is not something every backend has to give. bigWig’s payload repeats the already-mapped reference contig (BigWigLine.fchrom was set from it – that adapter’s fchrom was never a file contig at all), and a VCF record’s payload is a variant, whose file contig is record[PAYLOAD][VARIANT].contig. The slot would mean three different things, which is the sort of thing the five decoded slots exist to not do.

LineBuffer is NOT part of that removal and remains exported: it outlived the adapters it used to hold and now buffers records (see its own note below).

Removed method: ``LineBuffer.pop_first``. LineBuffer is in __all__ below, so this too is a breaking change to a public name of gain, recorded here for the same reason. It had no caller anywhere in the stack (gain or gpf) and no replacement is wanted: #250 gave the buffer an invariant that a bare popleft cannot keep. Eviction has to go through LineBuffer.prune(), which drops a record only when its pos_end has fallen below the query – wherever that record sits (gain#287), not merely while it is at the head. That rule is what makes the buffer complete from the pruned-to position onwards, and completeness is what the read path’s buffer-hit answer rests on. pop_first dropped the leftmost record unconditionally, so it could evict one that still overlapped later queries and leave the buffer answering from a hole – silently, and with no fall-through to the file to rescue it. (It would also leave _max_end/_max_width stale, but only ever high, which is the harmless direction – see LineBuffer. The completeness break is the real one.) Changed extension point: a backend now implements ``_load_file_chromosomes``, not ``get_file_chromosomes``. Neither name is in __all__ below, so this breaks no public name of gain – it is recorded here because get_file_chromosomes was an abstract method whose docstring named it the thing “to be overwritten by the subclass”, which makes it the documented way to write a backend, and an out-of-tree backend that overrides the old name now fails to instantiate (the new abstract hook is unimplemented). No such backend exists anywhere in the stack; every in-tree one was migrated with the change.

get_file_chromosomes still exists, unchanged in name, signature and meaning. What changed is that it is now CONCRETE on GenomicPositionTable, memoising per instance over the new hook. It carried functools.cache before, which keyed a class-level memo by self and so pinned every table that was ever opened for the life of the process – unbounded growth under grr_manage resource-repair, which builds one table per region task (gain#345). A backend migrates by renaming its override and deleting the decorator; it must not memoise on its own, since the base class now does.

Changed contract: a CLOSED table refuses reads. TabixGenomicPositionTable, BigWigTable and VCFGenomicPositionTable are in __all__ below, so this is a change to public names of gain – recorded here for the same reason as the removals above, because nothing else records it. #350 made close() release everything a table read out of its file, the contig order, the chromosome map and the get_file_chromosomes memo included, and the reads that used to be answered out of that retained state now fail instead. Four of them fail in one stated way, ValueError, on all four backends: get_chromosomes(), get_file_chromosomes(), find_chromosome_length() and get_chromosome_length(). Those four are what an out-of-tree caller may write an except ValueError around. find_chromosome_length is the one that reads the file handle – the raising wrapper above it only classifies what it returns – and it carries the guard for a reason beyond the diagnostic: its other answers include ContigExtent.EMPTY, which is not an error, so a closed table falling through to it would report every contig as holding no records and a whole-genome scan would skip the genome and still record a fresh stats_hash (gain#509).

The record reads (``get_all_records()``, ``get_records_in_region()``) refuse as well, but their exception type is NOT part of the contract: neither carries a not-open guard of its own, so measured on a closed table some backend/method pairs raise the same ValueError on their way through get_chromosomes() while the rest run into a pre-existing assert in the fetch path and raise a message-less AssertionError – or, under python -O, which strips asserts, whatever the next line makes of the released state (AttributeError on None, KeyError off an emptied contig dict). Those asserts are older than this contract and were left alone; do not catch on them, and do not read the uniformity of the first three as covering the record reads. For an out-of-tree caller all of it is the difference between an answer and an exception; the migration is to read inside the open table’s lifetime, or to reopen – open() re-establishes all of it, and a reopened table answers exactly as one that was never closed. Nothing in-tree was affected, which is why the ledger entry is the whole mitigation and there is no shim: every in-repo read sits behind GenomicScore.is_open(), and gpf has no non-test caller of get_chromosomes()/get_file_chromosomes() at all.

#358 then made that contract UNIFORM rather than changing it again, in the two places the backends disagreed. InmemoryGenomicPositionTable answered a closed get_file_chromosomes() with [] – the scanned-contig list its close() empties – and the base class’s memo cached that empty answer for the rest of the table’s life; BigWigTable refused with a bare assert, which python -O strips, leaving it answering [] from an emptied contig dict. Both now raise the ValueError the tabix and VCF backends already raised, so a caller catches one thing from all four (and a closed VCF table refuses for a never-opened one’s reason too: its guard is the absent handle, which covers both states). get_chromosome_length was brought into line the same way in the two backends that were not: the in-memory one used to raise out of the middle of a message it could not finish building – every contig takes the no-records branch on a closed table, and that branch interpolates get_chromosomes() – and the bigWig one guarded with the same bare assert, which under python -O let a closed table fall through into that very branch. Both now say the table is not open, as tabix and VCF already did.

Deliberately NOT changed: ``map_chromosome``/``unmap_chromosome`` pass through on a closed table. Both return their argument unchanged when the chromosome map is None, and close() sets it to None – so a closed mapped table hands reference-space names back as if they were the file’s, which is the one closed-table read that answers instead of refusing. It stays that way because the state cannot tell the two cases apart: _build_chrom_mapping sets chrom_map = None on an OPEN table that configures no mapping, so the field does not distinguish closed from mapping-free, and a table carries no open/closed flag. Introducing one would put a new invariant on every backend and a new failure on the read path, which is exactly what #350 avoided. Named here so the ambiguity is a decision on record rather than something a caller has to rediscover; the same note is on GenomicPositionTable.close.

Changed extension point: ``close()`` is no longer abstract, and a backend’s ``close()`` must now CALL UP into the base one. TabixGenomicPositionTable, BigWigTable and VCFGenomicPositionTable are in __all__ below, so this changes public names of gain and is recorded for the same reason as everything above. #354 gave the release policy of #350 a shared implementation: GenomicPositionTable.close() became CONCRETE and gives up the base class’s own file-derived state – chrom_order, chrom_map, rev_chrom_map and the get_file_chromosomes memo. The abstract set is now exactly open, get_all_records, get_records_in_region, find_chromosome_length and _load_file_chromosomes.

For an out-of-tree backend this breaks in the OPPOSITE shape to the two renames above: nothing fails at instantiation any more. A backend that already defines close() keeps working and simply acquires an obligation – end it with super().close(), or the table holds its contig order and its chromosome map for as long as its holder lives, which no exception reports and which open() would have rebuilt anyway. A backend that defines NO close() is the sharper case: it used to be impossible to construct (TypeError: Can't instantiate abstract class ... close) and now constructs fine, inheriting a close() that releases the chromosome state and never touches the backend’s own file handle – so GenomicScore.close() reports success over a live pysam/pyBigWig descriptor, which is the fd leak the whole #345/#350 line of work exists to prevent.

Recorded rather than fixed, and deliberately NOT reverted to an abstract close() over a separate _release_file_state() hook (gain#359). That would refuse, at the first INSTANTIATION, exactly the out-of-tree backends that define no close() – a backend that already defines one keeps constructing fine – to buy coverage the in-tree tests already give, at the price of changing a documented extension point a second time. What is enforced instead is that the tree’s own backends cannot slip past those tests – test_every_backend_in_the_tree_is_in_the_backend_list sweeps GenomicPositionTable.__subclasses__() and fails a concrete backend that no fixture builds, and test_a_closed_table_releases_what_open_established then catches both failure modes above on any backend it is handed. There is no such sweep for a backend outside this repo, which is what this entry is for: call up.

#361 added the header to what that base close() releases, under the default header_mode: file only. An out-of-tree backend that sets header at construction and consults it in open() – rather than reading it off the file there, as the tabular backends do – reopens with None unless it takes header_mode: list.

New optional capability: ``get_region_value_arrays`` and the ``supports_value_arrays`` flag that declares it (gain#398). BigWigTable, TabixGenomicPositionTable and VCFGenomicPositionTable are in __all__ below, so this adds public surface to gain and is recorded for the same reason as everything above.

get_region_value_arrays(chrom, pos_begin, pos_end, value_columns, batch_size) reads a region as batches of column arrays – (pos_begin, pos_end, {column index: raw cells}) – without building a Record per row. It is a fast path for a full sequential scan, and it is OPTIONAL: the base class refuses with TypeError, and a backend that serves it overrides the method and sets supports_value_arrays = True. Cells come back unparsed and rows are not clipped to the region; both stay with the caller, as on the record path. batch_size is a hint – BigWigTable ignores it, its batches being sized by its own adaptive fetch window.

Ask the flag; do not test the class. The capability is NOT derivable from the class hierarchy – VCFGenomicPositionTable subclasses TabixGenomicPositionTable, inherits its implementation, and sets supports_value_arrays back to False. An out-of-tree caller reaching for the method must consult the flag (or GenomicScore.supports_region_value_arrays(scores), which folds this flag together with the value types its own parse requires, and is answerable on an unopened score). Probing by calling and catching does NOT work: an unguarded call on a VCF table reaches the inherited tabix implementation and trips its assert isinstance(self.pysam_file, pysam.TabixFile), yielding a message-less AssertionError – and nothing at all under python -O.

Why the capability is declared rather than inferred, why VCF cannot honour the contract it inherits, and why this read path exists at all: see docs/adr/0001-bulk-read-path-for-statistics.md.

Changed payload: a VCF record’s PAYLOAD is now a FOUR-element tuple. VCFGenomicPositionTable is in __all__ below, so this changes public surface of gain and is recorded for the same reason as everything above. It was (variant, allele index); it is now (variant, allele index, info, info_meta), the last two being the pysam proxies an INFO lookup needs – variant.info and variant.header.info.

They are there because pysam allocates a FRESH proxy on every access (v.info is v.info is False, ~85ns each), so a reader that re-derived them per score paid ~170ns per score per record: measured, a 20-score read of a 3000-row VCF went from 8.50 to 10.76us/line. They used to be memoised on the per-line VCFScoreLine, resolved on its first score read. With the score lines removed, reading a value is a pure function of the record (_extract_vcf_value), so the memo had to move into the record – which is what a backend-defined payload is for.

The trade is that resolution is EAGER: a record whose scores are never read pays the ~170ns anyway. That case is narrow (AlleleScore fetches every record at a position and reads only the ref/alt match, so a 4-allele position wastes ~0.5us) and it buys a value read that needs no state, and so no per-line object to hold it. An out-of-tree reader that unpacked the payload as a pair now gets a ValueError; unpack four, or index by the VARIANT / ALLELE_INDEX / INFO / INFO_META constants in table_vcf.

Changed payload: a bigWig record’s PAYLOAD is now the VALUE ITSELF. BigWigTable is in __all__ below, so this changes public surface of gain and is recorded for the same reason as everything above. It was the four-tuple (chrom, pos_begin, pos_end, value) – three fields the record already carries in its decoded slots, repeated purely so the single value was addressable at payload[3]; it is now the bare float. An out-of-tree reader that indexed the payload gets a TypeError (‘float’ object is not subscriptable); read record[PAYLOAD], which IS the value, or go through GenomicScore.get_score_value_from_record.

This entry used to say the shape was deliberately NOT changed. It is kept, inverted, rather than deleted, because the three reasons it gave were real and someone will meet them again; each is answered below. What dissolved them is that the alternative to preserving a shape is not “break the deployed configs” – it is to keep the config and drop the shape.

(a) “The shape is config surface: every deployed bigWig says ``index: 3``.” It is config surface, and 16 deployed resources do say it (one with the comment # this makes no sense and should be removed already in its yaml). But the key is answered by ACCEPTING it as a deprecated no-op, not by keeping a payload shape for it to index into: bigwig_scores takes index: 3 at open, reports it once naming the resource, and resolves it – like the canonical config that addresses nothing at all – to the one column a bigWig has. No GRR has to change on the day this ships. Any OTHER index is now refused at open, by name; before, index: 2 read the position and called it a score.

That report is at DEBUG, not WARNING, and deliberately: all 150 deployed bigWig resources carry index: 3, so anything louder fires for every one of them on every open, which is noise wearing a severity label. The cost is that the message does not by itself drive the key out of the GRRs – that is a deliberate cleanup pass now, not something a log level nags into happening. _warn_inert_bigwig_keys splits the same way and says why: endemic keys (chrom/pos_begin/pos_end/header) report at DEBUG, the retired buffering knobs nobody sets stay at INFO, and keys nobody sets that DO mean something on another backend (zero_based, header_mode) stay at WARNING, because a message that fires for nearly every resource trains its reader to ignore the level.

(b) “``get_region_value_arrays`` reconstructs the four-tuple so a bad index raises the same ``IndexError`` the record path raises.” Superseded. That reconstruction existed to reproduce a failure; the failure is now prevented. The record path indexes nothing at all, and a bad index is refused when the SCORE is opened, in a message naming both the resource and the score – which is strictly better than an IndexError from inside a scan, and it fires before any file is opened rather than mid-repair. The bulk read keeps a backstop of its own for a caller that reaches the table directly: it serves column 0 and refuses everything else with a KeyError naming the resource. The bug that motivated the reconstruction – a misconfigured index served the chromosome string, turning an aborted repair into a silently all-zero histogram – is unreachable from either direction.

(c) “It buys nothing measurable.” That measurement was taken WITHOUT the parse removal, so it never argued against this change. It compared payload widths while the read still went through parse_value; narrowing the tuple alone saves an index, which is indeed noise. What the narrowing enables is the removal of the parse: a bigWig value arrives from pyBigWig as a float, the score is type: float (anything else is now refused), and the NA default for a bigWig score is empty (bigwig_scores), so parse_value on that pair is provably the identity – and the read becomes return record[PAYLOAD]. That is a real per-record saving, and it is only available once the payload is the value.

Renamed attribute: ``BigWigTable.direct_fetch_size`` -> ``fetch_size``, and the config key with it. BigWigTable is exported, so this is public surface. It was “direct” only in contrast to a second, buffered fetch strategy, and that strategy is gone: the table keeps no interval buffer across calls, and use_buffered_threshold no longer routes anything. The rename is not aliased – the capability survives, so a config naming it the old way means something specific, and failing validation lets an operator rename it rather than silently receive the default.

The two retired knobs, buffer_fetch_size and use_buffered_threshold, are the opposite case and are handled the opposite way: they configured a feature that no longer exists, so there is nothing to rename them to and refusing would take a resource offline to report a key that changes nothing. They stay accepted by the schema and are warned about (see utils._warn_inert_bigwig_keys).

The private buffer machinery went with them – _buffer, _buffer_region, _fill, _find, _fetch_buffered, _last_pos – and _fetch_direct, the surviving strategy, was first renamed _fetch and has since been inlined into get_records_in_region (gain#823, to drop a per-record tuple and a generator level). None of those were exported; they are named here only because the two invariants they carried were load-bearing enough to have their own regression tests, and both are now unreachable rather than maintained: a reopened table cannot serve a previous open’s values, and a fetch cannot resume from retained state after close().

Why – the measurements, the one workload where buffering still wins, and what would have to be true to reinstate it: see docs/adr/0002-remove-bigwig-fetch-buffering.md. Bringing it back for high-latency (http/s3) repositories is tracked as gain#449.

Changed signature: ``get_records_in_region(chrom)`` is now REQUIRED and non-optional. It was chrom: str | None = None, and None meant “every record in the table” – each of the three record backends opened the method with if chrom is None: yield from self.get_all_records(). An out-of-tree caller writing table.get_records_in_region() now gets a TypeError; call get_all_records() instead, which is what those three lines did and what the name says.

Two things made the old shape worth giving up. The default argument list was itself a legal call, so get_records_in_region() – easy to write by accident, and written by three tests in this repo – quietly scanned a whole genome. And the delegation was stated three times, once per backend, for a method that then had two jobs whose only shared code was the delegation itself. get_region_value_arrays had already settled on chrom: str, so the two region reads now agree.

The whole-table mode is not gone, only moved. It is live – grr_manage --region-size 0 computes statistics in a single pass – but it is expressed by ITERATING contigs, in scan.do_noregion_histograms, rather than by handing a null contig down. No member of the region-read family takes chrom=None: GenomicScore.fetch_records and fetch_region_segment_scores both require a contig, and a caller that wants every record of a table asks the table (get_all_records()).

``GenomicScore.fetch_region`` is gone; use ``fetch_region_segment_scores``. (The replacement was named fetch_region_values when this entry was written; the rename is its own entry below, and gain#844 has since removed both names in favour of fetch_region_segments_scores – see the end of this ledger.) There were two of them and they meant opposite things: on PositionScore fetch_region was a pure alias of fetch_region_values, while on AlleleScore it was the real read and fetch_region_values the adapter over it – yielding (pos, ref, alt, values) where every other kind yields (begin, end, values). fetch_region_values is now the single region read, uniform across all three kinds. Its REF/ALT were what the allele variant added, and nothing consumed them: a caller that needs the nucleotides reads record[REF] / record[ALT] off fetch_records, which is what AlleleScoreAnnotator and AlleleScore._fetch_allele_record already did.

New export ``ContigExtent`` and a new abstract method ``find_chromosome_length``; ``get_chromosome_length`` is no longer abstract. Recorded here because __all__ is this package’s public surface and the three backends in it inherit the method, which makes it a public name of gain (the same reason the closed-table note below is recorded).

For CALLERS this is purely additive. get_chromosome_length keeps its signature and its contract – still int or ValueError – and is now CONCRETE on GenomicPositionTable, a thin raising view of the new hook. No existing call site changes.

For an out-of-tree BACKEND it is a break, in the same shape as the ``get_file_chromosomes`` rename above: the class no longer instantiates. find_chromosome_length is an @abc.abstractmethod, so a backend that implements only get_chromosome_length – which used to be the abstract one – now fails at construction with TypeError: Can't instantiate abstract class ... without an implementation for abstract method 'find_chromosome_length'. It is a loud failure at the first instantiation rather than a silent behaviour change, which is why the hook was left abstract: the alternative, a base implementation delegating to get_chromosome_length, turns a backend that overrides neither into infinite recursion at call time. No such backend exists anywhere in the stack; all four in-tree ones were migrated with the change.

A backend migrates by renaming its get_chromosome_length to find_chromosome_length, widening the return type to int | ContigExtent, and returning a member instead of raising where it has no number. Overriding get_chromosome_length as well is allowed but pointless, and lets the two drift.

The enum is exported because a caller cannot interpret the new method’s answer without it, and its two members are deliberately NOT interchangeable: EMPTY means the backend PROVED the contig holds no records, UNDETERMINED means no length could be established and the contig may hold records anyway. Which one a backend can return is a property of the backend – the in-memory one holds the whole file and returns only EMPTY, the tabix and VCF ones read an index that carries only non-empty contigs and return only UNDETERMINED, and bigWig reads exact sizes from a header and returns neither. A caller that splits contigs into regions must treat the two oppositely: skip a proven-empty contig, read an undetermined one whole. Collapsing them – into None, or into one except ValueError – is what this shape exists to prevent, because it silently drops records from a scan whose stats_hash then claims the resource was scanned (gain#509).

ValueError keeps its old meaning and gains a sharper edge: it now marks the QUESTION as bad rather than the answer as absent – a table that is not open, or a contig outside get_chromosomes(). Note this splits one case the in-memory backend used to conflate: an unknown contig still raises, while a contig the table lists and has no rows for now answers EMPTY from the hook (and still raises from get_chromosome_length, with the same message as before).

Two ``get_chromosome_length`` messages did change, both on the tabix and VCF backends, because the raising view is now written once on the base class instead of per backend. A contig whose length the probe cannot determine used to raise Could not find contig '<file contig>' and now raises could not determine the length of contig <contig> in the table's contigs: [...]; a contig that will not unmap used to raise error in mapping chromsome ... and now reaches the same message, having become UNDETERMINED. Both name the contig in REFERENCE space where the old text named the file contig, so the one thing the old diagnostic carried that the new one does not is which file contig the probe was looking for – recoverable from the table’s chrom_mapping. The type is unchanged, so an except ValueError is unaffected; only a caller matching on the text is.

Opening a tabix table can now REFUSE the resource (gain#553). TabixGenomicPositionTable.open() reads the coordinate columns off the index it opened the file with and compares them against the column keys the table resolves; where they disagree it raises MalformedResourceError, before a record is read. Recorded here because every caller of a tabix-backed score inherits the new failure – and because the exception is a ValueError, so a caller already catching one around open() keeps catching this, and grr_manage reports it as the resource’s fault rather than as an internal error (ADR 0008).

A table configured over the columns its index was built from opens exactly as before, at the cost of one fixed-size header read per open. The refusal is uniform: it is not a mode, and there is no flag to turn it off – a resource whose index filters on one span while its records are read through another returns records that are fetched and then dropped without a trace, and no caller has any use for that.

``GenomicScore.fetch_region_values`` is renamed ``fetch_region_segment_scores``; the old name survives as a deprecated alias (gain#729). The method yields one tuple per underlying RECORD – a segment, with that record’s own clipped (begin, end) – not one value per position, and “values” is exactly what made callers read it as the latter. Living on the shared base, the rename covers PositionScore, AlleleScore and FragmentScore at once; region_values_from_records keeps its name, the statistics scan composing through it unchanged (ADR 0008).

Unlike fetch_region above, the old name is NOT gone yet: no in-tree and no known cross-repo caller used it, but the published docs/source/python_interface.rst showed it to external readers, so it stays as a thin forwarder raising DeprecationWarning until gain#730 – a dated removal, per the precedent of gain#343 – deletes it. (gain#844 did, ahead of that date; see the end of this ledger.)

The return type narrowed with the rename: the values slot was list[ScoreValue] | None and no code path could yield None there – both producers build the values slot as a list comprehension over _extract_value (through get_score_values_from_record until gain#823 hoisted the extractor out of the loop) – so the new name promises list[ScoreValue] and the narrowing runs down the private chain (region_values_from_records, _clipped_score_values, _score_segments). A caller’s None guard on a yielded values slot is dead code now, as aggregate_region’s was. (fetch_region_weighted_values carried the narrowing too from gain#734 until gain#1131 retired it.)

``PositionScore.fetch_position_scores`` is gone; use ``get_scores_at_position`` (gain#1268). It was the point read that kept a real | None – the one narrowing the paragraph above excepted, meaning “no record covers this position”. The logical plane spells that as a tuple of None, one per score id asked, because on the plane an uncovered position is a value and not an absence (#727), so the | None leaves with the method rather than being narrowed away.

Removed outright, with no forwarder and no DeprecationWarning: unlike fetch_region_values above, this name was never on docs/source/python_interface.rst, and no caller outside gain uses it – the gain#1131 situation, not the gain#730 one.

What a migrating caller has to know. The refusals are NOT new: a closed score, an unknown contig and an unknown score id were all refused by the old read too (its first statement went through get_all_chromosomes, which raises is not open). Three things do change, all of them where the old read answered something it should not have:

• a position below 1 is REFUSED (_guard_region_span). The old read passed the bound to fetch_records, and a backend that tests its bounds for truthiness read 0 as “unbounded” – so position 0 answered with the contig’s FIRST record. That leniency not reaching a reader is what the guard is for (#727).

• a record that does not actually COVER the position no longer answers. The old read took records[0] whatever it was; the plane clips, so a backend handing back a record outside the queried region (a table whose index and pos_end name different columns, gain#553) now reads as uncovered.

• a record whose end precedes its begin is refused rather than answered.

And the shape: a tuple of one value per id asked, with None per uncovered id, where the old read answered None for the whole position.

New method, and a new obligation on backend authors: ``buffered_record_count()`` (gain#1120). BigWigTable, TabixGenomicPositionTable and VCFGenomicPositionTable are in __all__ below, so this adds public surface to gain and is recorded for the same reason as everything above.

It answers how many records a table is holding from PREVIOUS reads – not its contents, and not the last region’s size. The base returns 0, which is the honest answer for a backend that carries nothing across queries: an in-memory table holds every record and buffers none of them. Only the tabix backend (and the VCF one that subclasses it) has a non-zero answer.

The obligation is the reason the method exists. A caller may stop iterating a region read at any point, so a backend that carries state between queries must release it from a finally – the code after a yield loop is never reached by a generator that is closed part-way. What it retains may not grow with the number of abandoned reads, and the reads that follow must answer as though none had been abandoned.

Releasing from a finally also hands the caller the timing: a held generator can be closed after other queries have moved the table on, so a backend whose release depends on query order must check that it still is the current reader (TabixGenomicPositionTable._prune_if_current). A table serves ONE live region read at a time – a held generator may be closed across another query, not resumed.

test_backend_record_contract.py holds every in-tree backend to both halves. There is no such sweep for a backend outside this repo, which is what this entry is for: a backend with cross-query state overrides the method and releases from a finally.

Changed aliasing: the tabix family’s ``get_chromosomes()`` returns the SAME list on every call (gain#1173). TabixGenomicPositionTable and VCFGenomicPositionTable are in __all__ below, so this changes public surface of gain and is recorded for the same reason as everything above.

The mapped, filtered contig list is derived once per open and held, because at the time every contig-membership check on the annotation path reached it through GenomicScore.get_all_chromosomes(): one annotated substitution made three such calls, each rebuilding the list, at a cost that grew with the file’s contig count – per CALL, 15.7us at hg38 primary-assembly contig counts and 48.8us with the alts, against 0.40us at a single contig. (Those screens have since moved to has_chromosome; see its entry below.) Reads are now a flat attribute fetch at any contig count, paid for by one pass inside every open() – including an open that never asks for contigs, which the memo this replaced left free. Measured against the open it rides inside, that pass is 0.08% of it at one contig, 2.7% at hg38 primary-assembly counts and 4.6% with the alts.

The observable change is aliasing, not content: the list is equal to what it always was, but a caller that MUTATES it in place – sort(), reverse(), append() – corrupts every later read for the life of the open table instead of scribbling on a throwaway. Copy before mutating. This is what the base class has always done (get_chromosomes() returns the stored chrom_order itself), so bigWig and in-memory callers were already living under this rule. Nothing in-tree was affected – every in-repo call site is an in, a for, a list()/set() or a splat – and gpf has no non-test caller of get_chromosomes() at all, which is why this entry is the whole mitigation and there is no defensive copy: returning a copy per call would give back most of what deriving once per open buys.

New method: ``has_chromosome(chrom)`` (gain#1304). BigWigTable, TabixGenomicPositionTable and VCFGenomicPositionTable are in __all__ below, so this adds public surface to gain and is recorded for the same reason as everything above. Concrete on the base, so every backend answers it and no caller has to know which one it holds.

It answers whether the table carries a contig, and nothing else – no extent, no length, no ordering. What it replaces is chrom not in table.get_chromosomes(), which was the shape of every membership screen in the read path: a walk of the ordered list, costing more the further down the list the contig sat, and most of all for a contig the table does NOT carry, since that walks all of it before it can say no. Measured on a tabix table at 640 contigs, one screen ran 4.3 us on a tail alt and 2.9 us on an absent contig, against 0.135 us for the predicate and a 5.3 us point read on the same table – and a miss and a tail alt contig, the two cases the scan is worst at, are exactly the two a screen exists for. The predicate is a set lookup: flat in the contig count and in the contig’s index. (GenomicPositionTable. has_chromosome carries the full measurement and the per-record totals.)

The set is derived FROM get_chromosomes(), once per open, and released at the two seams that release the get_file_chromosomes memo – close() and _build_chrom_mapping(). Deriving it from the accessor rather than beside it is what makes the two unable to disagree; a predicate that answered differently from the list would be this method’s quietest possible failure, since a screen answering “no” for a contig the table has makes the read above it report no data on a contig full of it, with nothing raised. A closed table refuses the predicate in whatever words that backend’s get_chromosomes() uses, for the same reason.

get_chromosomes() is unchanged – same signature, same order, same per-open aliasing – and remains the read for a caller that needs the order or the whole collection. What changed is that the annotation path no longer calls it: GenomicScore.get_all_chromosomes() is still public and still delegates here, but every in-tree screen (both annotators, the score’s shared region-read refusal, the position kind’s absent-contig branch, both allele reads, and the tabix, in-memory and bigWig backends’ own record-read and length screens) now asks has_chromosome instead. A backend outside this repo inherits the base implementation and needs no change; one that overrides get_chromosomes() gets a matching predicate for free, because the base derives from the override. Since gain#1303 the list IS chrom_order, derived into it by the tabix _build_chrom_mapping and read back by the single base-class get_chromosomes(); the override and the private list it answered from are gone. The rule above is therefore no longer a tabix-specific caveat but the one the whole hierarchy shares.

Changed refusal message: a closed or never-opened tabix or VCF table now says “genomic table not open” (gain#1303), where it used to say “tabix table not open” / “vcf table not open”. The read refuses because chrom_order was released – or never built – rather than by checking a handle of its own, so the raise is the base class’s. The exception TYPE is unchanged, and ValueError is what this package contracts for the four closed-table reads listed on GenomicPositionTable.close; the text never was. Recorded because a caller matching on the message – a log grep, a test – sees it change even though nothing it can catch does. get_file_chromosomes(), find_chromosome_length() and get_chromosome_length() are untouched and still refuse in each backend’s own words. has_chromosome follows get_chromosomes() here as it does everywhere – it is derived from it, so on these two backends it now refuses in the base class’s words too.

``GenomicScore.fetch_region_segment_scores`` and its alias ``fetch_region_values`` are gone; use ``fetch_region_segments_scores`` (gain#844, closing the gain#730 removal too). Both were deprecated forwarders since gain#827 – the first was exactly clip_to_region(fetch_region_segments_scores(...), pos_begin, pos_end), and the second forwarded to the first – and both raised DeprecationWarning on every call. The one stated reason to keep the names, the getting-started guide teaching them (gain#1383), went with this change: docs/source/python_interface.rst now reads through fetch_region_segments_scores and clips in the loop.

What a migrating caller has to know: fetch_region_segments_scores reports each record at its OWN extent, so a record straddling the window’s edge comes back whole. A caller that held the clipped spans composes genomic_scores.records.clip_to_region over the stream, or clips per position in its own loop; a caller that only ever unpacked (begin, end, values) over a window wider than the table sees no difference. An allele score never clipped – its read answers a point at the record’s position either way – so the override that only differed in its warning text is gone with the base method. There is no forwarder and no warning, the gain#1268 pattern: the names are removed outright, and a call to either is an AttributeError. ``GenomicScore.fetch_region_segments`` is renamed ``fetch_region_segments_scores`` (gain#1397), on all three kinds and in the same change as the removal above, with no deprecated alias: the two names would have differed from the removed fetch_region_segment_scores by one letter each, and a forwarder under the old name would have kept that confusion live. Signature, return type and meaning are unchanged – one (begin, end, values) tuple per record, at the record’s own extent – so a caller migrates by renaming the call. A call to fetch_region_segments is an AttributeError.

New export ``ChromLengthSource``, a new obligation on backend authors, and a new method ``ContigExtent.refusal(chrom, contigs)`` (gain#1413).

ChromLengthSource names what a length IS – a header’s exact size, an index probe’s upper bound, the rows’ extent, or (the one member no table produces) a reference genome’s – and carries is_exact. It is exported because the score layer’s chromosome-length resolver (genomic_scores.chrom_lengths) answers with it, and lives here rather than in the score layer because three of its four members are facts about a FORMAT: each backend now declares its own as the class attribute chrom_length_source, the way it declares yields_records. The obligation: the base class gives it NO default, so a backend that has not said is refused with an AttributeError the first time a length’s provenance is asked, rather than inheriting a label – and a trust level – that is not its own. The trust level is the member’s is_exact; the exactness bool that used to sit beside the declaration (#776) had one reader – the coverage denominator – and that reader now asks the implementation’s ladder, whose records carry the member (gain#1414).

refusal is the one home of the two “no length” messages: get_chromosome_length raised them inline, and the score’s method refuses the same two facts and must say them the same way.