What an annotator receives

An annotatable is the thing a pipeline annotates: one position, one region or one allele, on one chromosome. It is the first argument of every annotate call, and the only thing an annotator knows about the input row apart from the context the earlier annotators built.

Annotators are typed on the base class, Annotatable, rather than on a variant, because a pipeline does not only annotate variants. annotate_tabular builds a Position from a chromosome and a position column, a Region from an added end column, a VCFAllele when there are reference and alternative columns, and a CNVAllele for a large deletion or duplication. All four reach an annotator through the same interface, and an annotator that only makes sense for some of them checks the annotatable’s type — an Type — and answers the empty result for the rest.

Creating an annotatable

Each of the four classes has a constructor that takes the chromosome and the coordinates directly:

from gain.annotation.annotatable import (
    Annotatable,
    CNVAllele,
    Position,
    Region,
    VCFAllele,
)

position = Position("chr1", 11796321)
region = Region("chr1", 11796321, 11796400)
allele = VCFAllele("chr1", 11796321, "G", "A")
deletion = CNVAllele(
    "chr1", 11796321, 11896400, Annotatable.Type.LARGE_DELETION)

A CNVAllele takes its type as the fourth argument, and accepts only LARGE_DELETION and LARGE_DUPLICATION. The other three classes set their type themselves.

A tabular input row becomes an annotatable through gain.annotation.record_to_annotatable, the module that annotate_tabular uses. build_record_to_annotatable() takes the column names a source has, and returns a converter for the first column set it finds, in this order:

  • chrom, pos_beg, pos_end and cnv_type — a CNVAllele;

  • chrom, pos_beg and pos_end — a Region;

  • chrom, pos, ref and alt — a VCFAllele;

  • vcf_like — a VCFAllele from one chrom:pos:ref:alt column;

  • chrom, pos and variant, or location and variant — a VCFAllele from a DAE or CSHL variant description, read against the reference genome passed as ref_genome (a CSHL copy-number description gives a CNVAllele);

  • chrom and pos — a Position.

Its full signature is build_record_to_annotatable(renamed_columns, available_columns, ref_genome=None). The first argument, a mapping, renames the columns: the key col_<column> maps one of the names above to the column that holds it, and the value "-" excludes that column. The second argument is the set of column names the source has. The converter’s build method takes only the row, as a mapping from column name to value, and turns it into an annotatable. Here the CHROM column stands in for chrom:

from gain.annotation.record_to_annotatable import (
    build_annotatable_from_dict,
    build_record_to_annotatable,
)

converter = build_record_to_annotatable(
    {"col_chrom": "CHROM"}, {"CHROM", "pos"})
position = converter.build({"CHROM": "chr1", "pos": "5"})

build_annotatable_from_dict() does both steps for one dictionary, with no renaming:

allele = build_annotatable_from_dict(
    {"chrom": "chr1", "pos": "11796321", "ref": "G", "alt": "A"})

Both raise MalformedRecordError, a ValueError, when no column set matches or when a row has a field that cannot be read.

An annotatable holds nothing to open or close, so this page has no lifecycle section.

The interval

Every annotatable spans a closed, 1-based interval [pos, pos_end], both ends inclusive: a position has pos_end == pos, and len(annotatable) is pos_end - pos + 1. An annotator that queries a score or gene models queries exactly this span. The canonical spellings of the attributes are chrom, pos and pos_end, with chromosome, position and end_position as aliases.

to_dict writes a different set of keys for each class, and every class writes type, the name of its Type:

  • Position: type, chrom and pos.

  • Region and CNVAllele: type, chrom, pos_begin and pos_end.

  • VCFAllele: type, chrom, pos, ref and alt. It does not write pos_end, because the alleles determine it.

A VCFAllele derives both its Type and its interval from the two alleles, and the docstring below states the rule. The one thing to carry away is that for a deletion or a complex allele pos_end reaches one base past the last reference base.

Serialised form

An annotatable round-trips through a string of the form TYPE(args...): repr writes it and Annotatable.from_string reads it back, dispatching on the type token to the right subclass. That is the form an annotator writes when it hands its input to an external tool one line at a time, and the form such a tool parses on the other side; demo_annotator does exactly this in both directions.

A None annotatable

An input row may have no annotatable at all — an unparsable variant, a liftover that found nothing. The pipeline passes None through, and the answer for it is every attribute set to None, never an exception. AnnotatorBase handles the None before it reaches the subclass, so an annotator built on it never sees one in _do_annotate; a batch-only annotator that overrides _do_batch_annotate is responsible for its own None entries.

API

class Annotatable(chrom: str, pos: int, pos_end: int, annotatable_type: Type)[source]

Base class for annotatables used in annotation pipeline.

An annotatable is the thing a pipeline annotates: a position, a region or an allele on one chromosome. Every annotatable spans a closed, 1-based interval [pos, pos_end] – both ends inclusive, so a Position has pos_end == pos and len(annotatable) is pos_end - pos + 1. VCFAllele says how it derives pos_end from its alleles.

The canonical spellings of the attributes are chrom, pos and pos_end. chromosome, position and end_position are aliases kept for the callers that use them; each reads the same value as its twin. Equality compares the type, the chromosome and both ends.

to_dict() writes type and a set of keys that depends on the class: chrom and pos for a Position; chrom, pos_begin and pos_end for a Region and a CNVAllele; chrom, pos, ref and alt for a VCFAllele.

class Type(*values)[source]

Defines annotatable types.

static from_string(variant: str) → Type[source]

Construct annotatable type from string argument.

property chrom: str

The chromosome name, as given at construction.

property chromosome: str

Alias of chrom.

property end_position: int

Alias of pos_end.

static from_string(value: str) → Annotatable[source]

Deserialize an Annotatable instance from a string value.

property pos: int

The 1-based start of the interval, inclusive.

property pos_end: int

The 1-based end of the interval, inclusive.

Equal to pos for a single position; see the class docstring for the convention and VCFAllele for how an allele’s end is derived.

property position: int

Alias of pos.

abstractmethod to_dict() → dict[source]

Serialize the annotatable to a dictionary.

static tokenize(value: str) → tuple[str, list[str]][source]

Split the serialized form TYPE(arg1, arg2, ...) into its parts.

Returns the type token and the list of argument tokens, with whitespace stripped from the arguments. Raises ValueError for a value that is not exactly one call-like expression. The inverse of __repr__; from_string() dispatches on the type token and the concrete classes parse the arguments.

class Position(chrom: str, pos: int)[source]

Annotatable class representing a single position in a chromosome.

static from_string(value: str) → Position[source]

Deserialize an Annotatable instance from a string value.

to_dict() → dict[source]

Serialize the annotatable to a dictionary.

class Region(chrom: str, pos_begin: int, pos_end: int)[source]

Annotatable class representing a region in a chromosome.

static from_string(value: str) → Region[source]

Deserialize an Annotatable instance from a string value.

to_dict() → dict[source]

Serialize the annotatable to a dictionary.

class VCFAllele(chrom: str, pos: int, ref: str, alt: str)[source]

A small variant in VCF terms: chrom, pos, ref and alt.

The alleles decide both the Annotatable.Type and the interval:

  • one base to one base is a SUBSTITUTION, spanning pos alone;

  • a one-base reference that the alternative extends (same first base) is a SMALL_INSERTION, spanning pos to pos + 1 – the two bases the insertion falls between;

  • a reference longer than one base collapsed to its first base is a SMALL_DELETION, and any other pair is COMPLEX; both span pos to pos + len(ref).

So for a deletion or a complex allele pos_end reaches one base past the last reference base, which sits at pos + len(ref) - 1. Annotators query exactly this span.

The canonical spellings are ref and alt; reference and alternative are aliases. Equality also compares both alleles.

property alt: str

The alternative allele as written in VCF, anchor base included.

property alternative: str

Alias of alt.

static from_string(value: str) → VCFAllele[source]

Deserialize a VCFAllele from its __repr__ form.

Accepts VCFAllele(chrom, pos, ref, alt), and the same four arguments under any small-variant type name (SUBSTITUTION, SMALL_INSERTION, SMALL_DELETION, COMPLEX). Raises ValueError for another type token or argument count. The type is re-derived from the alleles, not taken from the token.

property ref: str

The reference allele as written in VCF, anchor base included.

property reference: str

Alias of ref.

to_dict() → dict[source]

Serialize to type, chrom, pos, ref and alt.

type is the type’s name. pos_end is not written: it is re-derived from the alleles.

class CNVAllele(chrom: str, pos_begin: int, pos_end: int, cnv_type: Type)[source]

Defines copy number variants annotatable.

static from_string(value: str) → CNVAllele[source]

Deserialize an Annotatable instance from a string value.

to_dict() → dict[source]

Serialize the annotatable to a dictionary.

build_record_to_annotatable(renamed_columns: dict[str, str], available_columns: set[str], ref_genome: ReferenceGenome | None = None) → RecordToAnnotable[source]

Transform a variant record into an annotatable.

Parameters:
  • renamed_columns (dict[str, str]) –

    Mapping from expected internal column identifiers (e.g. “col_<field>”) to the actual column names present in the input source. A column can be excluded from usage if an identifier is mapped to “-“. Example rename:

    "col_<field>": "<input source column name for the field>"
    

    Example exclude:

    "col_<field>": "-"
    

  • available_columns (set[str]) – The set of column names available in the input records.

  • ref_genome (ReferenceGenome | None, optional) – Optional reference genome context used for creating annotatables. Not all annotatables require it.

build_annotatable_from_dict(obj: Mapping[str, object], ref_genome: ReferenceGenome | None = None) → Annotatable[source]

Build an annotatable from a record held in a dictionary.

Raises MalformedRecordError when no converter matches the record’s keys or the matching converter refuses the record.