Adding a resource type

The resource types GAIn understands are not a fixed list. Each one is a resource implementation — a class that knows how to describe a resource of that type, compute its statistics, and render its HTML summary page — and implementations are discovered through a Python entry-point group. A package installed alongside GAIn can add a type without any change to GAIn itself.

This is the extension seam for a new kind of resource; adding a new annotator is a different seam, with its own entry-point group.

The interface

An implementation subclasses GenomicResourceImplementation, whose constructor takes the GenomicResource it wraps. Five members are abstract, and they divide into two jobs:

Describing the resource. get_info() and get_statistics_info() return the HTML for the resource’s summary and statistics pages. calc_info_hash() returns a digest of everything get_info depends on, so a page is only re-rendered when its inputs changed.

Computing statistics. create_statistics_build_tasks() returns the task-graph tasks that compute this resource’s statistics, and calc_statistics_hash() digests the inputs those tasks read — it is what lets grr_manage skip a resource whose statistics are already current.

The concrete files property should report every file the implementation reads. A caching or mirroring repository builds its fetch list from it, so a file left out is a file that may be missing from a cached copy. (Publishing works off the resource’s manifest instead, so it is unaffected.)

Registering

The entry-point group is gain.genomic_resources.implementations. Its keys are resource type names — the type field in a resource’s genomic_resource.yaml — and its values point at the builder for that type. A builder is any callable taking a GenomicResource and returning an implementation, so an implementation class whose constructor takes exactly that is its own builder.

GAIn’s own types are registered this way in core/pyproject.toml. The genome type is the smallest complete example:

[project.entry-points."gain.genomic_resources.implementations"]
genome = "gain.genomic_resources.implementations.reference_genome_impl:ReferenceGenomeImplementation"

A third-party package declares its own type the same way, in its own pyproject.toml:

[project.entry-points."gain.genomic_resources.implementations"]
my_resource_type = "my_package.my_impl:MyResourceImplementation"

from gain.genomic_resources import GenomicResource
from gain.genomic_resources.resource_implementation import (
    GenomicResourceImplementation,
)


class MyResourceImplementation(GenomicResourceImplementation):
    """Resource implementation for my_resource_type."""

    def __init__(self, resource: GenomicResource):
        super().__init__(resource)

    @property
    def files(self) -> set[str]:
        return {self.config["filename"]}

    def get_info(self, **kwargs):
        ...

    def get_statistics_info(self, **kwargs):
        ...

    def calc_info_hash(self) -> bytes:
        ...

    def calc_statistics_hash(self) -> bytes:
        ...

    def create_statistics_build_tasks(self, **kwargs):
        ...

Once the package is installed, grr_manage and the repository browser handle resources of that type with no further configuration.

Looking a builder up

get_resource_implementation_builder() is the lookup GAIn itself uses. It consults the in-process registry first and falls back to the entry points, loading and caching whatever it finds; it returns None for a type nothing has registered.

register_implementation() adds a builder to the in-process registry directly, without an entry point. That is the right tool for a test, or for a type defined in the same script that uses it; it is not a substitute for the entry point in a package meant to be installed, because it only takes effect once the registering module has been imported.

from gain.genomic_resources import (
    get_resource_implementation_builder,
    register_implementation,
)

register_implementation("my_resource_type", MyResourceImplementation)

builder = get_resource_implementation_builder("my_resource_type")
impl = builder(grr.get_resource("my/resource/id"))

A note on the shipped registrations: two keys may point at the same class. cnv_collection and fragment_score both map to FragmentScoreImplementation because the first is a deprecated spelling kept registered for repositories outside our control — opening one warns. Nothing prevents a third-party package from registering a type name GAIn already uses; the entry-point group is flat, so prefer a name qualified by your project.

API

class gain.genomic_resources.resource_implementation.GenomicResourceImplementation(genomic_resource: GenomicResource)

Base class used by resource implementations.

Resources are just a folder on a repository. Resource implementations are classes that know how to use the contents of the resource.

abstractmethod calc_info_hash() → bytes

Compute and return the info hash.

abstractmethod calc_statistics_hash() → bytes

Compute the statistics hash.

This hash is used to decide whether the resource statistics should be recomputed.

collect_index_info() → tuple[tuple[str, ...], tuple[str, ...]]

Collect resource info for FTS index building.

Returns a (header, row) pair where header contains field names and row contains the corresponding values for this resource. Label keys/values are appended after the fixed fields.

Raises ValueError if a label key cannot name an index field – every implementation reaches the index through here, and the index build reports a raise from here against this one resource (gain#464).

An override that contributes further fields must call super() and append to what it returns. This is the only place a label key is checked against the names the index reserves: the build’s own re-check sees the finished header, in which an implementation’s fields legitimately appear, so it cannot tell a field from a label (gain#542). A field added by an override belongs in GR_INDEX_NON_LABEL_COLUMNS.

abstractmethod create_statistics_build_tasks(**kwargs: Any) → list[TaskDesc]

Create tasks for calculating resource statistics for task graph.

property files: set[str]

Return a list of resource files the implementation utilises.

get_config() → dict

The resource’s configuration.

As read from the resource at construction; an implementation that validates its configuration replaces it with the validated form, and answers that here.

abstractmethod get_info(**kwargs: Any) → str

Construct the contents of the implementation’s HTML info page.

get_statistics() → ResourceStatistics | None

Try and load resource statistics.

abstractmethod get_statistics_info(**kwargs: Any) → str

Construct the contents of the implementation’s HTML statistics info page.

reload_statistics() → ResourceStatistics | None

Drop the cached statistics and reload via get_statistics().

For after the statistics were rebuilt on disk. Answers what get_statistics() answers: None unless the implementation overrides it.

property resource_id: str

The id of the resource this implementation wraps.

gain.genomic_resources.register_implementation(resource_type: str, builder: Callable[[GenomicResource], GenomicResourceImplementation]) → None

Register a resource type with a given builder function.

The builder has to be a builder function which takes a genomic resource and returns a ready to use implementation. The type is the type of resource to which this builder will be mapped. This is usually the “type” field in the resource’s config.

gain.genomic_resources.get_resource_implementation_builder(resource_type: str) → Callable[[GenomicResource], GenomicResourceImplementation] | None

Return an implementation builder for a certain resource type.

If the builder is not registered, then it will search for an entry point in the found implementations list. If an entry point is found, it will be loaded and registered and returned.