Groups and tracks

A run of binning_tool can take a long time, and a wrong grouping shows only in its file. GAIn provides functions that resolve an entry against a repository without a run, so a notebook sees the groups and the tracks of the entry first. Each function raises RunDefinitionError for an entry that binning_tool refuses, with the same message. The message names the entry as binners[0], and binning_tool names it by its position in the binners list. fragment_tracks takes a binner_index and names the entry as binners[<binner_index>].

Previewing the groups and tracks of an entry

cell_groups() gives the group that binning_tool puts each key of an entry in. The key of a cell grouping is the barcode of a cell, and the key of a value grouping is a value of its score. The result is a DataFrame with the columns resource_id, score_value and group. The result lists only the keys that have a group. A fragment with a key that no row lists reaches no track, so the cells that an entry drops are the barcodes of the fragments that the result does not list. An entry with a constant group gives no row.

fragment_tracks() gives the /tracks rows that binning_tool writes for an entry, one row for each track, in output order. The function reads the entry alone. When a sibling entry of the run makes a track with the same name, group and aggregator, binning_tool refuses the run, and fragment_tracks still gives the rows.

The example below builds the entry of Entries and previews it:

import gain.grr as grr
from gain.binning import (
    CellGrouping,
    FragmentEntry,
    MetaFilter,
    MetaSource,
    cell_groups,
    fragment_tracks,
)

repo = grr.build_genomic_resource_repository()
entry = FragmentEntry(
    "frags/*",
    group=CellGrouping(),
    meta=MetaSource(
        resource_id="meta/cells",
        filters=(MetaFilter("sample_id", label="sample_id"),),
    ),
)

groups = cell_groups(repo, entry)
print(groups)  # columns: resource_id, score_value, group

tracks = fragment_tracks(repo, entry)
print(tracks[["resource_ids", "selector"]])

Each function also takes the mapping under the fragment_score_binner key of a run definition in place of a FragmentEntry. The user page Binning infrastructure describes the keys of that mapping. A relative local meta file is read from base_dir, or from the current directory without it.

Previewing a whole run definition

resolve_binners() resolves the whole binners list of a run definition into its jobs, as binning_tool does before a run. A notebook calls it to check a run definition with position-score entries too, or to see the labels of the tracks of all its entries together. The list holds one mapping of a binner kind to its configuration for each entry, as the YAML of the run definition does.

tracks_frame() gives the /tracks rows of the jobs that resolve_binners returns, one row for each track, in the order of the file. The rows hold no name for a track. track_labels() builds the label of each row, as Labels of the tracks describes:

import gain.grr as grr
from gain.binning import resolve_binners, track_labels, tracks_frame

repo = grr.build_genomic_resource_repository()
binners = [
    {"position_score_binner": {"resource_query": "hg38/scores/phyloP*"}},
    {"fragment_score_binner": {"resource_query": "frags/*"}},
]

jobs = resolve_binners(binners, repo)
tracks = tracks_frame(jobs)
print(tracks[["binner_type", "resource_ids", "selector"]])
print(track_labels(tracks))

API

The names below are exported by the package gain.binning.

cell_groups(repo, entry, *[, base_dir])

Give the group that binning_tool puts each key of entry in.

fragment_tracks(repo, entry, *[, ...])

Give the /tracks rows that binning_tool writes for entry.

resolve_binners(binners, grr, *[, base_dir])

Resolve a binners: list against grr into its jobs.

tracks_frame(jobs)

Give the /tracks rows of jobs: one row per track, in order.

cell_groups(repo: GenomicResourceRepo, entry: FragmentEntry | dict[str, Any], *, base_dir: str | None = None) → DataFrame[source]

Give the group that binning_tool puts each key of entry in.

The key of a cell grouping is the barcode of a cell. The key of a value grouping is a value of its score. A fragment with a key that no row lists reaches no track. Each refusal names the entry as binners[0].

Parameters:
  • repo – The repository that the query of the entry searches.

  • entry – The entry, or the mapping under the fragment_score_binner key of an entry in a run definition.

  • base_dir – The directory that a relative local meta file is read from. None is the current directory.

Returns:

The columns of CELL_GROUPS_COLUMNS: one row for each key of each resource, in the order of the resources and then of the keys. An entry with a constant group gives no row.

Return type:

pandas.DataFrame

Raises:

RunDefinitionError – When binning_tool refuses the entry.

fragment_tracks(repo: GenomicResourceRepo, entry: FragmentEntry | dict[str, Any], *, binner_index: int = 0, base_dir: str | None = None) → DataFrame[source]

Give the /tracks rows that binning_tool writes for entry.

The function reads entry alone. A run names the tracks of all its entries together, but the /tracks rows hold no name, so a sibling entry does not change these rows. The function does not see the other entries of a run: when a sibling entry produces a track with the same name, group and aggregator, binning_tool refuses the run, and this function still gives the rows.

Parameters:
  • repo – The repository that the query of the entry searches.

  • entry – The entry, or the mapping under the fragment_score_binner key of an entry in a run definition.

  • binner_index – The position of the entry in the binners list. It is the binner_index of each row, and each refusal names the entry as binners[<binner_index>].

  • base_dir – The directory that a relative local meta file is read from. None is the current directory.

Returns:

The columns of the /tracks table of the file, one row for each track of the entry, in output order.

Return type:

pandas.DataFrame

Raises:

RunDefinitionError – When binning_tool refuses the entry.

resolve_binners(binners: Any, grr: GenomicResourceRepo, *, base_dir: str | None = None) → list[BinningJob][source]

Resolve a binners: list against grr into its jobs.

Each track of a job carries its binner_index, the position of its entry in binners, and its track_index, its position among the tracks of that entry.

Parameters:
  • binners – The binners: list of a run definition: one one-key mapping of binner kind to its configuration for each entry.

  • grr – The repository that the queries of the entries search.

  • base_dir – The directory that a relative path in an entry is read from. Without it, the kinds read such a path from the current directory.

Returns:

The jobs of the run, in the order of the entries, with their tracks named. tracks_frame() gives their /tracks rows.

Return type:

list of BinningJob

Raises:

RunDefinitionError – When binners is not a non-empty list, or an entry is refused. The message names the offending entry.

tracks_frame(jobs: Sequence[BinningJob]) → DataFrame[source]

Give the /tracks rows of jobs: one row per track, in order.

binner_type is the kind, binner_index the entry’s position in the binners list and track_index the track’s position in that entry. resource_ids is the comma-separated list of the resources a track is computed from, in resource-id order. A resource id never holds a comma. selector picks the track in its resources, and binner_def is the canonical JSON of the kind’s definition.

Parameters:

jobs – The resolved jobs, in the order of their tracks.

Returns:

The columns of TRACKS_COLUMNS. The two index columns are int64.

Return type:

pandas.DataFrame

Raises:

ValueError – For a definition with a NaN or an infinity.