Reading a run

A run of binning_tool writes one HDF5 file. The file holds the /values matrix, the /bins table with the coordinates of each row and the /tracks table with the provenance of each column. GAIn provides reader functions that load the three parts as numpy and pandas objects. The user page Binning infrastructure describes the layout of the file.

Reading the tables of a run

read_tracks() and read_bins() read /tracks and /bins as pandas DataFrames. Each function takes a path or an open h5py.File. The columns come from the file, the text fields are str, and the index is the position in /values.

read_values() reads /values as a numpy float64 matrix of shape (n_bins, n_tracks), not as a DataFrame. The function reads the whole matrix into memory.

track_labels() returns one display label for each row of a tracks DataFrame. The example below reads one column of a run and prints its first values on chr1:

from gain.binning import read_bins, read_tracks, read_values, track_labels

values = read_values("binning_run.h5")  # numpy matrix, bins x tracks
bins = read_bins("binning_run.h5")
labels = track_labels(read_tracks("binning_run.h5"))

column = labels.index("hg38/scores/phyloP100way:mean")
on_chr1 = (bins["chrom"] == "chr1").to_numpy()
print(values.shape)
print(values[on_chr1, column][:5])

Row i of values belongs to row i of bins, and column j belongs to labels[j], the label of row j of the tracks.

Labels of the tracks

The file does not store a name for a track, so track_labels builds each label from resource_ids, selector, binner_type and binner_def:

  • The label of a position-score track is its resource id and its aggregator, for example hg38/scores/phyloP100way:mean.

  • The label of a fragment track is its resource, the group and the aggregator, for example frags/s1:B:sum.

A name of the entry prefixes the resource id of a position-score track and of an unpooled fragment track. In the label of a pooled fragment track, the name replaces the resource query. Tracks with equal labels get #<position> appended, where the position is the column of the track.

Errors

read_tracks raises ValueError for a file with a missing or unsupported tracks_layout. read_tracks and read_bins raise ValueError for a table that is missing, not a dataset, or not compound. read_values raises ValueError for a /values that is missing, not a dataset, or not two-dimensional. A path that is missing or not an HDF5 file raises the OSError of h5py.

API

The names below are exported by the package gain.binning.

read_tracks(source)

Read /tracks of a tracks_layout = 2 file.

read_bins(source)

Read /bins of a binning_tool file.

read_values(source)

Read /values of a binning_tool file as a numpy matrix.

track_labels(tracks_df)

Return one display label for each track, in row order.

read_tracks(source: Source) → DataFrame[source]

Read /tracks of a tracks_layout = 2 file.

Row j describes column j of /values. The text fields are str.

Parameters:

source – A path or an open h5py.File. A path is opened and closed inside the call; an open file is only read, never closed.

Returns:

The columns of the /tracks table of the file, one row for each track. The index is the position of the track in /values.

Return type:

pandas.DataFrame

Raises:
  • ValueError – If /tracks is missing, is not a dataset or is not compound, or if the tracks_layout attribute of the file is missing or is not 2. The message names the file.

  • OSError – If a path is missing or is not an HDF5 file. h5py raises it.

read_bins(source: Source) → DataFrame[source]

Read /bins of a binning_tool file.

Row i describes row i of /values. The text fields are str.

Parameters:

source – A path or an open h5py.File. A path is opened and closed inside the call; an open file is only read, never closed.

Returns:

The columns of the /bins table of the file, one row for each bin. The index is the position of the bin in /values.

Return type:

pandas.DataFrame

Raises:
  • ValueError – If /bins is missing, is not a dataset or is not compound. The message names the file and the dataset.

  • OSError – If a path is missing or is not an HDF5 file. h5py raises it.

read_values(source: Source) → ndarray[tuple[Any, ...], dtype[float64]][source]

Read /values of a binning_tool file as a numpy matrix.

The matrix has the shape (n_bins, n_tracks): row i belongs to row i of /bins and column j to row j of /tracks. A bin with no data is NaN. The whole matrix is read into memory. For a large file, slice h5["values"][start:stop] with h5py instead.

Parameters:

source – A path or an open h5py.File. A path is opened and closed inside the call; an open file is only read, never closed.

Returns:

A float64 array of two dimensions.

Return type:

numpy.ndarray

Raises:
  • ValueError – If /values is missing, is not a dataset or is not two-dimensional. The message names the file and the dataset.

  • OSError – If a path is missing or is not an HDF5 file. h5py raises it.

track_labels(tracks_df: DataFrame) → list[str][source]

Return one display label for each track, in row order.

The label is read from resource_ids, selector and the JSON binner_def alone. Equal labels get #<position> appended.

Parameters:

tracks_df – The /tracks rows, as read_tracks() returns them.

Returns:

One label for each row of tracks_df, in row order. Label j names column j of /values.

Return type:

list of str

Raises:
  • KeyError – If tracks_df has no binner_type, resource_ids, selector or binner_def column.

  • ValueError – If a binner_def is not JSON.