Histograms

Histograms summarize the distribution of score values in a genomic resource. GAIn computes histograms during a resource’s statistics build, stores the results with the resource, and displays them on its HTML summary page. Histograms can also be retrieved and examined through the Python interface.

Histogram types

GAIn supports three types of histograms, depending on the kind of data being summarized:

• Numeric histograms divide a range of numerical values into bins and count the values falling within each bin.

• Categorical histograms count the occurrences of individual values or categories.

• Null histograms represent cases where a histogram is unavailable or was intentionally not computed, together with an explanation.

Each histogram type has a corresponding Python class and configuration class:

Type	Histogram class	Configuration class
Numeric	NumberHistogram	NumberHistogramConfig
Categorical	CategoricalHistogram	CategoricalHistogramConfig
Null	NullHistogram	NullHistogramConfig

The names Histogram and HistogramConfig are type aliases that represent the three histogram classes and their corresponding configuration classes:

Histogram = NullHistogram | CategoricalHistogram | NumberHistogram
HistogramConfig = (
    NullHistogramConfig | CategoricalHistogramConfig | NumberHistogramConfig
)

These aliases are used in type annotations. For example, a function declared to return a Histogram may return any of the three histogram types. They do not define additional Python classes.

The isinstance() function can be used to identify its class before accessing type-specific attributes.

Reading numeric histograms

Numeric histograms divide a range of numerical values into bins and count the values falling within each bin. Histograms can be retrieved from genomic score and gene score resources using get_score_histogram().

For example, the following code retrieves the histogram of the phastCons100way conservation score:

import gain.grr as grr
from gain.genomic_resources.histogram import NumberHistogram

repo = grr.build_genomic_resource_repository()
score = grr.build_score_from_resource_id(
    "hg38/scores/phastCons100way", repo
)

hist = score.get_score_histogram("phastCons100way")
assert isinstance(hist, NumberHistogram)

A NumberHistogram contains two arrays describing the distribution:

• bins contains the boundaries of the histogram bins.

• bars contains the number of values falling within each bin.

For example:

print(len(hist.bins), len(hist.bars))
print(hist.bins[:3], hist.bars[:3])
print(hist.min_value, hist.max_value, hist.out_of_range_bins)

101 100
[0.   0.01 0.02] [1829125896  177725064  102444378]
0.0 1.0 [0, 0]

The bins array has one more element than bars because each bar represents the interval between two consecutive bin boundaries. For example, the first bar counts values between 0 and 0.01.

The min_value and max_value attributes report the smallest and largest values observed during histogram computation. Values outside the configured histogram range are counted separately in out_of_range_bins, which contains two counts: values below and values above the range.

When a resource defines only one score, the score ID can be omitted from get_score_histogram(). For example, phastCons100way defines only one score, so the following code verifies that both calls return equivalent histograms:

hist = score.get_score_histogram()
assert hist.to_dict() == score.get_score_histogram("phastCons100way").to_dict()

For resources containing multiple scores, the score ID must be specified. Otherwise, get_score_histogram() raises a ValueError listing the available score IDs.

Numeric histograms also provide summary statistics through three attributes:

• count gives the total number of values, calculated as the sum of bars and both counts in out_of_range_bins.

• mean gives the weighted mean of the values.

• std gives the population standard deviation, rather than the sample standard deviation.

These values can be accessed directly, or through moments_summary(), which returns formatted label-value pairs suitable for the resource summary page:

print(hist.count, hist.mean, hist.std)
print(hist.moments_summary())

2944992024 0.09809510937890936 0.23381027247397798
(('n', '2,944,992,024'), ('mean', '0.0981'), ('sd', '0.234'))

Older histogram files may lack the stored information needed to calculate these statistics. In that case, count, mean, and std are None. When no values were included during computation, count is zero, while mean and std are None.

The moments_summary() method returns None when the summary statistics are unavailable or no values were counted. Code that uses mean or std in calculations should therefore check that they are not None.

Reading categorical histograms

Categorical histograms count the occurrences of individual values rather than dividing a numerical range into bins. They are useful for scores representing categories, such as variant classifications.

For example, the CLNVC score in the ClinVar resource describes the type of sequence variant. Its histogram can be retrieved as follows:

from gain.genomic_resources.histogram import CategoricalHistogram

score = grr.build_score_from_resource_id(
    "hg38/scores/ClinVar_20240730", repo
)

hist = score.get_score_histogram("CLNVC")
assert isinstance(hist, CategoricalHistogram)

print(hist.truncated)
print(hist.unique_values, hist.total_count)
print(hist.raw_values)
print(hist.display_values)

False
8 2906433
{'single_nucleotide_variant': 2667265, 'Microsatellite': 32873, 'Indel': 14277, 'Deletion': 123084, 'Duplication': 55642, 'Insertion': 11669, 'Variation': 366, 'Inversion': 1257}
{'single_nucleotide_variant': 2667265, 'Deletion': 123084, 'Duplication': 55642, 'Microsatellite': 32873, 'Indel': 14277, 'Insertion': 11669, 'Inversion': 1257, 'Variation': 366}

A CategoricalHistogram provides several attributes describing the distribution:

• raw_values contains the counts for each category stored in the histogram.

• display_values contains the categories selected and ordered for display on the resource summary page.

• unique_values gives the total number of distinct values.

• total_count gives the total number of values counted.

The distinction between raw_values and display_values is important. While raw_values contains the stored category counts, display_values follows the histogram configuration, which determines which categories are displayed and in what order. In the CLNVC example, all eight categories are displayed, ordered by their counts from highest to lowest.

Some categorical scores have very large numbers of distinct values. For these scores, GAIn can store a smaller, truncated version of the histogram alongside the complete one. This additional file, called a truncated sidecar, contains only the categories selected for display.

The CLNDN score in ClinVar, which describes conditions associated with variants, provides an example. The complete histogram and its truncated version can be retrieved separately:

full = score.get_score_histogram("CLNDN")
small = score.get_score_histogram("CLNDN", truncated=True)

for hist in (full, small):
    print(
        hist.truncated,
        hist.unique_values,
        hist.total_count,
        len(hist.raw_values),
        len(hist.display_values),
    )

False 166334 2905738 166334 11
True 166334 2905738 10 10

The complete histogram contains counts for all 166,334 distinct values, although only 11 are selected for display. The truncated histogram retains just 10 categories in raw_values and display_values.

Importantly, unique_values and total_count still describe the complete distribution, even when the histogram is truncated. The smaller histogram therefore preserves the overall statistics without requiring every category count to be loaded.

Using truncated=True is useful when only the displayed categories and overall statistics are needed. If no truncated sidecar exists, GAIn returns the complete histogram instead.

The configuration options controlling the number and ordering of displayed categories, together with the limits on categorical histograms, are described in Categorical limits and truncation.

Reading null histograms

A null histogram represents a score for which a histogram was not computed or is unavailable. Unlike numeric and categorical histograms, it contains no counts. Instead, a NullHistogram object provides a reason explaining why the histogram is unavailable.

For example, the RS score in the ClinVar resource has a null histogram:

from gain.genomic_resources.histogram import NullHistogram

score = grr.build_score_from_resource_id(
    "hg38/scores/ClinVar_20251019", repo
)

hist = score.get_score_histogram("RS")
assert isinstance(hist, NullHistogram)

print(hist.reason)

Histogram is not available for this score.

The reason may come from the score’s histogram configuration or from the statistics build. For example, a histogram may be intentionally disabled, or a categorical score may contain too many distinct values to be counted under the configured limits.

Importantly, a null histogram is an actual Python object, not None. It provides the same general interface as the other histogram types, allowing GAIn to handle different histogram types consistently. Its plot() method produces no plot, and the resource summary page displays the reason instead.

The reason attribute is required and originates from the corresponding NullHistogramConfig. When no specific explanation is available, default_config() provides the default reason "Unspecified reason".

A null histogram can also result from an unsuccessful attempt to load a stored histogram. These cases are described in the following section.

Loading histogram files

Histograms are stored as part of a resource’s statistics, using .yaml files in older resources and .json files in newer ones. The load_histogram() function can be used to load a histogram directly from a resource file.

For example:

from gain.genomic_resources.histogram import load_histogram

score = grr.build_score_from_resource_id(
    "hg38/scores/phastCons100way", repo
)
resource = repo.get_resource("hg38/scores/phastCons100way")

filename = score.get_histogram_filename("phastCons100way")
hist = load_histogram(resource, filename)

print(hist.type, hist.values_domain())

number_histogram [0, 1]

The get_histogram_filename() method determines the appropriate histogram filename. It returns the gzipped .json.gz filename when the resource manifest lists it together with a truncated sidecar. Otherwise, it returns the .yaml filename when one is listed in the resource manifest, or the .json filename. This is preferable to specifying the filename manually.

When load_histogram() encounters a missing file, an unrecognized file extension, or a histogram that cannot be deserialized, it returns a NullHistogram containing an explanation of the problem.

However, not all loading errors are handled this way. For example, malformed YAML or JSON and missing required fields such as config may raise exceptions rather than returning a NullHistogram.

Histogram configuration

Histogram settings are defined in the histogram: block of a score’s scores: entry in the resource definition. These settings control properties such as bin counts, logarithmic scales, view ranges, value ordering, and custom plotting functions. The available options, including an example of a custom plot_function, are described in Histogram configuration.

In Python, these settings are represented by HistogramConfig objects. GAIn processes the configuration and computes the histogram through the following steps:

1. Reading score definitions. The parse_scoredef_config() function reads the scores: block of a resource definition and creates a GenomicScoreDef object for each score.

2. Reading histogram configurations. For each score, build_histogram_config() reads its histogram: block and creates the corresponding configuration object, such as NumberHistogramConfig, CategoricalHistogramConfig, or NullHistogramConfig. If no histogram: block is present, the function returns None rather than creating a default configuration.

3. Applying default configurations. When no histogram configuration is explicitly provided, GAIn uses build_default_histogram_conf() to select one based on the score’s declared value_type. This happens when statistics are computed and again when the configuration is retrieved through ScoreResource.get_histogram_config(). As a result, a score can have a histogram even without an explicit histogram: block. The default histogram type depends on the declared value type, not on the values observed in the data.

4. Computing and storing the histogram. During the statistics build, GAIn creates a histogram using the selected configuration and populates it with score values. Methods such as add_value() and the vectorized add_batch() are used to accumulate values. The completed histogram is serialized and stored with the resource.

5. Retrieving the histogram. The ScoreResource.get_score_histogram() method retrieves the stored histogram through load_histogram(). An exception is a score configured to use a null histogram, either explicitly or by default. In this case, no histogram file is stored. Instead, get_score_histogram() constructs a NullHistogram using the reason specified by the configuration. Calling load_histogram() directly for the nonexistent file would instead return a NullHistogram indicating that the file is missing.

Merging allows histogram statistics to be computed in parallel. Different tasks calculate histograms for separate genomic regions, and their results are combined using methods such as merge().

Merging requires compatible histogram configurations. For example, numeric histograms with different bin boundaries cannot be combined. The histogram configuration must therefore be established before computation begins, rather than determined from the data as values are processed.

Categorical limits and truncation

Categorical histograms maintain a count for each distinct value. GAIn uses two separate mechanisms to manage scores with many distinct values: cardinality limits and truncation.

Cardinality limits prevent a histogram from being computed when the number of distinct values becomes too large. For scores that are not explicitly configured as categorical, GAIn applies a limit of 100 unique values (UNIQUE_VALUES_LIMIT). If this limit is exceeded, add_value() raises a HistogramError, and the statistics build replaces the histogram with a NullHistogram explaining the problem. No partial categorical histogram is retained.

This limit does not apply when a score is explicitly configured as categorical. The distinction is controlled by the enforce_type setting in default_config(), whose name can be counterintuitive in this context.

Truncation, by contrast, applies to categorical histograms that have been computed successfully. It limits the categories stored for display without discarding the complete histogram.

The display_values attribute contains the categories selected and ordered for the resource summary page. This selection is controlled by three configuration options:

• displayed_values_count limits the number of displayed categories.

• displayed_values_percent specifies the percentage of total counts represented by the displayed categories.

• value_order determines their ordering.

The serialize_truncated() method writes the selected categories to a smaller file alongside the complete histogram. This truncated sidecar retains the unique_values and total_count of the full histogram, even though it stores counts for only a subset of categories.

Thus, exceeding the cardinality limit prevents a categorical histogram from being created, while truncation provides a smaller representation of a successfully computed histogram.

API

The functions and classes below are defined in the module gain.genomic_resources.histogram. gain.grr does not export any of them.

load_histogram(resource, filename)	Load a stored histogram from a resource.
NumberHistogram(config[, bins, bars])	A histogram of numeric values in bins.
NumberHistogram.add_batch(values, weights)	Add a batch of (value, weight) pairs, vectorized.
NumberHistogram.add_value(value[, count])	Add a value to the histogram.
NumberHistogram.count	The weighted count of the added values, or None when unknown.
NumberHistogram.deserialize(content)	Rebuild a number histogram from serialize output.
NumberHistogram.from_dict(data)	Build a number histogram from a dict.
NumberHistogram.mean	The weighted mean, or None when unknown or nothing was added.
NumberHistogram.merge(other)	Add the counts of another histogram into this one.
NumberHistogram.moments	The count, sum and sum of squares, or None when unknown.
NumberHistogram.moments_summary()	The count, mean and standard deviation, as text for the page.
NumberHistogram.plot(outfile, score_id[, ...])	Plot the histogram and write the image into outfile.
NumberHistogram.serialize()	Render this histogram as the JSON stored in the resource.
NumberHistogram.std	The population standard deviation, or None when mean is.
NumberHistogram.to_dict()	Render this histogram as the mapping from_dict reads back.
NumberHistogram.values_domain()	The observed value range, as text for the summary page.
NumberHistogram.view_max()	The high edge of the last bin.
NumberHistogram.view_min()	The low edge of the first bin.
CategoricalHistogram(config[, counter, ...])	A histogram that counts each distinct value of a score.
CategoricalHistogram.add_batch(values, weights)	Add a batch of (value, weight) pairs, vectorized.
CategoricalHistogram.add_value(value[, count])	Add a value to the categorical histogram.
CategoricalHistogram.deserialize(content)	Rebuild a categorical histogram from serialize output.
CategoricalHistogram.display_values	The counts that the summary page shows, in display order.
CategoricalHistogram.from_dict(data)	Build a categorical histogram from a dict.
CategoricalHistogram.merge(other)	Add the counts of another histogram into this one.
CategoricalHistogram.moments_summary()	Return None, because a categorical value has no mean.
CategoricalHistogram.plot(outfile, score_id, *)	Plot the displayed values and write the image into outfile.
CategoricalHistogram.raw_values	Every counted value with its count, unordered.
CategoricalHistogram.serialize()	Render the full histogram as the JSON stored in the resource.
CategoricalHistogram.serialize_gzipped()	Render the serialize JSON as gzip bytes.
CategoricalHistogram.serialize_truncated()	Render the small truncated form of this histogram as JSON.
CategoricalHistogram.to_dict()	Render this histogram as the mapping from_dict reads back.
CategoricalHistogram.total_count	The sum of all counts of the full histogram.
CategoricalHistogram.unique_values	The number of distinct values of the full histogram.
CategoricalHistogram.values_domain()	The displayed values, as text for the summary page.
NullHistogram(config)	A histogram that has no counts and records why.
NullHistogram.add_value(value[, count])	Discard the value.
NullHistogram.deserialize(content)	Rebuild a null histogram from serialize output.
NullHistogram.from_dict(data)	Build a null histogram from a dict.
NullHistogram.merge(other)	Do nothing, because there are no counts to add.
NullHistogram.moments_summary()	Return None, because a null histogram has no moments.
NullHistogram.plot(_outfile, _score_id)	Draw nothing and leave outfile untouched.
NullHistogram.serialize()	Render this histogram as the JSON stored in the resource.
NullHistogram.to_dict()	Render this histogram as the mapping from_dict reads back.
NullHistogram.values_domain()	Report that there is no domain.
NumberHistogramConfig(view_range[, ...])	Configuration of a number histogram.
NumberHistogramConfig.default_config(min_max)	Build the config of a score that declares no histogram.
NumberHistogramConfig.from_dict(parsed)	Build a number histogram config from a parsed yaml file.
NumberHistogramConfig.has_view_range()	Whether both ends of the view range are pinned.
NumberHistogramConfig.to_dict()	Transform number histogram config to dict.
CategoricalHistogramConfig([...])	Configuration of a categorical histogram.
CategoricalHistogramConfig.default_config()	Build the config of a score that declares no histogram.
CategoricalHistogramConfig.from_dict(parsed)	Create categorical histogram config from a configuration dict.
CategoricalHistogramConfig.to_dict()	Transform categorical histogram config to dict.
NullHistogramConfig(reason)	Configuration of a null histogram.
NullHistogramConfig.default_config()	Build a null config for a caller with no reason to give.
NullHistogramConfig.from_dict(parsed)	Create a null histogram config from a configuration dict.
NullHistogramConfig.to_dict()	Render this config as the mapping from_dict reads back.

load_histogram(resource: GenomicResource, filename: str) → NullHistogram | CategoricalHistogram | NumberHistogram

Load a stored histogram from a resource.

Parameters:

• resource (GenomicResource) – The resource that holds the file.

• filename (str) – The file name in the resource. The suffix selects the format: .yaml, .json or .json.gz (gzipped JSON).

Returns:

The histogram. A missing file, an unknown suffix, an unknown histogram type and a body that fails to deserialize each give a NullHistogram with the reason.

Return type:

Histogram

Raises:

• KeyError – If the parsed file has no config key or no type in it.

• yaml.YAMLError – If a .yaml file is not valid YAML.

• json.JSONDecodeError – If a .json or .json.gz file is not valid JSON.

class NumberHistogram(config: NumberHistogramConfig, bins: ndarray | None = None, bars: ndarray | None = None)

A histogram of numeric values in bins.

Parameters:

• config (NumberHistogramConfig) – The configuration. Its view range must have both edges set.

• bins (np.ndarray | None) – The bin edges, one more than the bars. Pass both bins and bars or neither.

• bars (np.ndarray | None) – The count of each bin.

Raises:

• ValueError – If the view range has an unset or NaN edge, if a log scale has no x_min_log, or if only one of bins and bars is given.

• Attributes: – type (str): The type name that a stored file carries, number_histogram. config (NumberHistogramConfig): The configuration. bins (np.ndarray): The bin edges. There is one more edge than there are bars. bars (np.ndarray): The count of each bin. Bar i counts the values from bins[i] up to bins[i + 1]. out_of_range_bins (list[int]): The counts of the values below and of the values above the view range. min_value (float): The smallest value that was added. NaN when there is none. max_value (float): The largest value that was added. NaN when there is none. view_range (tuple[float, float]): The low and the high edge of the binned range.

add_batch(values: ndarray, weights: ndarray) → None

Add a batch of (value, weight) pairs, vectorized.

The result is the same as calling add_value for each pair in order, the sums behind mean and std included.

Parameters:

• values (np.ndarray) – The values. NaN values are skipped. The array is converted to float64.

• weights (np.ndarray) – The weight of each value, of the same length as values.

add_value(value: float | generic | None, count: int = 1) → None

Add a value to the histogram.

Parameters:

• value (float | np.generic | None) – The value. None and NaN are skipped and nothing is counted.

• count (int) – The weight of the value.

Raises:

TypeError – If the value is not a real number, for example text.

property count: int | None

The weighted count of the added values, or None when unknown.

static deserialize(content: str) → NumberHistogram

Rebuild a number histogram from serialize output.

Parameters:

content (str) – The JSON text.

Returns:

The histogram.

Return type:

NumberHistogram

static from_dict(data: dict[str, Any]) → NumberHistogram

Build a number histogram from a dict.

Parameters:

data (dict[str, Any]) – A mapping like the result of to_dict. A missing key min_value or max_value gives NaN. A missing out_of_range_bins gives two zeros. A mapping without the stored count, sum and sum of squares gives a histogram with unknown count, mean and std.

Returns:

The histogram.

Return type:

NumberHistogram

Raises:

KeyError – If the key config is missing.

property mean: float | None

The weighted mean, or None when unknown or nothing was added.

merge(other: NumberHistogram) → None

Add the counts of another histogram into this one.

Parameters:

other (NumberHistogram) – A histogram with the same bin edges. This method changes only self.

Raises:

AssertionError – If the bin edges of the two histograms differ.

property moments: Moments | None

The count, sum and sum of squares, or None when unknown.

moments_summary() → tuple[tuple[str, str], ...] | None

The count, mean and standard deviation, as text for the page.

Returns:

Three (label, text) pairs: n, mean and sd. None when the numbers are unknown or nothing was added.

Return type:

MomentsSummary | None

plot(outfile: IO, score_id: str, y_axis_label: str | None = None, small_values_description: str | None = None, large_values_description: str | None = None) → None

Plot the histogram and write the image into outfile.

Parameters:

• outfile (IO) – A binary file object that receives the image.

• score_id (str) – The score name. It is the x axis label.

• y_axis_label (str | None) – The y axis label. None means count.

• small_values_description (str | None) – The text under the low end of the x axis. The plot shows both descriptions or neither.

• large_values_description (str | None) – The text under the high end of the x axis.

serialize() → str

Render this histogram as the JSON stored in the resource.

Returns:

The JSON text.

Return type:

str

property std: float | None

The population standard deviation, or None when mean is.

to_dict() → dict[str, Any]

Render this histogram as the mapping from_dict reads back.

Returns:

A mapping that JSON can store as it is. The bin edges and the counts are lists.

Return type:

dict[str, Any]

values_domain() → str

The observed value range, as text for the summary page.

Returns:

The smallest and the largest added value, as [min, max]. It is not the view range.

Return type:

str

view_max() → float

The high edge of the last bin.

Returns:

The high edge of the view range. Values above it are counted in out_of_range_bins and not binned.

Return type:

float

view_min() → float

The low edge of the first bin.

Returns:

The low edge of the view range. Values below it are counted in out_of_range_bins and not binned. It is not the smallest value that was added.

Return type:

float

class CategoricalHistogram(config: CategoricalHistogramConfig, counter: dict[str | int, int] | None = None, *, truncated: bool = False, unique_values: int | None = None, total_count: int | None = None)

A histogram that counts each distinct value of a score.

Parameters:

• config (CategoricalHistogramConfig) – The configuration.

• counter (dict[str | int, int] | None) – The counts of the values. None means an empty histogram.

• truncated (bool) – True when counter holds only the values of a truncated file.

• unique_values (int | None) – The distinct values of the full histogram. None means the length of counter.

• total_count (int | None) – The sum of the counts of the full histogram. None means the sum of counter.

• Attributes –

  type (str): The type name that a stored file carries,

  categorical_histogram.

  UNIQUE_VALUES_LIMIT (int): The most distinct values that a

  histogram accepts when its config does not set enforce_type. The value is 100.

  config (CategoricalHistogramConfig): The configuration.

  enforce_type (bool): The enforce_type flag of the config.

  truncated (bool): True when this histogram was loaded from a

  truncated file. Its counts are then only the kept values.

  y_log_scale (bool): The y_log_scale flag of the config.

add_batch(values: ndarray, weights: ndarray) → None

Add a batch of (value, weight) pairs, vectorized.

The result is the same as calling add_value for each pair in order.

Parameters:

• values (np.ndarray) – The values. None cells are skipped.

• weights (np.ndarray) – The weight of each value, of the same length as values.

Raises:

• TypeError – If a value is neither text nor an integer.

• HistogramError – If the config does not set enforce_type and the batch takes the histogram past UNIQUE_VALUES_LIMIT.

add_value(value: str | int | None, count: int = 1) → None

Add a value to the categorical histogram.

Parameters:

• value (str | int | None) – The value. None is skipped and nothing is counted.

• count (int) – The weight of the value.

Raises:

• TypeError – If the value is neither text nor an integer.

• HistogramError – If the config does not set enforce_type and the histogram now holds more than UNIQUE_VALUES_LIMIT distinct values.

static deserialize(content: str) → CategoricalHistogram

Rebuild a categorical histogram from serialize output.

Parameters:

content (str) – The JSON text.

Returns:

The histogram.

Return type:

CategoricalHistogram

property display_values: dict[str | int, int]

The counts that the summary page shows, in display order.

The config selects and orders the values. The values that the selection leaves out are summed under the key Other.

Raises:

ValueError – If the config sets value_order and the order leaves out a counted value.

static from_dict(data: dict[str, Any]) → CategoricalHistogram

Build a categorical histogram from a dict.

Parameters:

data (dict[str, Any]) – A mapping like the result of to_dict, or the truncated form. A full histogram has no truncated, unique_values or total_count key. They default to False and to the values computed from the counts.

Returns:

The histogram.

Return type:

CategoricalHistogram

Raises:

KeyError – If the key config is missing.

merge(other: CategoricalHistogram) → None

Add the counts of another histogram into this one.

Parameters:

other (CategoricalHistogram) – A full histogram with the same config.

Raises:

• HistogramError – If either histogram is truncated, or if the config does not set enforce_type and the sum has too many distinct values.

• AssertionError – If the two configs differ.

moments_summary() → None

Return None, because a categorical value has no mean.

Returns:

Always.

Return type:

None

plot(outfile: IO, score_id: str, *, y_axis_label: str | None = None, small_values_description: str | None = None, large_values_description: str | None = None) → None

Plot the displayed values and write the image into outfile.

Parameters:

• outfile (IO) – A binary file object that receives the image.

• score_id (str) – The score name. It is the x axis label.

• y_axis_label (str | None) – The y axis label. None means count.

• small_values_description (str | None) – The text under the first value. The plot shows both descriptions or neither.

• large_values_description (str | None) – The text under the last value.

property raw_values: dict[str | int, int]

Every counted value with its count, unordered.

A truncated histogram returns only the values that its file kept.

serialize() → str

Render the full histogram as the JSON stored in the resource.

Returns:

The JSON text.

Return type:

str

serialize_gzipped() → bytes

Render the serialize JSON as gzip bytes.

Returns:

The same histogram gives the same bytes on every call.

Return type:

bytes

serialize_truncated() → str

Render the small truncated form of this histogram as JSON.

Returns:

The JSON text. It holds the config, the values that the config selects for display, unique_values and total_count of the full histogram, and the flag truncated. A value in value_order that has no count gets the count 0.

Return type:

str

to_dict() → dict[str, Any]

Render this histogram as the mapping from_dict reads back.

Returns:

A mapping with the keys config and values. It holds every counted value, not only the displayed ones.

Return type:

dict[str, Any]

property total_count: int

The sum of all counts of the full histogram.

A truncated histogram returns the sum of the full data, not the sum of the values it kept.

property unique_values: int

The number of distinct values of the full histogram.

A truncated histogram returns the count of the full data, not the count of the values it kept.

values_domain() → str

The displayed values, as text for the summary page.

Returns:

The displayed values, separated by commas. A truncated histogram adds the text (top N of M values).

Return type:

str

class NullHistogram(config: NullHistogramConfig | None)

A histogram that has no counts and records why.

Parameters:

• config (NullHistogramConfig | None) – The configuration. None gives the reason Unspecified reason.

• Attributes –

  type (str): The type name that a stored file carries,

  null_histogram.

  reason (str): Why there is no histogram.

add_value(value: Any, count: int = 1) → None

Discard the value.

Parameters:

• value (Any) – Ignored.

• count (int) – Ignored.

static deserialize(content: str) → NullHistogram

Rebuild a null histogram from serialize output.

Parameters:

content (str) – The JSON text.

Returns:

The histogram.

Return type:

NullHistogram

static from_dict(data: dict[str, Any]) → NullHistogram

Build a null histogram from a dict.

Parameters:

data (dict[str, Any]) – A mapping like the result of to_dict. A missing reason gives an empty text.

Returns:

The histogram.

Return type:

NullHistogram

Raises:

TypeError – If the type in config is not null.

merge(other: NullHistogram | CategoricalHistogram | NumberHistogram) → None

Do nothing, because there are no counts to add.

Parameters:

other (Histogram) – Ignored.

moments_summary() → None

Return None, because a null histogram has no moments.

Returns:

Always.

Return type:

None

plot(_outfile: IO, _score_id: str) → None

Draw nothing and leave outfile untouched.

Parameters:

• _outfile (IO) – Ignored.

• _score_id (str) – Ignored.

serialize() → str

Render this histogram as the JSON stored in the resource.

Returns:

The JSON text.

Return type:

str

to_dict() → dict[str, Any]

Render this histogram as the mapping from_dict reads back.

Returns:

A mapping with the key config. It holds the reason, which is all the content of a null histogram.

Return type:

dict[str, Any]

values_domain() → str

Report that there is no domain.

Returns:

The text NO DOMAIN.

Return type:

str

class NumberHistogramConfig(view_range: tuple[float | None, float | None], number_of_bins: int = 100, x_log_scale: bool = False, y_log_scale: bool = False, x_min_log: float | None = None, plot_function: str | None = None)

Configuration of a number histogram.

Variables:

• view_range (tuple[float | None, float | None]) – The low and the high edge of the binned range. None at either end means the edge is not pinned yet. The histogram needs both edges.

• number_of_bins (int) – The number of bars. The default is 100.

• x_log_scale (bool) – Whether the bins are spaced on a log scale. A log scale requires x_min_log.

• y_log_scale (bool) – Whether the plot draws the bar heights on a log scale. The bin counts are the same.

• x_min_log (float | None) – The high edge of the first bin on a log scale. None when the scale is linear.

• plot_function (str | None) – The file:function name of a custom plot function. None means the built-in plot.

static default_config(min_max: MinMaxValue | None) → NumberHistogramConfig

Build the config of a score that declares no histogram.

Parameters:

min_max (MinMaxValue | None) – The observed minimum and maximum of the score. None means they are not known yet.

Returns:

A config with 100 linear bins. The view range is the observed range. It is (None, None) when min_max is None. When the minimum equals the maximum, the range is the minimum to the minimum plus one.

Return type:

NumberHistogramConfig

static from_dict(parsed: dict[str, Any]) → NumberHistogramConfig

Build a number histogram config from a parsed yaml file.

Parameters:

parsed (dict[str, Any]) – The histogram: mapping of a score. Its type must be number. A missing key takes the default of the field.

Returns:

The config. A missing view_range gives (None, None).

Return type:

NumberHistogramConfig

Raises:

TypeError – If type is not number.

has_view_range() → bool

Whether both ends of the view range are pinned.

Returns:

True when neither edge of view_range is None. False means the config still needs the minimum and the maximum of the score before a histogram can be built from it.

Return type:

bool

to_dict() → dict[str, Any]

Transform number histogram config to dict.

Returns:

The mapping that from_dict reads back. The key plot_function is present only when the config sets one.

Return type:

dict[str, Any]

class CategoricalHistogramConfig(displayed_values_count: int | None = 20, displayed_values_percent: float | None = None, value_order: list[str | int] | None = None, y_log_scale: bool = False, label_rotation: int = 0, plot_function: str | None = None, enforce_type: bool = True, natural_order: bool = False, allow_only_whole_values_y: bool = False)

Configuration of a categorical histogram.

Variables:

• displayed_values_count (int | None) – The number of values that the summary page shows. The default is 20. None means no limit, when displayed_values_percent is set.

• displayed_values_percent (float | None) – Show the most common values until they cover this percent of the counts. None means the count rule applies.

• value_order (list[str | int] | None) – The values to show, in this order. None or an empty list means the order is by count.

• y_log_scale (bool) – Whether the plot draws the counts on a log scale.

• label_rotation (int) – The rotation of the x labels, in degrees.

• plot_function (str | None) – The file:function name of a custom plot function. None means the built-in plot.

• enforce_type (bool) – False means the histogram refuses more than CategoricalHistogram.UNIQUE_VALUES_LIMIT distinct values. True means there is no limit. The flag name is the reverse of its effect.

• natural_order (bool) – Order the displayed values by value, numbers before text, and not by count.

• allow_only_whole_values_y (bool) – Whether the plot uses whole numbers for the y ticks.

static default_config() → CategoricalHistogramConfig

Build the config of a score that declares no histogram.

Returns:

A config that enforces CategoricalHistogram.UNIQUE_VALUES_LIMIT.

Return type:

CategoricalHistogramConfig

static from_dict(parsed: dict[str, Any]) → CategoricalHistogramConfig

Create categorical histogram config from a configuration dict.

Parameters:

parsed (dict[str, Any]) – The histogram: mapping of a score. Its type must be categorical. When it sets neither displayed_values_count nor displayed_values_percent, the count is 20.

Returns:

The config. It does not enforce the unique values limit.

Return type:

CategoricalHistogramConfig

Raises:

• TypeError – If type is not categorical.

• ValueError – If both displayed_values_count and displayed_values_percent are set.

to_dict() → dict[str, Any]

Transform categorical histogram config to dict.

Returns:

The mapping that from_dict reads back. The keys displayed_values_count, displayed_values_percent and plot_function are present only when they differ from the default.

Return type:

dict[str, Any]

class NullHistogramConfig(reason: str)

Configuration of a null histogram.

Variables:

reason (str) – Why the score has no histogram. The reason is required, so a summary page can always explain a null histogram.

static default_config() → NullHistogramConfig

Build a null config for a caller with no reason to give.

Returns:

A config with the reason Unspecified reason.

Return type:

NullHistogramConfig

static from_dict(parsed: dict[str, Any]) → NullHistogramConfig

Create a null histogram config from a configuration dict.

Parameters:

parsed (dict[str, Any]) – The histogram: mapping of a score. Its type must be null.

Returns:

The config. A missing reason gives Unspecified reason.

Return type:

NullHistogramConfig

Raises:

TypeError – If type is not null.

to_dict() → dict[str, Any]

Render this config as the mapping from_dict reads back.

Returns:

The mapping with the keys type and reason.

Return type:

dict[str, Any]