annotate_tabular

annotate_tabular annotates the variants, positions or regions of a tabular file with an annotation pipeline. It reads a TSV or CSV file with a header, plain or gzip/bgzip-compressed, and a pipeline, a YAML file or a GRR pipeline resource id. It writes a copy of the table with one new column for each attribute of the pipeline. Use it for most annotation jobs. When the input is sorted, bgzip-compressed and tabix-indexed, the tool splits the job by genomic region and runs the parts in parallel.

Examples

annotate_tabular small_input.csv pipeline/hg38_clinical_annotation

Annotates the variants of small_input.csv with the pipeline/hg38_clinical_annotation pipeline resource and writes small_input.annotated.csv.

annotate_tabular input.tsv annotation.yaml --col-chrom CHROMOSOME -o my_output.tsv

Reads the chromosome from the CHROMOSOME column instead of chrom and writes the result to my_output.tsv.

annotate_tabular variants.sorted.tsv.bgz pipeline/hg38_clinical_annotation -r 30_000_000 -j 10

Splits a tabix-indexed input into 30 Mbp regions and annotates them with 10 parallel jobs.

See also

• Quick annotation test: run a first annotation.

• Custom annotation pipelines: annotate with a pipeline file.

• Parallelizing large annotation jobs: prepare a large input and annotate it in parallel.

• Annotating positions and regions: annotate positions and regions.

• Reannotation: reuse the attributes of a previous annotation.

• annotate_tabular: the column options and the common options.

• Running in parallel: the number of workers, the named clusters and the reruns.

Option reference

Annotate columns

usage: annotate_tabular [-h] [--col-pos COL_POS] [--col-variant COL_VARIANT]
                        [--col-pos-end COL_POS_END] [--col-alt COL_ALT]
                        [--col-pos-beg COL_POS_BEG] [--col-ref COL_REF]
                        [--col-location COL_LOCATION]
                        [--col-vcf-like COL_VCF_LIKE]
                        [--col-cnv-type COL_CNV_TYPE] [--col-chrom COL_CHROM]
                        [--input-separator INPUT_SEPARATOR]
                        [--output-separator OUTPUT_SEPARATOR] [-n] [--version]
                        [-r REGION_SIZE] [-w WORK_DIR] [-o OUTPUT]
                        [--reannotate REANNOTATE] [--full-reannotation]
                        [--keep-parts] [--no-keep-parts] [--keep-work-dir]
                        [--batch-size BATCH_SIZE] [--allow-remote-resources]
                        [-g GRR_FILENAME] [--grr-directory GRR_DIRECTORY]
                        [-R REFERENCE_GENOME_RESOURCE_ID]
                        [-G GENE_MODELS_RESOURCE_ID] [-ar] [-j JOBS]
                        [--process-pool] [-N DASK_CLUSTER_NAME]
                        [-c DASK_CLUSTER_CONFIG_FILE]
                        [--task-log-dir TASK_LOG_DIR]
                        [-t TASK_IDS [TASK_IDS ...]] [--keep-going]
                        [--fork-tasks] [--force] [-d TASK_STATUS_DIR]
                        [--verbose] [--logfile LOGFILE]
                        [input] [pipeline] [{run,list,status}]

Positional Arguments

input

the input file; gzip/bgzip-compressed inputs (.gz/.bgz), optionally tabix-indexed, are detected by extension

Default: '-'

pipeline

The pipeline definition file. By default, or if the value is gpf_instance, the annotation pipeline from the configured gpf instance will be used.

Default: 'context'

Named Arguments

--col-pos

The column name that stores pos. Use “-” to exclude this column requirement, causing annotatable patterns that require pos to be skipped.

Default: 'pos'

--col-variant

The column name that stores variant. Use “-” to exclude this column requirement, causing annotatable patterns that require variant to be skipped.

Default: 'variant'

--col-pos-end

The column name that stores pos_end. Use “-” to exclude this column requirement, causing annotatable patterns that require pos_end to be skipped.

Default: 'pos_end'

--col-alt

The column name that stores alt. Use “-” to exclude this column requirement, causing annotatable patterns that require alt to be skipped.

Default: 'alt'

--col-pos-beg

The column name that stores pos_beg. Use “-” to exclude this column requirement, causing annotatable patterns that require pos_beg to be skipped.

Default: 'pos_beg'

--col-ref

The column name that stores ref. Use “-” to exclude this column requirement, causing annotatable patterns that require ref to be skipped.

Default: 'ref'

--col-location

The column name that stores location. Use “-” to exclude this column requirement, causing annotatable patterns that require location to be skipped.

Default: 'location'

--col-vcf-like

The column name that stores vcf_like. Use “-” to exclude this column requirement, causing annotatable patterns that require vcf_like to be skipped.

Default: 'vcf_like'

--col-cnv-type

The column name that stores cnv_type. Use “-” to exclude this column requirement, causing annotatable patterns that require cnv_type to be skipped.

Default: 'cnv_type'

--col-chrom

The column name that stores chrom. Use “-” to exclude this column requirement, causing annotatable patterns that require chrom to be skipped.

Default: 'chrom'

--input-separator, --in-sep

The column separator in the input; defaults to a tab, or a comma when the input filename ends in .csv (optionally .gz/.bgz compressed)

--output-separator, --out-sep

The column separator in the output

-n, --dry-run

Print the annotation/reannotation plan and exit without writing any output.

Default: False

--version

Show the GAIn version and exit

Default: False

-r, --region-size

region size to parallelize by; zero or negative values disable parallelization

Default: 300000000

-w, --work-dir

Directory to store intermediate output files in

-o, --output

Filename of the output result; a .gz/.bgz suffix produces a compressed, tabix-indexed output. If the suffix is omitted, a compressed input’s suffix is mirrored onto the output.

--reannotate

Old pipeline config to reannotate over

--full-reannotation, --fr

Ignore any previous annotation and run a full reannotation.

Default: False

--keep-parts, --keep-intermediate-files

Keep intermediate files after annotatio.

Default: False

--no-keep-parts, --no-keep-intermediate-files

Remove intermediate files after annotatio.

Default: True

--keep-work-dir

Keep the working directory after a successful annotation (by default a working directory the tool created is removed).

Default: False

--batch-size

Annotate in batches of

Default: 0

--allow-remote-resources

Skip the check that warns or aborts when annotating many variants against non-local (http/https/s3) genomic resources.

Default: False

-g, --grr-filename, --grr

The GRR configuration file. If the argument is absent, the a GRR repository from the current genomic context (i.e. gpf_instance) will be used or, if that fails, the default GRR configuration will be used.

--grr-directory

Local GRR directory to use as repository.

-R, --reference-genome-resource-id, --ref

The resource id for the reference genome. If the argument is absent the reference genome from the current genomic context will be used.

-G, --gene-models-resource-id, --genes

The resource is of the gene models resource. If the argument is absent the gene models from the current genomic context will be used.

-ar, --allow-repeated-attributes

Rename repeated attributes instead of raising an error.

Default: False

--verbose, -v, -V

Default: 0

--logfile

File to log output to. If not set, logs to console.

Task Graph Executor

-j, --jobs

Number of jobs to run in parallel. Defaults to the number of processors on the machine

--process-pool, --pp

Use a process pool executor with the specified number of processes instead of a dask distributed executor.

Default: False

-N, --dask-cluster-name, --dcn

The name of a named cluster in the dask config, under ‘gain_named_cluster.clusters’. The tasks run on it.

-c, --dccf, --dask-cluster-config-file

dask cluster config file

--task-log-dir

Path to directory where to store tasks’ logs

Execution Mode

command

Possible choices: run, list, status

Command to execute on the import configuration. run - runs the import process list - lists the tasks to be executed but doesn’t run them status - synonym for list

Default: 'run'

-t, --task-ids

--keep-going

Whether or not to keep executing in case of an error

Default: False

--fork-tasks, --fork-task, --fork

Whether to fork a new worker process for each task

Default: False

--force, -f

Ignore precomputed state and always rerun all tasks.

Default: False

-d, --task-status-dir, --tsd

Directory to store the task progress.