---
title: "Introduction_3_access_cnv_frequency"
author: "Hangjia Zhao"
date: "`r Sys.Date()`"
output: 
  BiocStyle::html_document:
  toc: true
vignette: >
  %\VignetteIndexEntry{Introduction_3_access_cnv_frequency}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
    collapse = TRUE,
    comment = "#>"
)
```

[Progenetix](https://progenetix.org/) is an open data resource that provides curated individual cancer copy number variation (CNV) profiles along with associated metadata sourced from published oncogenomic studies and various data repositories. This vignette offers a comprehensive guide on accessing and visualizing CNV frequency data within the Progenetix database. CNV frequency is pre-calculated based on CNV segment data in Progenetix and reflects the CNV pattern in a cohort. It is defined as the percentage of samples showing a CNV for a genomic region (1MB-sized genomic bins in this case) over the total number of samples in a cohort specified by filters. 

If your focus lies in cancer cell lines, you can access data from [*cancercelllines.org*](https://cancercelllines.org/) by setting the `domain` parameter to `"https://cancercelllines.org"` in `pgxLoader` function. This data repository originates from CNV profiling data of cell lines initially collected as part of Progenetix and currently includes additional types of genomic mutations.


# Load library 

```{r setup}
library(pgxRpi)
library(SummarizedExperiment) # for pgxmatrix data
library(GenomicRanges) # for pgxfreq data
```

## `pgxLoader` function 

This function loads various data from `Progenetix` database via the Beacon v2 API with some extensions (BeaconPlus).  

The parameters of this function used in this tutorial:

* `type`: A string specifying output data type. Only `"cnv_frequency"` is used in this tutorial.
* `output`: A string specifying output file format. When the parameter `type` is `"cnv_frequency"`, available options are `"pgxfreq"` (default) or `"pgxmatrix"`.
* `filters`: Identifiers used in public repositories, bio-ontology terms, or custom terms such as `c("NCIT:C7376", "pgx:icdom-85003")`. When multiple filters are used, they are combined using OR logic when the parameter `type` is   `"cnv_frequency"`. For more information about filters, see the [documentation](https://docs.progenetix.org/common/beacon-api/#filters-filtering-terms).
* `domain`: A string specifying the domain of the query data resource. For CNV frequency data, the value should be either `"http://progenetix.org"` or `"https://cancercelllines.org"`.

# Retrieve CNV frequency

## The first output format (`output` = "pgxfreq")

```{r}
freq_pgxfreq <- pgxLoader(type="cnv_frequency", output ="pgxfreq",
                         filters=c("NCIT:C3058","NCIT:C3493"))

freq_pgxfreq
```

The returned data is stored in [*GRangesList*](https://bioconductor.org/packages/release/bioc/html/GenomicRanges.html) container which consists of multiple *GRanges* objects. Each *GRanges* object stores CNV frequency from samples pecified by a particular filter. 

Within each GRanges object, annotation columns provide the CNV frequencies for each genomic region, expressed as percentage values across samples (%). These percentages represent level-specific gains and losses that overlap the corresponding genomic intervals. For more details on CNV levels, refer to the [ontology tree](http://www.ebi.ac.uk/efo/EFO_0030066).

These genomic intervals are derived from the partitioning of the entire genome (GRCh38). Most of these bins have a size of 1MB, except for a few bins located near the telomeres. In total, there are 3106 intervals encompassing the genome.

To access the CNV frequency data from specific filters, you could access like this

```{r}
freq_pgxfreq[["NCIT:C3058"]]
```

To get metadata such as count of samples used to calculate frequency, use `mcols` function from *GenomicRanges* package:

```{r}
mcols(freq_pgxfreq)
```

## The second output format (`output` = "pgxmatrix")

Choose 8 NCIt codes of interests that correspond to different tumor types

```{r}
code <-c("C3059","C3716","C4917","C3512","C3493","C3771","C4017","C4001")
# add prefix for query
code <- paste0("NCIT:",code)
```

load data with the specified codes

```{r}
freq_pgxmatrix <- pgxLoader(type="cnv_frequency",output ="pgxmatrix",filters=code)
freq_pgxmatrix
```

The returned data is stored in [*RangedSummarizedExperiment*](https://bioconductor.org/packages/release/bioc/html/SummarizedExperiment.html) object, which is a matrix-like container where rows represent ranges of interest (as a GRanges object) and columns represent filters. 

To get metadata such as count of samples used to calculate frequency, use `colData` function from *SummarizedExperiment* package:

```{r}
colData(freq_pgxmatrix)
```

To access the CNV frequency matrix, use `assay` accesssor from *SummarizedExperiment* package

```{r}
head(assay(freq_pgxmatrix,"lowlevel_cnv_frequency"))
```

The matrix has 6212 rows (genomic regions) and 8 columns (filters). The rows comprised 3106 intervals with “gain status” plus 3106 intervals with “loss status”. 

The value is the percentage of samples from the corresponding filter having one or more CNV events in the specific genomic intervals. For example, if the value in the second row and first column is 6.839, it means that 6.839% samples from the corresponding filter NCIT:C3059 having one or more low-level duplication events in the genomic interval in chromosome 1: 400000-1400000.

You could get the interval information by `rowRanges` function from *SummarizedExperiment* package.

```{r}
rowRanges(freq_pgxmatrix)
```

Note: it is different from CNV fraction matrix introduced in *Introduction_2_query_variants*. Value in this matrix is **percentage (%) of samples** having one or more CNVs overlapped with the binned interval while the value in CNV fraction matrix is **fraction in individual samples** to indicate how much the binned interval overlaps with one or more CNVs in one sample.

# Calculate CNV frequency 

## `segtoFreq` function

This function computes the binned CNV frequency from segment data.

The parameters of this function:

* `data`: Segment data containing CNV states. The first four columns should represent sample ID, chromosome, start position, and end position, respectively. The fifth column can contain the number of markers or other relevant information. The column representing CNV states (with a column index of 6 or higher) should either contain "DUP" for duplications and "DEL" for deletions, or level-specific CNV states such as "EFO:0030072", "EFO:0030071", "EFO:0020073", and "EFO:0030068", which correspond to high-level duplication, low-level duplication, high-level deletion, and low-level deletion, respectively.
* `cnv_column_idx`: Index of the column specifying CNV state. Default is 6, following the "pgxseg" format used in Progenetix. If the input segment data uses the general `.seg` file format, it might need to be set differently.
* `cohort_name`: A string specifying the cohort name. Default is "unspecified cohort".
* `assembly`: A string specifying the genome assembly version for CNV frequency calculation. Allowed options are "hg19" or "hg38". Default is "hg38".
* `bin_size`: Size of genomic bins used to split the genome, in base pairs (bp). Default is 1,000,000.
* `overlap`: Numeric value defining the amount of overlap between bins and segments considered as bin-specific CNV, in base pairs (bp). Default is 1,000.
* `soft_expansion`: Fraction of `bin_size` to determine merge criteria. During the generation of genomic bins, division starts at the centromere and expands towards the telomeres on both sides. If the size of the last bin is smaller than `soft_expansion` * bin_size, it will be merged with the previous bin. Default is 0.1.

Suppose you have segment data from several biosamples:

```{r}
# access variant data
variants <- pgxLoader(type="g_variants",biosample_id = c("pgxbs-kftvhmz9", "pgxbs-kftvhnqz","pgxbs-kftvhupd"),output="pgxseg")
# only keep segment cnv data
segdata <- variants[variants$variant_type %in% c("DUP","DEL"),]
head(segdata)
```

You can then calculate the CNV frequency for this cohort based on the specified CNV calls. The output will be stored in the "pgxfreq" format.

In the following example, the CNV frequency is calculated based on "DUP" and "DEL" calls:

```{r}
segfreq1 <- segtoFreq(segdata,cnv_column_idx = 6, cohort_name="c1")
segfreq1
```

For level-specific CNV calls, set the `cnv_column_idx` parameter to the column containing CNV states represented as [EFO codes](https://www.ebi.ac.uk/efo/).

```{r}
segfreq2 <- segtoFreq(segdata,cnv_column_idx = 9,cohort_name="c1")
segfreq2
```

# Visualize CNV frequency

## `pgxFreqplot` function

This function provides CNV frequency plots by genome or chromosomes as you request.

The parameters of this function:

* `data`: CNV frequency object returned by `pgxLoader` or `segtoFreq` functions. 
* `chrom`: A vector specifying which chromosomes to plot. If NULL, the plot will cover the entire genome. 
#' If specified, the frequencies are plotted with one panel for each chromosome. Default is NULL.
* `layout`: Number of rows and columns in plot. Only used in plot by chromosome. Default is c(1,1).
* `filters`: Index or string value indicating which filter to plot. The length of filters is limited to one if the parameter `circos` is FALSE. Default is the first filter.
* `circos`: A logical value indicating whether to return a circos plot. If TRUE, it returns a circos plot that can display and compare multiple filters. Default is FALSE.
* `assembly`: A string specifying the genome assembly version to apply to CNV frequency plotting. Allowed options are "hg19" and "hg38". Default is "hg38".

## CNV frequency plot by genome

### Input is `pgxfreq` object

```{r, fig.width=7, fig.height=5}
pgxFreqplot(freq_pgxfreq, filters="NCIT:C3058")
```

### Input is `pgxmatrix` object 

```{r, fig.width=7, fig.height=5}
pgxFreqplot(freq_pgxmatrix, filters = "NCIT:C3493")
```

## CNV frequency plot by chromosomes

```{r, fig.width=7, fig.height=5}
pgxFreqplot(freq_pgxfreq, filters='NCIT:C3058',chrom=c(7,9), layout = c(2,1))  
```

## CNV frequency circos plot 

The circos plot supports multiple group comparison

```{r,fig.width=6, fig.height=6}
pgxFreqplot(freq_pgxmatrix,filters= c("NCIT:C3493","NCIT:C3512"),circos = TRUE) 
```

# Session Info

```{r echo = FALSE}
sessionInfo()
```

