# How to create supercells

## Introduction

This vignette describes the steps to generate supercells for cytometry data using SuperCellCyto R package.

Briefly, supercells are “mini” clusters of cells that are similar in their marker expressions. The motivation behind supercells is that instead of analysing millions of individual cells, you can analyse thousands of supercells, making downstream analysis much faster while maintaining biological interpretability.

See other vignettes for how to:

- Prepare your data for SuperCellCyto
- Use SuperCellCyto for downsampling
- Integrate SuperCellCyto output with SingleCellExperiment objects
- Integrate SuperCellCyto output with Seurat objects

## Installation

You can install stable version of SuperCellCyto from Bioconductor using:

    if (!requireNamespace("BiocManager", quietly = TRUE))
        install.packages("BiocManager")
    BiocManager::install("SuperCellCyto")

For the latest development version, you can install it from GitHub using `pak`:

    if (!requireNamespace("pak", quietly = TRUE))
        install.packages("pak")
    pak::install_github("phipsonlab/SuperCellCyto")

## Preparing your dataset

The function which creates supercells is called `runSuperCellCyto`, and it operates on a `data.table` object, an enhanced version of R native `data.frame`.

In addition to needing the data stored in a `data.table` object it also requires:

1. The markers you will be using to create supercells to have been appropriately transformed, typically using either arcsinh transformation or linear binning (using FlowJo). `runSuperCellCyto` does not perform any data transformation or scaling.
2. The object to have a column denoting the unique ID of each cell. You most likely have to create this column yourself, and it can simply just be a numerical value ranging from 1 to however many cells you have in your data.
3. The object to have a column denoting the biological sample each cell comes from. This column is critical to ensure that cells from different samples will not be mixed in a supercell.

If you are not sure how to import CSV or FCS files into `data.table` object, and/or how to subsequently prepare the object ready for SuperCellCyto, please consult this vignette. In that vignette, we also provide an explanation behind why we need to have the cell ID and sample column.

For this vignette, we will simulate some toy data using the `simCytoData` function. Specifically, we will simulate 15 markers and 3 samples, with each sample containing 10,000 cells. Hence in total, we will have a toy dataset containing 15 markers and 30,000 cells.

    n_markers <- 15
    n_samples <- 3
    dat <- simCytoData(nmarkers = n_markers, ncells = rep(10000, n_samples))
    head(dat)
    #>     Marker_1 Marker_2 Marker_3  Marker_4 Marker_5  Marker_6 Marker_7 Marker_8
    #>        <num>    <num>    <num>     <num>    <num>     <num>    <num>    <num>
    #> 1:  9.315074 13.39828 6.397198  9.479054 10.55900  9.740150 3.614300 16.29052
    #> 2:  9.700245 13.78750 6.744832 10.716371 11.37244 10.759087 6.136447 16.77705
    #> 3: 10.906566 14.59839 5.725258  9.435580 10.66876 11.872051 5.758289 17.72742
    #> 4: 10.619693 12.57775 4.978187  9.295793 10.22006 10.418023 6.114746 18.73151
    #> 5: 10.979242 14.84759 5.873013 11.041803 12.52420 10.384525 5.755281 17.51018
    #> 6: 10.429068 14.51718 5.831599  9.907831 11.56802  9.815596 5.859096 17.68116
    #>     Marker_9 Marker_10 Marker_11 Marker_12 Marker_13 Marker_14 Marker_15
    #>        <num>     <num>     <num>     <num>     <num>     <num>     <num>
    #> 1: 10.200603  18.02635  9.296707 10.487740  19.40476  16.38931  15.68835
    #> 2:  8.873645  18.61807  7.556627  9.453701  19.68330  17.43861  14.48696
    #> 3:  8.303948  18.14428  8.789046  8.545920  20.86251  17.79681  14.70041
    #> 4:  9.399640  18.12874  9.856992  7.424088  19.37119  18.54269  17.07235
    #> 5:  8.925056  18.22276  7.463695  9.515401  19.04910  17.53389  14.29022
    #> 6:  9.427662  16.90242 10.021131  8.712085  20.70994  19.32909  16.30072
    #>      Sample Cell_Id
    #>      <char>  <char>
    #> 1: Sample_1  Cell_1
    #> 2: Sample_1  Cell_2
    #> 3: Sample_1  Cell_3
    #> 4: Sample_1  Cell_4
    #> 5: Sample_1  Cell_5
    #> 6: Sample_1  Cell_6

For our toy dataset, we will transform our data using arcsinh transformation. We will use the base R `asinh` function to do this:

    # Specify which columns are the markers to transform
    marker_cols <- paste0("Marker_", seq_len(n_markers))
    # The co-factor for arc-sinh
    cofactor <- 5

    # Do the transformation
    dat_asinh <- asinh(dat[, marker_cols, with = FALSE] / cofactor)

    # Rename the new columns
    marker_cols_asinh <- paste0(marker_cols, "_asinh")
    names(dat_asinh) <- marker_cols_asinh

    # Add them our previously loaded data
    dat <- cbind(dat, dat_asinh)

    head(dat[, marker_cols_asinh, with = FALSE])
    #>    Marker_1_asinh Marker_2_asinh Marker_3_asinh Marker_4_asinh Marker_5_asinh
    #>             <num>          <num>          <num>          <num>          <num>
    #> 1:       1.380640       1.711963      1.0658526       1.396046       1.492544
    #> 2:       1.416499       1.738838      1.1079571       1.505933       1.560058
    #> 3:       1.521900       1.792736      0.9803152       1.391982       1.501899
    #> 4:       1.497727       1.652991      0.8782854       1.378815       1.463146
    #> 5:       1.527941       1.808763      0.9996121       1.533115       1.649027
    #> 6:       1.481367       1.787459      0.9942318       1.435361       1.575689
    #>    Marker_6_asinh Marker_7_asinh Marker_8_asinh Marker_9_asinh Marker_10_asinh
    #>             <num>          <num>          <num>          <num>           <num>
    #> 1:       1.420149      0.6712937       1.897053       1.461435        1.994245
    #> 2:       1.509539      1.0333268       1.925221       1.338102        2.025403
    #> 3:       1.599551      0.9846535       1.978142       1.280768        2.000530
    #> 4:       1.480411      1.0305824       2.031272       1.388611        1.999704
    #> 5:       1.477509      0.9842591       1.966280       1.343138        2.004692
    #> 6:       1.427019      0.9978064       1.975627       1.391240        1.932358
    #>    Marker_11_asinh Marker_12_asinh Marker_13_asinh Marker_14_asinh
    #>              <num>           <num>           <num>           <num>
    #> 1:        1.378902        1.486428        2.065427        1.902834
    #> 2:        1.201029        1.393677        2.079235        1.962342
    #> 3:        1.329766        1.305468        2.135723        1.981902
    #> 4:        1.430771        1.186312        2.063751        2.021486
    #> 5:        1.190729        1.399432        2.047524        1.967581
    #> 6:        1.445524        1.322129        2.128586        2.061644
    #>    Marker_15_asinh
    #>              <num>
    #> 1:        1.861105
    #> 2:        1.785490
    #> 3:        1.799326
    #> 4:        1.941954
    #> 5:        1.772574
    #> 6:        1.897651

We will also create a column Cell_id_dummy which uniquely identify each cell. It will have values such as `Cell_1, Cell_2,` all the way until `Cell_x` where x is the number of cells in the dataset.

    dat$Cell_id_dummy <- paste0("Cell_", seq_len(nrow(dat)))
    head(dat$Cell_id_dummy, n = 10)
    #>  [1] "Cell_1"  "Cell_2"  "Cell_3"  "Cell_4"  "Cell_5"  "Cell_6"  "Cell_7"
    #>  [8] "Cell_8"  "Cell_9"  "Cell_10"

By default, the `simCytoData` function will generate cells for multiple samples, and that the resulting `data.table` object will already have a column called Sample that denotes the sample the cells come from.

    unique(dat$Sample)
    #> [1] "Sample_1" "Sample_2" "Sample_3"

Let’s take note of the sample and cell id column for later.

    sample_col <- "Sample"
    cell_id_col <- "Cell_id_dummy"

## Creating supercells

Now that we have our data, let’s create some supercells. To do this, we will use `runSuperCellCyto` function and pass the markers, sample and cell ID columns as parameters.

The reason why we need to specify the markers is because the function will create supercells based on only the expression of those markers. We highly recommend creating supercells using all markers in your data, let that be cell type or cell state markers. However, if for any reason you only want to only use a subset of the markers in your data, then make sure you specify them in a vector that you later pass to `runSuperCellCyto` function.

For this tutorial, we will use all the arcsinh transformed markers in the toy data.

    supercells <- runSuperCellCyto(
        dt = dat,
        markers = marker_cols_asinh,
        sample_colname = sample_col,
        cell_id_colname = cell_id_col
    )

Let’s dig deeper into the object it created:

    class(supercells)
    #> [1] "list"

It is a list containing 3 elements:

    names(supercells)
    #> [1] "supercell_expression_matrix" "supercell_cell_map"
    #> [3] "supercell_object"

### Supercell object

The `supercell_object` contains the metadata used to create the supercells. It is a list, and each element contains the metadata used to create the supercells for a sample. This will come in handy if we need to either regenerate the supercells using different gamma values (so we get more or less supercells) or do some debugging later down the line. More on regenerating supercells on Controlling supercells granularity section below.

### Supercell expression matrix

The `supercell_expression_matrix` contains the marker expression of each supercell. These are calculated by taking the average of the marker expression of all the cells contained within a supercell.

    head(supercells$supercell_expression_matrix)
    #>    Marker_1_asinh Marker_2_asinh Marker_3_asinh Marker_4_asinh Marker_5_asinh
    #>             <num>          <num>          <num>          <num>          <num>
    #> 1:       1.588044       1.726726      0.9493814       1.410140       1.563313
    #> 2:       1.523931       1.708795      0.9125682       1.477365       1.604081
    #> 3:       1.621819       1.750462      0.9519644       1.433920       1.533079
    #> 4:       1.385402       1.739363      1.0208867       1.432065       1.421143
    #> 5:       1.519851       1.730332      0.9271164       1.471414       1.541534
    #> 6:       1.526475       1.772485      1.0740442       1.294606       1.404883
    #>    Marker_6_asinh Marker_7_asinh Marker_8_asinh Marker_9_asinh Marker_10_asinh
    #>             <num>          <num>          <num>          <num>           <num>
    #> 1:       1.447923      0.9930169       1.961236       1.321826        1.986173
    #> 2:       1.609267      0.9673037       1.962105       1.368044        1.983764
    #> 3:       1.538806      0.8541165       1.975796       1.264504        1.997170
    #> 4:       1.554971      0.9251693       1.949919       1.377190        1.973525
    #> 5:       1.498982      0.7287037       1.975099       1.388573        1.989190
    #> 6:       1.525154      1.0126650       1.973239       1.350585        1.990486
    #>    Marker_11_asinh Marker_12_asinh Marker_13_asinh Marker_14_asinh
    #>              <num>           <num>           <num>           <num>
    #> 1:        1.283881        1.480768        2.077761        2.014966
    #> 2:        1.426610        1.360709        2.091171        2.016872
    #> 3:        1.330409        1.380269        2.092923        2.020688
    #> 4:        1.403115        1.422075        2.097608        2.014596
    #> 5:        1.215406        1.313242        2.073853        2.016238
    #> 6:        1.347591        1.468077        2.106449        2.029260
    #>    Marker_15_asinh   Sample                 SuperCellId
    #>              <num>   <char>                      <char>
    #> 1:        1.836279 Sample_1 SuperCell_1_Sample_Sample_1
    #> 2:        1.838550 Sample_1 SuperCell_2_Sample_Sample_1
    #> 3:        1.843024 Sample_1 SuperCell_3_Sample_Sample_1
    #> 4:        1.842856 Sample_1 SuperCell_4_Sample_Sample_1
    #> 5:        1.833321 Sample_1 SuperCell_5_Sample_Sample_1
    #> 6:        1.863906 Sample_1 SuperCell_6_Sample_Sample_1

Therein, we will have the following columns:

1. All the markers we previously specified in the `markers_col` variable. In this example, they are the arcsinh transformed markers in our toy data.
2. A column (`Sample` in this case) denoting which sample a supercell belongs to, (note the column name is the same as what is stored in `sample_col` variable).
3. The `SuperCellId` column denoting the unique ID of the supercell.

#### SuperCellId

Let’s have a look at `SuperCellId`:

    head(unique(supercells$supercell_expression_matrix$SuperCellId))
    #> [1] "SuperCell_1_Sample_Sample_1" "SuperCell_2_Sample_Sample_1"
    #> [3] "SuperCell_3_Sample_Sample_1" "SuperCell_4_Sample_Sample_1"
    #> [5] "SuperCell_5_Sample_Sample_1" "SuperCell_6_Sample_Sample_1"

Let’s break down one of them, `SuperCell_1_Sample_Sample_1`. `SuperCell_1` is a numbering (1 to however many supercells there are in a sample) used to uniquely identify each supercell in a sample. Notably, you may encounter this (`SuperCell_1`, `SuperCell_2`) being repeated across different samples, e.g.,

    supercell_ids <- unique(supercells$supercell_expression_matrix$SuperCellId)
    supercell_ids[grep("SuperCell_1_", supercell_ids)]
    #> [1] "SuperCell_1_Sample_Sample_1" "SuperCell_1_Sample_Sample_2"
    #> [3] "SuperCell_1_Sample_Sample_3"

While these 3 supercells’ id are pre-fixed with `SuperCell_1`, it does not make them equal to one another! `SuperCell_1_Sample_Sample_1` will only contain cells from `Sample_1` while `SuperCell_1_Sample_Sample_2` will only contain cells from `Sample_2`.

By now, you may have noticed that we appended the sample name into each supercell id. This aids in differentiating the supercells in different samples.

### Supercell cell map

`supercell_cell_map` maps each cell in our dataset to the supercell it belongs to.

    head(supercells$supercell_cell_map)
    #>                      SuperCellID CellId   Sample
    #>                           <char> <char>   <char>
    #> 1:  SuperCell_81_Sample_Sample_1 Cell_1 Sample_1
    #> 2: SuperCell_215_Sample_Sample_1 Cell_2 Sample_1
    #> 3:  SuperCell_32_Sample_Sample_1 Cell_3 Sample_1
    #> 4:  SuperCell_33_Sample_Sample_1 Cell_4 Sample_1
    #> 5:  SuperCell_32_Sample_Sample_1 Cell_5 Sample_1
    #> 6:  SuperCell_13_Sample_Sample_1 Cell_6 Sample_1

This map is very useful if we later need to expand the supercells out. Additionally, this is also the reason why we need to have a column in the dataset which uniquely identify each cell.

## Running `runSuperCellCyto` in parallel

By default, `runSuperCellCyto` will process each sample one after the other. As each sample is processed independent of one another, strictly speaking, we can process all of them in parallel.

To do this, we need to:

1. Create a `BiocParallelParam` object from the BiocParallel package. This object can either be of type `MulticoreParam`or `SnowParam`. We highly recommend consulting their vignette for more information.
2. Set the number of tasks for the `BiocParallelParam` object to the number of samples we have in the dataset.
3. Set the `load_balancing` parameter for `runSuperCellCyto` function to TRUE. This is to ensure even distribution of the supercell creation jobs. As each sample will be processed by a parallel job, we don’t want a job that processs large sample to also be assigned other smaller samples if possible. If you want to know more how this feature works, please refer to our manuscript.

    supercell_par <- runSuperCellCyto(
        dt = dat,
        markers = marker_cols_asinh,
        sample_colname = sample_col,
        cell_id_colname = cell_id_col,
        BPPARAM = MulticoreParam(tasks = n_samples),
        load_balancing = TRUE
    )

## Controlling supercells granularity

This is described in the `runSuperCellCyto` function’s documentation, but let’s briefly go through it here.

The `runSuperCellCyto` function is equipped with various parameters which can be customised to alter the composition of the supercells. The one that is very likely to be used the most is the gamma parameter, denoted as `gam` in the function. By default, the value for `gam` is set to 20, which we found work well for most cases.

The gamma parameter controls how many supercells to generate, and indirectly, how many cells are captured within each supercell. This parameter is resolved into the following formula `gamma=n_cells/n_supercells` where `n_cell` denotes the number of cells and `n_supercells` denotes the number of supercells.

In general, the larger gamma parameter is set to, the less supercells we will get. Say for instance we have 10,000 cells. If gamma is set to 10, we will end up with about 1,000 supercells, whereas if gamma is set to 50, we will end up with about 200 supercells.

You may have noticed, after reading the sections above, `runSuperCellCyto` is ran on each sample independent of each other, and that we can only set 1 value as the gamma parameter. Indeed, for now, the same gamma value will be used across all samples, and that depending on how many cells we have in each sample, we will end up with different number of supercells for each sample. For instance, say we have 10,000 cells for sample 1, and 100,000 cells for sample 2. If gamma is set to 10, for sample 1, we will get 1,000 supercells (10,000/10) while for sample 2, we will get 10,000 supercells (100,000/10).

Do note: whatever gamma value you chose, you should not expect each supercell to contain exactly the same number of cells. This behaviour is intentional to ensure rare cell types are not intermixed with non-rare cell types in a supercell.

### Adjusting gamma value after one run of runSuperCellCyto

If you have run `runSuperCellCyto` once and have not discarded the SuperCell object it generated (no serious, please don’t!), you can use the object to quickly regenerate supercells using different gamma values.

As an example, using the SuperCell object we have generated for our toy dataset, we will regenerate the supercells using gamma of 10 and 50. The function to do this is `recomputeSupercells`. We will store the output in a list, one element per gamma value.

    addt_gamma_vals <- c(10, 50)
    supercells_addt_gamma <- lapply(addt_gamma_vals, function(gam) {
        recomputeSupercells(
            dt = dat,
            sc_objects = supercells$supercell_object,
            markers = marker_cols_asinh,
            sample_colname = sample_col,
            cell_id_colname = cell_id_col,
            gam = gam
        )
    })

We should end up with a list containing 2 elements. The 1st element contains supercells generated using gamma = 10, and the 2nd contains supercells generated using gamma = 50.

    supercells_addt_gamma[[1]]
    #> $supercell_expression_matrix
    #>       Marker_1_asinh Marker_2_asinh Marker_3_asinh Marker_4_asinh
    #>                <num>          <num>          <num>          <num>
    #>    1:       1.619882      1.7109131      0.9374779       1.408611
    #>    2:       1.558627      1.7881596      0.8978865       1.380494
    #>    3:       1.382389      1.7643805      0.8299199       1.504205
    #>    4:       1.532624      1.7508302      0.7168823       1.391255
    #>    5:       1.455634      1.7355615      0.8593505       1.395789
    #>   ---
    #> 2996:       1.594267      1.1391835      1.9834430       1.330684
    #> 2997:       1.533164      1.0262835      1.9248983       1.409359
    #> 2998:       1.510185      1.0049125      1.9488959       1.397837
    #> 2999:       1.563628      0.7849512      1.9619143       1.569661
    #> 3000:       1.582692      1.0883017      1.9764000       1.504162
    #>       Marker_5_asinh Marker_6_asinh Marker_7_asinh Marker_8_asinh
    #>                <num>          <num>          <num>          <num>
    #>    1:       1.552914       1.407008      0.9662059       1.966676
    #>    2:       1.462402       1.384449      1.0093953       1.978287
    #>    3:       1.492840       1.417487      1.0161855       1.947490
    #>    4:       1.495843       1.505933      1.0423500       1.924113
    #>    5:       1.415179       1.596666      0.9906499       1.953394
    #>   ---
    #> 2996:       1.248963       1.921573      2.0600027       1.141246
    #> 2997:       1.351964       1.849494      2.0344957       1.036142
    #> 2998:       1.530825       1.773541      2.0519734       1.303504
    #> 2999:       1.232730       1.737078      2.1027488       1.097485
    #> 3000:       1.444470       1.824790      2.0735766       1.257505
    #>       Marker_9_asinh Marker_10_asinh Marker_11_asinh Marker_12_asinh
    #>                <num>           <num>           <num>           <num>
    #>    1:       1.297519        1.976380        1.363762        1.480512
    #>    2:       1.369874        2.025192        1.154531        1.471561
    #>    3:       1.316895        2.010657        1.319326        1.417575
    #>    4:       1.542978        1.990643        1.277690        1.266279
    #>    5:       1.308852        2.018792        1.437966        1.571309
    #>   ---
    #> 2996:       1.449196        1.767504        1.384806        1.615038
    #> 2997:       1.492713        1.738564        1.501584        1.535362
    #> 2998:       1.535457        1.769083        1.378444        1.533670
    #> 2999:       1.482284        1.832032        1.321298        1.428864
    #> 3000:       1.529954        1.792146        1.316376        1.436527
    #>       Marker_13_asinh Marker_14_asinh Marker_15_asinh   Sample
    #>                 <num>           <num>           <num>   <char>
    #>    1:        2.069682        2.025534        1.825203 Sample_1
    #>    2:        2.103563        2.011352        1.859572 Sample_1
    #>    3:        2.098900        2.003430        1.858946 Sample_1
    #>    4:        2.099810        1.981980        1.828336 Sample_1
    #>    5:        2.092745        2.012400        1.842924 Sample_1
    #>   ---
    #> 2996:        1.575348        1.958259        1.693637 Sample_3
    #> 2997:        1.440329        1.890399        1.668224 Sample_3
    #> 2998:        1.592834        1.967538        1.528899 Sample_3
    #> 2999:        1.627613        1.921720        1.668712 Sample_3
    #> 3000:        1.510413        1.936107        1.617106 Sample_3
    #>                          SuperCellId
    #>                               <char>
    #>    1:    SuperCell_1_Sample_Sample_1
    #>    2:    SuperCell_2_Sample_Sample_1
    #>    3:    SuperCell_3_Sample_Sample_1
    #>    4:    SuperCell_4_Sample_Sample_1
    #>    5:    SuperCell_5_Sample_Sample_1
    #>   ---
    #> 2996:  SuperCell_996_Sample_Sample_3
    #> 2997:  SuperCell_997_Sample_Sample_3
    #> 2998:  SuperCell_998_Sample_Sample_3
    #> 2999:  SuperCell_999_Sample_Sample_3
    #> 3000: SuperCell_1000_Sample_Sample_3
    #>
    #> $supercell_cell_map
    #>                          SuperCellID     CellId   Sample
    #>                               <char>     <char>   <char>
    #>     1: SuperCell_441_Sample_Sample_1     Cell_1 Sample_1
    #>     2: SuperCell_641_Sample_Sample_1     Cell_2 Sample_1
    #>     3: SuperCell_469_Sample_Sample_1     Cell_3 Sample_1
    #>     4: SuperCell_897_Sample_Sample_1     Cell_4 Sample_1
    #>     5: SuperCell_404_Sample_Sample_1     Cell_5 Sample_1
    #>    ---
    #> 29996: SuperCell_562_Sample_Sample_3 Cell_29996 Sample_3
    #> 29997: SuperCell_174_Sample_Sample_3 Cell_29997 Sample_3
    #> 29998: SuperCell_161_Sample_Sample_3 Cell_29998 Sample_3
    #> 29999: SuperCell_342_Sample_Sample_3 Cell_29999 Sample_3
    #> 30000: SuperCell_627_Sample_Sample_3 Cell_30000 Sample_3

The output generated by `recomputeSupercells` is essentially a list:

1. `supercell_expression_matrix`: A data.table object that contains the marker expression for each supercell.
2. `supercell_cell_map`: A data.table that maps each cell to its corresponding supercell.

As mentioned before, gamma dictates the granularity of supercells. Compared to the previous run where gamma was set to 20, we should get more supercells for gamma = 10, and less for gamma = 50. Let’s see if that’s the case.

    n_supercells_gamma20 <- nrow(supercells$supercell_expression_matrix)
    n_supercells_gamma10 <- nrow(
        supercells_addt_gamma[[1]]$supercell_expression_matrix
    )
    n_supercells_gamma50 <- nrow(
        supercells_addt_gamma[[2]]$supercell_expression_matrix
    )

    n_supercells_gamma10 > n_supercells_gamma20
    #> [1] TRUE

    n_supercells_gamma50 < n_supercells_gamma20
    #> [1] TRUE

### Specifying different gamma value for different samples

In the future, we may add the ability to specify different `gam` value for different samples. For now, if we want to do this, we will need to break down our data into multiple `data.table` objects, each containing data from 1 sample, and run `runSuperCellCyto` function on each of them with different `gam` parameter value. Something like the following:

    n_markers <- 10
    dat <- simCytoData(nmarkers = n_markers)
    markers_col <- paste0("Marker_", seq_len(n_markers))
    sample_col <- "Sample"
    cell_id_col <- "Cell_Id"

    samples <- unique(dat[[sample_col]])
    gam_values <- c(10, 20, 10)

    supercells_diff_gam <- lapply(seq_len(length(samples)), function(i) {
        sample <- samples[i]
        gam <- gam_values[i]
        dat_samp <- dat[dat$Sample == sample, ]
        supercell_samp <- runSuperCellCyto(
            dt = dat_samp,
            markers = markers_col,
            sample_colname = sample_col,
            cell_id_colname = cell_id_col,
            gam = gam
        )
        return(supercell_samp)
    })

Subsequently, to extract and combine the `supercell_expression_matrix` and `supercell_cell_map`, we will need to use `rbind`:

    supercell_expression_matrix <- do.call(
        "rbind", lapply(
            supercells_diff_gam, function(x) x[["supercell_expression_matrix"]]
        )
    )

    supercell_cell_map <- do.call(
        "rbind", lapply(
            supercells_diff_gam, function(x) x[["supercell_cell_map"]]
        )
    )

    rbind(
        head(supercell_expression_matrix, n = 3),
        tail(supercell_expression_matrix, n = 3)
    )
    #>     Marker_1  Marker_2  Marker_3  Marker_4  Marker_5 Marker_6  Marker_7
    #>        <num>     <num>     <num>     <num>     <num>    <num>     <num>
    #> 1:  5.969757 11.976861  5.407556 11.348353 13.748592 19.62255  9.798173
    #> 2:  5.432511  8.884816  4.811551 10.703703 14.737980 18.66859  9.972150
    #> 3:  5.720493 11.924626  4.607327 11.403772 14.379121 18.25234 11.932391
    #> 4: 16.949561  7.724016 20.441084  5.902415  6.655565 14.73518  7.581732
    #> 5: 20.181256  6.935856 20.734140  6.848722  5.763488 14.11917  8.503059
    #> 6: 18.106466  4.999725 20.552179  7.812968  6.222262 13.37958  5.837531
    #>     Marker_8 Marker_9 Marker_10   Sample                   SuperCellId
    #>        <num>    <num>     <num>   <char>                        <char>
    #> 1: 11.240442 20.51260  14.24034 Sample_1   SuperCell_1_Sample_Sample_1
    #> 2: 10.685672 19.80530  15.40499 Sample_1   SuperCell_2_Sample_Sample_1
    #> 3:  9.304139 18.53660  15.71860 Sample_1   SuperCell_3_Sample_Sample_1
    #> 4: 12.193218 17.37633  10.95094 Sample_2 SuperCell_498_Sample_Sample_2
    #> 5: 14.533176 16.97182  11.20817 Sample_2 SuperCell_499_Sample_Sample_2
    #> 6: 12.284387 17.46486  12.21454 Sample_2 SuperCell_500_Sample_Sample_2

    rbind(head(supercell_cell_map, n = 3), tail(supercell_cell_map, n = 3))
    #>                      SuperCellID     CellId   Sample
    #>                           <char>     <char>   <char>
    #> 1: SuperCell_403_Sample_Sample_1     Cell_1 Sample_1
    #> 2: SuperCell_648_Sample_Sample_1     Cell_2 Sample_1
    #> 3: SuperCell_998_Sample_Sample_1     Cell_3 Sample_1
    #> 4:  SuperCell_82_Sample_Sample_2 Cell_19998 Sample_2
    #> 5: SuperCell_114_Sample_Sample_2 Cell_19999 Sample_2
    #> 6: SuperCell_398_Sample_Sample_2 Cell_20000 Sample_2

## Mixing cells from different samples in a supercell

If for whatever reason you don’t mind (or perhaps more to the point want) each supercell to contain cells from different biological samples, you still need to have the sample column in your `data.table`. However, what you need to do is essentially set the value in the column to exactly one unique value. That way, SuperCellCyto will treat all cells as coming from one sample.

Just note, the parallel processing feature in SuperCellCyto won’t work for this as you will essentially only have 1 sample and nothing for SuperCellCyto to parallelise.

## I have more cells than RAM in my computer

Is your dataset so huge that you are constantly running out of RAM when generating supercells? This thing happens and we have a solution for it.

Since supercells are generated for each sample independent of others you can easily break up the process. For example:

1. Load up a subset of the samples (say 1-10).
2. Generate supercells for those samples.
3. Save the output using the qs package.
4. Extract the `supercell_expression_matrix` and `supercell_cell_map`, and export them out as a csv file using `data.table`’s `fwrite` function.
5. Load another sets of samples (say 11-20), rinse and repeat step 2-4.

Once you have processed all the samples, you can then load all `supercell_expression_matrix` and `supercell_cell_map` csv files and analyse them.

If you want to regenerate the supercells using different gamma values, load the relevant output saved using the qs package and the relevant data (remember to note which output belongs to which sets of samples!), and run `recomputeSupercells` function.

## Session information

    sessionInfo()
    #> R version 4.6.0 (2026-04-24)
    #> Platform: x86_64-pc-linux-gnu
    #> Running under: Ubuntu 24.04.4 LTS
    #>
    #> Matrix products: default
    #> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
    #> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
    #>
    #> locale:
    #>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
    #>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
    #>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
    #>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
    #>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
    #> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
    #>
    #> time zone: Etc/UTC
    #> tzcode source: system (glibc)
    #>
    #> attached base packages:
    #> [1] parallel  stats4    stats     graphics  grDevices utils     datasets
    #> [8] methods   base
    #>
    #> other attached packages:
    #>  [1] BiocParallel_1.46.0         future_1.70.0
    #>  [3] Seurat_5.5.0                SeuratObject_5.4.0
    #>  [5] sp_2.2-1                    qs2_0.2.0
    #>  [7] bluster_1.22.0              scater_1.40.0
    #>  [9] ggplot2_4.0.3               BiocSingular_1.28.0
    #> [11] scran_1.40.0                scuttle_1.22.0
    #> [13] SingleCellExperiment_1.34.0 SummarizedExperiment_1.42.0
    #> [15] Biobase_2.72.0              GenomicRanges_1.64.0
    #> [17] Seqinfo_1.2.0               IRanges_2.46.0
    #> [19] S4Vectors_0.50.0            BiocGenerics_0.58.0
    #> [21] generics_0.1.4              MatrixGenerics_1.24.0
    #> [23] matrixStats_1.5.0           SuperCellCyto_1.2.0
    #> [25] flowCore_2.24.0             data.table_1.18.2.1
    #> [27] BiocStyle_2.40.0
    #>
    #> loaded via a namespace (and not attached):
    #>   [1] RcppAnnoy_0.0.23       splines_4.6.0          later_1.4.8
    #>   [4] tibble_3.3.1           polyclip_1.10-7        fastDummies_1.7.6
    #>   [7] lifecycle_1.0.5        edgeR_4.10.0           globals_0.19.1
    #>  [10] lattice_0.22-9         MASS_7.3-65            magrittr_2.0.5
    #>  [13] limma_3.68.0           plotly_4.12.0          sass_0.4.10
    #>  [16] rmarkdown_2.31         jquerylib_0.1.4        yaml_2.3.12
    #>  [19] metapod_1.20.0         httpuv_1.6.17          otel_0.2.0
    #>  [22] sctransform_0.4.3      spam_2.11-3            spatstat.sparse_3.1-0
    #>  [25] reticulate_1.46.0      cowplot_1.2.0          pbapply_1.7-4
    #>  [28] buildtools_1.0.0       RColorBrewer_1.1-3     abind_1.4-8
    #>  [31] Rtsne_0.17             purrr_1.2.2            ggrepel_0.9.8
    #>  [34] irlba_2.3.7            listenv_0.10.1         spatstat.utils_3.2-2
    #>  [37] maketools_1.3.2        goftest_1.2-3          RSpectra_0.16-2
    #>  [40] spatstat.random_3.4-5  dqrng_0.4.1            fitdistrplus_1.2-6
    #>  [43] parallelly_1.47.0      codetools_0.2-20       DelayedArray_0.38.1
    #>  [46] tidyselect_1.2.1       farver_2.1.2           ScaledMatrix_1.20.0
    #>  [49] viridis_0.6.5          spatstat.explore_3.8-0 jsonlite_2.0.0
    #>  [52] BiocNeighbors_2.6.0    progressr_0.19.0       ggridges_0.5.7
    #>  [55] survival_3.8-6         tools_4.6.0            ica_1.0-3
    #>  [58] Rcpp_1.1.1-1.1         glue_1.8.1             gridExtra_2.3
    #>  [61] SparseArray_1.12.0     xfun_0.57              dplyr_1.2.1
    #>  [64] withr_3.0.2            BiocManager_1.30.27    fastmap_1.2.0
    #>  [67] digest_0.6.39          rsvd_1.0.5             R6_2.6.1
    #>  [70] mime_0.13              scattermore_1.2        tensor_1.5.1
    #>  [73] spatstat.data_3.1-9    tidyr_1.3.2            FNN_1.1.4.1
    #>  [76] httr_1.4.8             htmlwidgets_1.6.4      S4Arrays_1.12.0
    #>  [79] uwot_0.2.4             pkgconfig_2.0.3        gtable_0.3.6
    #>  [82] RProtoBufLib_2.24.0    lmtest_0.9-40          S7_0.2.2
    #>  [85] XVector_0.52.0         sys_3.4.3              htmltools_0.5.9
    #>  [88] dotCall64_1.2          scales_1.4.0           png_0.1-9
    #>  [91] spatstat.univar_3.1-7  knitr_1.51             reshape2_1.4.5
    #>  [94] nlme_3.1-169           cachem_1.1.0           zoo_1.8-15
    #>  [97] stringr_1.6.0          KernSmooth_2.23-26     miniUI_0.1.2
    #> [100] vipor_0.4.7            pillar_1.11.1          grid_4.6.0
    #> [103] vctrs_0.7.3            RANN_2.6.2             promises_1.5.0
    #> [106] stringfish_0.19.0      cytolib_2.24.0         beachmat_2.28.0
    #> [109] xtable_1.8-8           cluster_2.1.8.2        beeswarm_0.4.0
    #> [112] evaluate_1.0.5         cli_3.6.6              locfit_1.5-9.12
    #> [115] compiler_4.6.0         rlang_1.2.0            future.apply_1.20.2
    #> [118] labeling_0.4.3         plyr_1.8.9             ggbeeswarm_0.7.3
    #> [121] stringi_1.8.7          deldir_2.0-4           viridisLite_0.4.3
    #> [124] SuperCell_1.1          lazyeval_0.2.3         spatstat.geom_3.7-3
    #> [127] Matrix_1.7-5           RcppHNSW_0.6.0         patchwork_1.3.2
    #> [130] statmod_1.5.1          shiny_1.13.0           ROCR_1.0-12
    #> [133] igraph_2.3.0           RcppParallel_5.1.11-2  bslib_0.10.0