---
title: "Pipeline Use of dnaEPICO"
author:
    -
        name: Paul Ruiz
        affiliation:
            - Queensland University of Technology
        email: ruizpint@qut.edu.au
output:
    BiocStyle::html_document:
        self_contained: yes
        toc: true
        toc_float: true
        toc_depth: 2
date: "`r Sys.Date()`"
package: dnaEPICO
bibliography: dnaEPICO.bib
vignette: >
    %\VignetteIndexEntry{3. Pipeline Use of dnaEPICO}
    %\VignetteEncoding{UTF-8}
    %\VignetteEngine{knitr::rmarkdown}
---

```{r setup, include = FALSE}
startTime <- Sys.time()
knitr::opts_chunk$set(
    collapse = TRUE,
    comment = "#>",
    crop = NULL,
    message = FALSE,
    warning = FALSE
)

library(dnaEPICO)

can_run_preprocessing <- requireNamespace("minfiData", quietly = TRUE) &&
    requireNamespace("IlluminaHumanMethylation450kmanifest", quietly = TRUE) &&
    requireNamespace(
        "IlluminaHumanMethylation450kanno.ilmn12.hg19",
        quietly = TRUE
    )

can_run_sva <- can_run_preprocessing
can_run_glm <- requireNamespace(
    "IlluminaHumanMethylation450kanno.ilmn12.hg19",
    quietly = TRUE
)
can_run_lme <- can_run_glm && requireNamespace("lmerTest", quietly = TRUE)
```

# Introduction

This vignette describes the file-based `dnaEPICO` workflow. Functions return
structured objects and write files when `saveOutputs = TRUE`. Use this workflow
for reproducible directories, Make execution, and resumable analyses on local
or HPC systems.

# Citation

The `r Biocpkg("dnaEPICO")` package uses methods from several external
packages. Because no single manuscript describes all components, the guidance
below explains how to cite `dnaEPICO` depending on the functions you use.

This citation guidance is adapted from the vignettes and user guides of
[minfi](https://github.com/hansenlab/minfi/blob/devel/vignettes/minfi.Rmd),
[ENmix](https://github.com/xuz1/ENmix/blob/master/vignettes/ENmix.Rmd), and
[wateRmelon](https://github.com/schalkwyk/wateRmelon).

- If you use `make f3 MODEL=model1`, please cite @minfi; @minfiEPIC; @ENmix;
    @wateRmelon; @SWAN; @funnorm; @noob; @quantile; @ewastools.
- If you use `make f4 MODEL=model1`, please cite @minfi; @minfiEPIC; @ENmix;
    @wateRmelon; @SWAN; @funnorm; @noob; @quantile; @ewastools; @glm2.
- If you use `make f3lme MODEL=model1`, please cite @minfi; @minfiEPIC;
    @ENmix; @wateRmelon; @SWAN; @funnorm; @noob; @quantile; @ewastools; @lme4.
- If you use `make all MODEL=model1`, please cite @minfi; @minfiEPIC; @ENmix;
    @wateRmelon; @SWAN; @funnorm; @noob; @quantile; @ewastools; @glm2; @lme4.

# Required knowledge

`r Biocpkg("dnaEPICO")` is built on core Bioconductor infrastructure for
high-dimensional genomic data, with a focus on Illumina DNA methylation arrays.
This vignette assumes familiarity with a general DNA methylation workflow. For
an introduction, see this tutorial:
<https://paulyrp.github.io/2025-cpgpneurogenomics-workshop/tutorial.html>.
It covers the main concepts and analysis steps.

Preprocessing and quality control are performed using established Bioconductor
tools, including `r Biocpkg("minfi")`, `r Biocpkg("ENmix")`, and
`r Biocpkg("wateRmelon")`. Downstream statistical modelling uses CpG-wise
generalised linear models and longitudinal mixed-effects models fitted with
`lmerTest`/`lme4` or `nlme`. The `nlme` engine can include AR1 or CAR1
within-participant residual correlation. Users are expected to have basic
familiarity with R, command-line execution, and Illumina IDAT file structures.

For an introduction to Bioconductor, see the
[installation guide](https://www.bioconductor.org/install/).

## Probe-exclusion reference files

The lists of excluded probes depend on the Illumina methylation-array platform.

- For **Illumina HumanMethylationEPIC v2.0**, use the cross-reactive
    probe-exclusion file from
    [Peters et al. (2024)](https://doi.org/10.1186/s12864-024-10027-5).

- For **Illumina MethylationEPIC**, also known as the **850k array**, use
    the probe-exclusion resources from
    [Pidsley et al. (2016)](https://doi.org/10.1186/s13059-016-1066-1).
    The supporting files commonly used together are
    `13059_2016_1066_MOESM1_ESM.csv`,
    `13059_2016_1066_MOESM4_ESM.csv`,
    `13059_2016_1066_MOESM5_ESM.csv`, and
    `13059_2016_1066_MOESM6_ESM.csv`.

- For **Illumina HumanMethylation450k**, use the cross-reactive and
    polymorphic probe resources from
    [Chen et al. (2013)](https://doi.org/10.4161/epi.23470).

- Multiple probe-exclusion files can be supplied as a semicolon-separated
    value in `probeExclusionPath`. `dnaEPICO` reads probe IDs from each file,
    uses `probeExclusionIdColumn` when supplied, or otherwise auto-detects
    common probe-ID columns such as `ProbeID`, `TargetID`, `IlmnID`, and
    `Name`. The unique union of all probe IDs is then used to filter the
    normalised object.

- For **EPICv2**, setting `useEpicV2Manifest = TRUE` also retrieves the
    expanded Peters et al. manifest from AnnotationHub resource
    [AH116484](https://bioconductor.org/packages/EPICv2manifest). Probes
    flagged in selected manifest columns are added to the same exclusion set.
    By default, probes flagged by `CH_WGBS_evidence`, `CH_BLAT`, or
    `MissingPos` are removed, while `MismatchPos` is retained unless explicitly
    enabled.

# Installation

Install the development version of `r Biocpkg("dnaEPICO")` and its dependencies
with BiocManager:

```{r install-dnaepico, eval = FALSE}
if (!requireNamespace("BiocManager", quietly = TRUE)) {
    install.packages("BiocManager")
}

BiocManager::install("paulYRP/dnaEPICO")

BiocManager::valid()
```

```{r load-dnaepico, eval = FALSE, message = FALSE}
library("dnaEPICO")
```

# Extract the example Makefile

The package includes a template Makefile that can be copied into a project
directory and adapted to local paths and model definitions.

```{r makefile-example}
makefilePath <- extractMake(destDir = tempdir(), overwrite = TRUE)
basename(makefilePath)
```

After extraction, the file should be edited so that the project-specific paths,
model labels, and cluster settings match the target analysis environment.

Arguments:

- `destDir = tempdir()` writes the template to a temporary directory so the
    example does not modify the working tree.
- `overwrite = TRUE` allows the template to be replaced if it already exists in
    that temporary location.

# Write preprocessingMinfiEwasWater outputs to disk

The file-based route usually starts with `preprocessingMinfiEwasWater()`. This
step writes the filtered `RGSet`, normalized metric matrices, quality-control
figures, and the `phenoLC.csv` file that will be consumed by `svaEnmix()` and
`preprocessingPheno()`.

### Create example input files

The next chunk shows how to recreate the temporary files used in this example.
It writes a small phenotype table to a temporary directory, copies a small set
of IDAT files, and records the probe-exclusion reference used during probe
filtering.

```{r preprocessing-pipeline-inputs, eval = can_run_preprocessing}
preprocessing_inputs <- dnaEPICO:::exampleMinfiIdatInputsDnaEpico(n = 6L)

names(preprocessing_inputs)
preprocessing_inputs$tempDir
basename(preprocessing_inputs$phenoFile)
head(
    basename(list.files(
        preprocessing_inputs$idatFolder,
        full.names = TRUE
    )),
    4
)
basename(preprocessing_inputs$probeExclusionPath)
```

`preprocessing_inputs` is a small list returned by the internal example helper.
`names(preprocessing_inputs)` shows its main elements: `tempDir` is the
temporary working directory, `idatFolder` contains the copied example IDAT
files, `phenoFile` is the phenotype table used by the function, `targets` is the
same phenotype information already loaded into memory, `arrayType` and
`annotationVersion` describe the array platform, and `probeExclusionPath`
points to the probe-exclusion reference used during probe filtering.

```{r preprocessing-pipeline, eval = can_run_preprocessing}
preprocessPipelineResult <- preprocessingMinfiEwasWater(
    phenoFile = preprocessing_inputs$phenoFile,
    idatFolder = preprocessing_inputs$idatFolder,
    outputLogs = file.path(preprocessing_inputs$tempDir, "logs"),
    nSamples = 6,
    SampleID = "Sample_Name",
    arrayType = preprocessing_inputs$arrayType,
    annotationVersion = preprocessing_inputs$annotationVersion,
    scriptLabel = "preprocessingMinfiEwasWater",
    baseDataFolder = file.path(preprocessing_inputs$tempDir, "rData"),
    figureBaseDir = file.path(preprocessing_inputs$tempDir, "figures"),
    detPThreshold = 1,
    normMethods = "quantile",
    sexColumn = "Sex",
    removeSexMismatch = FALSE,
    pvalThreshold = 1,
    chrToRemove = "",
    snpsToRemove = "SBE",
    mafThreshold = 1,
    probeExclusionPath = preprocessing_inputs$probeExclusionPath,
    plotGroupVar = "Sex",
    lcRef = "saliva",
    phenoOrder = "Sample_Name;Sex;Basename;Sentrix_ID;Sentrix_Position",
    lcPhenoDir = file.path(
        preprocessing_inputs$tempDir,
        "data",
        "preprocessingMinfiEwasWater"
    ),
    display = FALSE,
    verbose = FALSE,
    logs = TRUE,
    saveOutputs = TRUE
)

preprocess_paths <- c(
    phenoLC = file.path(
        preprocessing_inputs$tempDir,
        "data",
        "preprocessingMinfiEwasWater",
        "phenoLC.csv"
    ),
    rgset = file.path(
        preprocessing_inputs$tempDir,
        "rData",
        "preprocessingMinfiEwasWater",
        "objects",
        "RGSet.RData"
    ),
    beta = file.path(
        preprocessing_inputs$tempDir,
        "rData",
        "preprocessingMinfiEwasWater",
        "metrics",
        "beta_NomFilt_MSetF_Flt_Rxy_Ds_Rc.RData"
    ),
    qc = file.path(
        preprocessing_inputs$tempDir,
        "figures",
        "preprocessingMinfiEwasWater",
        "qc",
        "quality_control(MSet).tiff"
    )
)

class(preprocessPipelineResult)
basename(preprocess_paths)
file.exists(preprocess_paths)
```

The returned object still has class `dnaEPICO_preprocessingMinfiEwasWater`, but
the printed vectors in this chunk focus on the written outputs. In
`preprocess_paths`, `phenoLC` is the phenotype table with estimated cell
proportions, `rgset` is the filtered `RGSet`, `beta` is the saved beta matrix
used later by phenotype preparation, and `qc` is an example quality-control
figure. `basename(preprocess_paths)` shows the file names, while
`file.exists(preprocess_paths)` confirms that each one was written successfully.

Arguments:

- `phenoFile` and `idatFolder` define the phenotype table and the IDAT folder
    used to start the pipeline.
- `SampleID = "Sample_Name"` is the key used to align phenotype rows with the
    methylation objects.
- `arrayType` and `annotationVersion` define the array manifest and annotation
    used by `minfi`.
- `baseDataFolder`, `figureBaseDir`, and `lcPhenoDir` define where serialized
    objects, figures, and `phenoLC.csv` are written.
- `detPThreshold`, `normMethods`, `pvalThreshold`, `chrToRemove`,
    `snpsToRemove`, `mafThreshold`, and `probeExclusionPath` define the main
    preprocessing and filtering choices.
- `sexColumn` identifies reported sex for comparison with methylation-predicted
    sex. `removeSexMismatch = FALSE` retains and reports confirmed mismatches;
    `TRUE` removes samples only when both values are known and disagree. Missing
    or unknown sex values are retained.
- `plotGroupVar`, `lcRef`, and `phenoOrder` control QC grouping, cell-type
    estimation, and the structure of the exported phenotype table.
- `saveOutputs = TRUE` is what turns this function into the first file-producing
    step of the pipeline.

# Write SVA outputs to disk

The next example shows `svaEnmix()` in file-writing mode. The function still
returns a structured result object, but the `savedFiles` element now records the
main output files written to disk.

### Reuse the temporary files written in Step 1

In the file-based route, `svaEnmix()` consumes the `phenoLC.csv` and
`RGSet.RData` files written by `preprocessingMinfiEwasWater()`. The next chunk
shows how those temporary file paths are reconstructed locally.

```{r sva-pipeline-inputs, eval = can_run_sva, message = FALSE, warning = FALSE}
sva_targets_file <- file.path(
    preprocessing_inputs$tempDir,
    "data",
    "preprocessingMinfiEwasWater",
    "phenoLC.csv"
)
sva_rgset_file <- file.path(
    preprocessing_inputs$tempDir,
    "rData",
    "preprocessingMinfiEwasWater",
    "objects",
    "RGSet.RData"
)

basename(c(sva_targets_file, sva_rgset_file))
file.exists(c(sva_targets_file, sva_rgset_file))
```

`sva_targets_file` and `sva_rgset_file` point to the Step 1 outputs:
`phenoLC.csv`, containing the phenotype table and cell-composition estimates,
and `RGSet.RData`, containing the filtered methylation object.

```{r sva-pipeline, eval = can_run_sva, message = FALSE, warning = FALSE}
svaPipelineResult <- svaEnmix(
    phenoFile = sva_targets_file,
    rgsetData = sva_rgset_file,
    outputLogs = file.path(preprocessing_inputs$tempDir, "logs"),
    SampleID = "Sample_Name",
    arrayType = "IlluminaHumanMethylation450k",
    annotationVersion = "ilmn12.hg19",
    SentrixIDColumn = "Sentrix_ID",
    SentrixPositionColumn = "Sentrix_Position",
    ctrlSvaPercVar = 0.90,
    ctrlSvaFlag = 1,
    scriptLabel = "svaEnmix",
    dataBaseDir = file.path(preprocessing_inputs$tempDir, "data"),
    rBaseDir = file.path(preprocessing_inputs$tempDir, "rData"),
    figureBaseDir = file.path(preprocessing_inputs$tempDir, "figures"),
    display = FALSE,
    verbose = FALSE,
    logs = TRUE,
    saveOutputs = TRUE
)

class(svaPipelineResult$savedFiles)
names(svaPipelineResult$savedFiles)
basename(unlist(svaPipelineResult$savedFiles, use.names = FALSE))
```

The returned `savedFiles` object records the main paths written by this step.
`names(svaPipelineResult$savedFiles)` identifies the output groups: `svaRData`
is the serialized surrogate-variable matrix, `svaCSV` is the CSV version of
that matrix, `phenoWithSva` is the same `phenoLC.csv` after the SVA columns are
appended, and `dataDir` / `rDir` are the parent directories used for the saved
outputs. The `basename(...)` call shows the file names generated from those
paths. The phenotype is validated, written to a temporary file, read back, and
then replaced with rollback protection. Sample identifiers, row order,
cell-composition columns, and all other existing columns must remain unchanged.
Existing PC names are rejected before replacement.

Arguments:

- `phenoFile` and `rgsetData` identify the phenotype table and saved `RGSet`
    produced by the earlier preprocessing step.
- `SampleID`, `SentrixIDColumn`, and `SentrixPositionColumn` specify how
    samples and array positions are identified in the phenotype table.
- `ctrlSvaPercVar = 0.90` keeps enough control-derived surrogate variables to
    explain 90% of the control-probe variance.
- `ctrlSvaFlag = 1` enables the ENmix control-based SVA workflow.
- `dataBaseDir` and `rBaseDir` define where file outputs are written when
    `saveOutputs = TRUE`.
- `display = FALSE` and `verbose = FALSE` keep the console output quiet, while
    `logs = TRUE` writes the log files used by the report.

# Write preprocessingPheno outputs to disk

`preprocessingPheno()` is the step that usually prepares the largest number of
pipeline-ready files because it writes timepoint-specific tables, combined
longitudinal tables, and the Clock Foundation export inputs.

### Create example input files

The next chunk shows how to recreate the temporary phenotype and matrix files
used in this example. The helper writes a phenotype table plus aligned beta,
M-value, and copy-number objects to a temporary directory.

```{r preprocessingPheno-pipeline-inputs, message = FALSE, warning = FALSE}
pheno_inputs <- dnaEPICO:::examplePreprocessingPhenoStateDnaEpico()

names(pheno_inputs)
pheno_inputs$tempDir
basename(c(
    pheno_inputs$phenoPath,
    pheno_inputs$betaPath,
    pheno_inputs$mPath,
    pheno_inputs$cnPath
))
```

`pheno_inputs` is a list that bundles the temporary files and the aligned
in-memory objects used in this stage. `names(pheno_inputs)` shows that it
contains the temporary directory, the phenotype table, the saved beta, M-value,
and copy-number files, and precomputed helper objects such as `metricsData`,
`timepointData`, `combinedData`, and `clockFoundation`.

```{r preprocessingPheno-pipeline, message = FALSE, warning = FALSE}
phenoPipelineResult <- preprocessingPheno(
    phenoFile = pheno_inputs$phenoPath,
    betaPath = pheno_inputs$betaPath,
    mPath = pheno_inputs$mPath,
    cnPath = pheno_inputs$cnPath,
    SampleID = "Sample_Name",
    timeVar = "Timepoint",
    timepoints = "1,2",
    combineTimepoints = "1,2",
    outputPheno = file.path(pheno_inputs$tempDir, "data", "preprocessingPheno"),
    outputRData = file.path(
        pheno_inputs$tempDir,
        "rData",
        "preprocessingPheno",
        "metrics"
    ),
    outputRDataMerge = file.path(
        pheno_inputs$tempDir,
        "rData",
        "preprocessingPheno",
        "mergeData"
    ),
    sexColumn = "Sex",
    outputLogs = file.path(pheno_inputs$tempDir, "logs"),
    outputDir = file.path(pheno_inputs$tempDir, "clockFoundation"),
    verbose = FALSE,
    logs = TRUE,
    saveOutputs = TRUE
)

names(phenoPipelineResult$savedFiles)
names(phenoPipelineResult$savedFiles$timepoints)
basename(unlist(phenoPipelineResult$savedFiles$timepoints[["1"]]))
basename(unlist(phenoPipelineResult$savedFiles[c(
    "combinedPheno",
    "combinedPhenoMethylation",
    "betaCSV",
    "phenoCF"
)]))
```

These outputs are the files most commonly consumed by downstream modeling
functions. `names(phenoPipelineResult$savedFiles)` shows the high-level output
groups returned by the function.
`names(phenoPipelineResult$savedFiles$timepoints)`
lists the available timepoint-specific subsets. The `basename(...)` calls then
show examples of the concrete files written for one timepoint and for the
combined outputs. In this structure, `combinedPhenoMethylation` is the merged
phenotype-plus-methylation object that becomes the direct input to the GLM and
LME functions. It defaults to beta values, but can be written as M-values or
copy-number values when `methylationScale` is changed. The `betaCSV`, `betaZIP`,
and `phenoCF` outputs remain beta-based Clock Foundation exports.

Arguments:

- `phenoFile`, `betaPath`, `mPath`, and `cnPath` point to the phenotype table
    and the aligned methylation matrices from the previous workflow stages.
- `SampleID = "Sample_Name"` defines the sample key used during alignment.
- `timeVar = "Timepoint"`, `timepoints = "1,2"`, and
    `combineTimepoints = "1,2"` determine the timepoint-specific outputs and the
    combined longitudinal object.
- `methylationScale = "beta"` selects the merged modeling scale. Use `"m"` for
    M-values or `"cn"` for copy-number values. Clock Foundation exports continue
    to use beta values.
- `outputPheno`, `outputRData`, `outputRDataMerge`, and `outputDir` define the
    directories used for phenotype exports, metric objects, merged objects, and
    Clock Foundation inputs.
- `saveOutputs = TRUE` is what makes this step useful in a file-based pipeline,
    because later modeling steps consume these written files.

# Write GLM outputs to disk

The modeling function also separates in-memory results from file outputs. The
next example runs a small GLM analysis and prints the names of the files written
to disk.

### Create example input files

The next chunk shows how to recreate the temporary merged phenotype-plus-beta
input file used by the GLM example.

```{r glm-pipeline-inputs, eval = can_run_glm, message = FALSE, warning = FALSE}
glm_inputs <- dnaEPICO:::exampleMethylationGLMStateDnaEpico()

names(glm_inputs)
glm_inputs$tempDir
basename(glm_inputs$inputPath)
file.exists(glm_inputs$inputPath)
```

`glm_inputs` is a list returned by the GLM example helper. `names(glm_inputs)`
shows that it contains the temporary directory, the saved merged
phenotype-plus-beta input file (`inputPath`), and the corresponding in-memory
objects produced during helper construction: `preparedData`, `modelResults`, and
`modelSummaries`.

```{r glm-pipeline, eval = can_run_glm, message = FALSE, warning = FALSE}
glmPipelineResult <- methylationGLM(
    inputPheno = glm_inputs$inputPath,
    phenotypes = "status",
    covariates = "sex,age",
    factorVars = "status,sex",
    scaleVars = "age",
    cpgLimit = 2,
    nCores = 1,
    outputLogs = file.path(glm_inputs$tempDir, "logs"),
    outputRData = file.path(glm_inputs$tempDir, "rData", "models"),
    outputPlots = file.path(glm_inputs$tempDir, "figures"),
    significantCpGDir = file.path(glm_inputs$tempDir, "significant"),
    summaryTxtDir = file.path(glm_inputs$tempDir, "summaries"),
    summaryPval = 1,
    significantCpGPval = 1,
    annotationPackage = "IlluminaHumanMethylation450kanno.ilmn12.hg19",
    annotationCols = "Name,chr,pos",
    annotatedGLMOut = file.path(glm_inputs$tempDir, "annotated"),
    reportAssetsDir = file.path(
        preprocessing_inputs$tempDir,
        "reports", "example", "assets", "results", "glm_results"
    ),
    display = FALSE,
    verbose = FALSE,
    logs = TRUE,
    saveOutputs = TRUE
)

class(glmPipelineResult$savedFiles)
names(glmPipelineResult$savedFiles)
```

The returned `savedFiles` object records the groups of outputs written by the
GLM step. `names(glmPipelineResult$savedFiles)` includes compact phenotype
summary files, significant-CpG exports, summary text files, and the annotated
results workbook. Inspect this object to identify the written modeling outputs
and their groups.

Arguments:

- `inputPheno` points to the merged phenotype-plus-beta object generated by
    `preprocessingPheno()`.
- `phenotypes` lists the outcome variables that will be modeled one at a time.
- `covariates` lists the adjustment variables included in every model.
- `factorVars` identifies the categorical variables that should be treated as
    factors before model fitting.
- `scaleVars` identifies numeric fixed-effect variables to centre and divide by
    their sample standard deviations before fitting.
- Numeric CpG columns are passed unchanged to glm2. Native messages, warnings,
    or errors are stored in `<Phenotype>_Model.Message`; model attempts without
    a returned p-value are counted in workbook metadata and are not added to the
    annotated result table.
- `cpgLimit = 2` keeps the example fast by fitting only two CpG models.
- `nCores = 1` keeps the example deterministic and lightweight. In production,
    `nCores` is a maximum; the effective worker count is also limited by the
    workload, detected CPUs, and available memory.
- `annotationPackage` and `annotationCols` control which array annotation is
    merged into the final summary tables.
- `gencodeHub = TRUE` retrieves the package-managed gene resource from
    AnnotationHub and adds direct gene-body and nearest strand-specific TSS
    annotations. This option requires a GRCh38 array annotation; the GENCODE
    release is read from the resource metadata and retained in the output
    column names, workbook metadata, and dictionary.
- `outputRData`, `outputPlots`, `significantCpGDir`, `summaryTxtDir`, and
    `annotatedGLMOut` define where compact phenotype summaries, plots, text
    summaries, and the
    annotated GLM workbook are written.
- `resumeFromSummary = TRUE` reuses a complete phenotype summary only when it
    was generated from the same input file and model configuration. A phenotype
    without a complete summary restarts from its first CpG.
- `reportAssetsDir` writes the compressed report table directly into the report
    project, leaving the annotated workbook directory free of report sidecars.
- `saveOutputs = TRUE` enables the file-based pipeline behaviour expected by
    Makefile and HPC use.

# Write LME outputs to disk

The longitudinal modeling step follows the same pattern as the GLM function, but
the saved outputs now correspond to mixed-effects models, longitudinal summary
tables, and interaction-specific result files.

### Create example input files

The next chunk shows how to recreate the temporary combined longitudinal
phenotype-plus-beta input file used by the LME example.

```{r lme-pipeline-inputs, eval = can_run_lme, message = FALSE, warning = FALSE}
lme_inputs <- dnaEPICO:::exampleMethylationLMEStateDnaEpico()

names(lme_inputs)
lme_inputs$tempDir
basename(lme_inputs$inputPath)
file.exists(lme_inputs$inputPath)
```

`lme_inputs` plays the same role for the longitudinal example. The helper
returns a list with the temporary directory, the saved combined longitudinal
input file (`inputPath`), and the in-memory objects used to build that example:
`preparedData`, `modelResults`, and `modelSummaries`.

```{r lme-pipeline, eval = can_run_lme, message = FALSE, warning = FALSE}
lmePipelineResult <- methylationLME(
    inputPheno = lme_inputs$inputPath,
    outputLogs = file.path(lme_inputs$tempDir, "logs"),
    outputRData = file.path(lme_inputs$tempDir, "rData", "models"),
    outputPlots = file.path(lme_inputs$tempDir, "figures"),
    personVar = "person",
    SampleID = "Sample_Name",
    timeVar = "Timepoint",
    phenotypes = "score",
    covariates = "sex",
    factorVars = "sex",
    scaleVars = "score",
    cpgLimit = 2,
    nCores = 1,
    summaryPval = 1,
    saveSignificantInteractions = TRUE,
    significantInteractionDir = file.path(
        lme_inputs$tempDir,
        "results",
        "cpgs",
        "methylationLME"
    ),
    significantInteractionPval = 1,
    saveTxtSummaries = TRUE,
    summaryTxtDir = file.path(
        lme_inputs$tempDir,
        "results",
        "summary",
        "methylationLME"
    ),
    annotationPackage = "IlluminaHumanMethylation450kanno.ilmn12.hg19",
    annotationCols = "Name,chr,pos",
    annotatedLMEOut = file.path(lme_inputs$tempDir, "annotated"),
    reportAssetsDir = file.path(
        preprocessing_inputs$tempDir,
        "reports", "example", "assets", "results", "lme_results"
    ),
    display = FALSE,
    verbose = FALSE,
    logs = TRUE,
    saveOutputs = TRUE
)

class(lmePipelineResult$savedFiles)
names(lmePipelineResult$savedFiles)
```

The returned `savedFiles` object records the groups of outputs written by the
LME step. `names(lmePipelineResult$savedFiles)` includes compact phenotype
summary files, summary text files, significant interaction exports, and the
annotated longitudinal results table. Inspect this object to identify the
written longitudinal modeling outputs and their groups.

Arguments:

- `inputPheno` points to the combined longitudinal phenotype-plus-beta object.
- `personVar` defines the participant identifier used for the random effect.
- `SampleID` identifies the sample-level column used to derive `personVar` when
    that participant column is absent. Automatic derivation removes a terminal
    `A` or `B`; other identifier formats require an explicit participant column.
- `timeVar` defines the repeated-measures time variable used for longitudinal
    summaries and preprocessing checks.
- `phenotypes`, `covariates`, and optional `interactionTerm` define the
    fixed-effect portion of the mixed model.
- `omnibusTest = TRUE` adds one joint fixed-effect F test per CpG. With an
    interaction, all estimable phenotype-by-interaction coefficients are tested
    together; without an interaction, the phenotype main effect is tested.
- `omnibusDdf` selects `"Satterthwaite"` or `"Kenward-Roger"` denominator
    degrees of freedom. Kenward-Roger testing requires `pbkrtest` and is more
    computationally intensive for an EWAS.
- Numeric CpG columns are passed unchanged to the selected lmerTest/lme4 or
    nlme engine. Native conditions are stored in
    `<Phenotype>_Model.Message`; model attempts without a returned coefficient
    or omnibus p-value are counted in workbook metadata and are not added to
    the annotated result table.
- `cpgLimit = 2` keeps the example short by fitting only two CpG models.
- `nCores = 1` keeps the example lightweight. In production, `nCores` is a
    maximum; the effective worker count is also limited by the workload,
    detected CPUs, and available memory.
- `saveSignificantInteractions`, `significantInteractionDir`, and
    `significantInteractionPval` control the export of interaction-specific
    results.
- `summaryTxtDir` defines where plain-text model summaries are written.
- `annotationPackage`, `annotationCols`, and `annotatedLMEOut` define the
    annotation resources and output location for the final longitudinal summary
    workbook.
- `gencodeHub = TRUE` applies the same release-aware AnnotationHub annotation
    to lmerTest/lme4 and nlme results when the array coordinates use GRCh38.
- `reportAssetsDir` writes the compressed longitudinal report table directly
    into the report project rather than beside the workbook.
- `resumeFromSummary = TRUE` reuses a complete phenotype summary only when it
    was generated from the same input file and model configuration. A phenotype
    without a complete summary restarts from its first CpG.
- `saveOutputs = TRUE` enables the file-based longitudinal workflow expected by
    Makefile and HPC use.

For a full-cohort longitudinal interaction analysis, the following structure
fits a participant-level random intercept and returns one joint interaction
p-value per CpG in addition to the component coefficient p-values:

```{r lme-omnibus-example, eval = FALSE}
omnibusResult <- methylationLME(
    inputPheno = "rData/model/preprocessingPheno/mergeData/phenoBT1T2T3.RData",
    personVar = "Participant",
    timeVar = "Timepoint",
    phenotypes = "Timepoint",
    covariates = "Age,Sex,PC1,PC2,PC3",
    factorVars = "Timepoint,Profession,Sex",
    interactionTerm = "Profession",
    omnibusTest = TRUE,
    omnibusDdf = "Satterthwaite"
)
```

This model uses the fixed effects `Timepoint * Profession` together with
`(1 | Participant)`. A significant omnibus result indicates that at least one
timepoint contrast differs by profession; the component coefficients are used
to identify its direction and location.

## Generate the report from the example outputs

After the file-based examples above have been run, `dnamReport()` can assemble
the phenotype table, ENmix control figures, quality-control figures, batch
effect figures, metrics figures, model annotation tables, and logs into one
website report. The example below uses the concrete output paths created by the
previous chunks.

The preprocessing and SVA examples share `preprocessing_inputs$tempDir`, while
the GLM and LME examples write model outputs into their own temporary
directories. Because `dnamReport()` accepts one path per tab, those outputs can
be passed directly.

When running this section from an R session, `dnamReport()` still needs the
Quarto command line interface because the function renders the `.qmd` files into
the final website. Quarto is not required to install or load `dnaEPICO`, or to
run its preprocessing and statistical-modeling functions. Check whether R can
find Quarto with:

```{r quarto-check-r, eval = FALSE}
Sys.which("quarto")
system2("quarto", "--version")
```

If `Sys.which("quarto")` returns an empty string, install the Quarto command
line interface from the
[Quarto get-started page](https://quarto.org/docs/get-started/) before
rendering the report. The R package `quarto` provides R helper functions for an
existing Quarto installation; it does not replace the Quarto command line
interface used by `dnamReport()`.

If Quarto is installed but not on `PATH`, set the executable path before calling
`dnamReport()`:

```{r quarto-path-r, eval = FALSE}
Sys.setenv(QUARTO_BIN = "/full/path/to/quarto")
```

```{r dnamReport-pipeline-example, eval = FALSE}
report_log_dir <- file.path(preprocessing_inputs$tempDir, "logs", "report")
dir.create(report_log_dir, recursive = TRUE, showWarnings = FALSE)

report_logs <- c(
    file.path(
        preprocessing_inputs$tempDir,
        "logs",
        "log_preprocessingMinfiEwasWater.txt"
    ),
    file.path(pheno_inputs$tempDir, "logs", "log_preprocessingPheno.txt"),
    file.path(preprocessing_inputs$tempDir, "logs", "log_svaEnmix.txt"),
    file.path(glm_inputs$tempDir, "logs", "log_methylationGLM.txt"),
    file.path(lme_inputs$tempDir, "logs", "log_methylationLME.txt")
)

existing_logs <- report_logs[file.exists(report_logs)]
if (length(existing_logs)) {
    file.copy(existing_logs, report_log_dir, overwrite = TRUE)
}

reportResult <- dnamReport(
    outputDir = file.path(preprocessing_inputs$tempDir, "reports", "example"),
    phenoTab = file.path(
        preprocessing_inputs$tempDir,
        "data",
        "preprocessingMinfiEwasWater",
        "phenoLC.csv"
    ),
    enmixTab = file.path(
        preprocessing_inputs$tempDir,
        "figures",
        "preprocessingMinfiEwasWater",
        "enmix"
    ),
    qcTab = file.path(
        preprocessing_inputs$tempDir,
        "figures",
        "preprocessingMinfiEwasWater",
        "qc"
    ),
    svaTab = file.path(preprocessing_inputs$tempDir, "figures", "svaEnmix"),
    metricTab = file.path(
        preprocessing_inputs$tempDir,
        "figures",
        "preprocessingMinfiEwasWater",
        "metrics"
    ),
    glmTab = glmPipelineResult$savedFiles$annotatedGLM,
    lmeTab = lmePipelineResult$savedFiles$annotatedLME,
    logTab = report_log_dir,
    detPPath = file.path(
        preprocessing_inputs$tempDir,
        "rData",
        "preprocessingMinfiEwasWater",
        "qc",
        "detP_RGSet.RData"
    ),
    detPThreshold = 0.01,
    verbose = FALSE,
    logs = TRUE
)

reportResult$outputFile
```

## GLM model structure and PRS terms

For each CpG, `methylationGLM()` fits a regression model of the form:

$$
\text{Methylation}_{CpG_i} =
\beta_0 +
\beta_1 \cdot \text{Phenotype} +
\sum_{k = 1}^{K} \beta_{k + 1} \cdot \text{Covariate}_k +
\varepsilon
$$

where $\text{Methylation}_{CpG_i}$ is the methylation value at CpG $i$,
$\beta_0$ is the intercept, the phenotype term is the variable of interest, and
the remaining fixed effects represent the listed covariates.

Setting `omnibusTest = TRUE` adds one joint Wald F test per CpG using
`car::linearHypothesis()`. Without an interaction, the complete phenotype term
is tested. With an interaction, the complete phenotype-by-interaction term is
tested. One-degree-of-freedom terms, including a numeric phenotype entered as
one linear term, are also tested and reproduce the corresponding coefficient
p-value. Omnibus p-values are adjusted across valid CpGs within each phenotype
and tested term using `padjmethod`.

If no polygenic risk score is included, a model can be read schematically as:

$$
\begin{aligned}
\text{Methylation}_{CpG_i} =\;&
\beta_0 +
\beta_1 \cdot \text{Phenotype} +
\beta_2 \cdot \text{Age} +
\beta_3 \cdot \text{Sex} +
\beta_4 \cdot \text{Ethnicity} \\
&+
\varepsilon
\end{aligned}
$$

When `prsMap` is supplied, a phenotype-specific PRS term is appended only for
the matching phenotype. For example, the mapping
`"Pheno1:PRS_1,Pheno2:PRS_2"` means that:

- models for `Pheno1` include `PRS_1`
- models for `Pheno2` include `PRS_2`
- phenotypes without a mapping are fit without a PRS term

For a phenotype mapped to a PRS, the model becomes:

$$
\text{Methylation}_{CpG_i} =
\beta_0 +
\beta_1 \cdot \text{Phenotype} +
\sum_{k = 1}^{K} \beta_{k + 1} \cdot \text{Covariate}_k +
\beta_{\text{PRS}} \cdot \text{PRS} +
\varepsilon
$$

If `interactionTerm` is supplied, the fixed effects include its main effect and
its interaction with the phenotype.

## Longitudinal mixed-effects structure

The longitudinal function `methylationLME()` follows the same file-based
pattern, but uses a mixed-effects model with a participant-level random
intercept. In schematic form:

$$
\begin{aligned}
\text{Methylation}_{CpG_i} =\;&
\beta_0 +
\beta_1 \cdot \text{Phenotype} +
\beta_2 \cdot \text{Interaction} +
\beta_3 \cdot (\text{Phenotype} \times \text{Interaction}) +
\sum_{k = 1}^{K} \beta_{k + 3} \cdot \text{Covariate}_k +
\beta_{\text{PRS}} \cdot \text{PRS} +
b_{\text{person}} +
\varepsilon
\end{aligned}
$$

where $b_{\text{person}}$ is the subject-specific random intercept. As in the
GLM workflow, the PRS term is included only when the current phenotype is
matched in `prsMap`.

Common arguments in this longitudinal stage are:

- `inputPheno` for the combined longitudinal phenotype-plus-beta object
- `personVar` for the participant identifier used in the random effect
- `SampleID` for the configured sample identifier used only when `personVar`
    must be derived
- `timeVar` for repeated-measures summaries and preprocessing checks
- `phenotypes`, `covariates`, and optional `interactionTerm` for the
    fixed-effect
    structure
- `prsMap` for phenotype-specific PRS terms
- `cpgLimit` and `nCores` for computational control
- `annotationPackage` and `annotationCols` for annotated summaries

# Makefile use

Running the exported workflow requires GNU Make 4.3 or later. When
`extractMake()` creates the Makefile, it records the absolute path to the
`Rscript` executable from the current R installation in the `RSCRIPT` Make
variable. Therefore, `Rscript` does not need to be on the system path. Report
targets also require Quarto. GNU Make and Quarto are not required to install or
load `dnaEPICO`; they support the exported workflow and report-rendering
features, respectively. The exported rules use relative paths and support
project directories containing spaces. A minimal layout is:

```text
my_project/
    Makefile
    metadata/
    pheno_model1.csv
    data/
    preprocessingMinfiEwasWater/
        idats/
        <IDAT files>
```

The exported template should then be edited so that at least the model name,
phenotype path, and directory variables match the project. A minimal example is:

```make
MODEL = model1
PHENO_FILE = metadata/pheno_model1.csv
LOGS_DIR = logs
DATA_DIR = data
RDATA_DIR = rData
RESULTS_DIR = results
FIGURES_DIR = figures
```

A typical command sequence is:

```bash
make step1 MODEL=model1
make step2 MODEL=model1
make step3 MODEL=model1
make f3 MODEL=model1
make f4 MODEL=model1
make f3lme MODEL=model1
make all MODEL=model1
make status MODEL=model1
make clean MODEL=model1
```

To use another R installation, override the recorded executable when invoking
Make. This is useful after loading an R module on a high-performance computing
system or after moving the Makefile to another computer:

```bash
make f4 MODEL=model1 RSCRIPT=/path/to/Rscript
```

Alternatively, regenerate the Makefile with `extractMake()` from the R
installation that will run the workflow.

The targets run these workflow stages:

- `make step1 MODEL=model1` runs preprocessing only.
- `make step2 MODEL=model1` runs the SVA step using the files written by Step 1.
- `make step3 MODEL=model1` runs phenotype preparation after Step 2 appends
    surrogate-variable columns to `phenoLC.csv`.
- `make f3 MODEL=model1` runs Steps 1-3 and generates a preprocessing report.
- `make f4 MODEL=model1` runs Steps 1-4 and generates a report with
    preprocessing and GLM sections.
- `make f3lme MODEL=model1` runs Steps 1-3 and LME, then generates a report with
    preprocessing and LME sections.
- `make all MODEL=model1` runs Steps 1-5 and generates the complete report.
- `make status MODEL=model1` reports available outputs.
- `make clean MODEL=model1` removes outputs for the selected model while
    preserving shared raw inputs.

If several models are declared in `MODELS`, the grouped multi-model targets can
be used instead:

```bash
make f3_models
make f4_models
make f3lme_models
make models
```

These targets dispatch every model listed in `MODELS`:

- `make f3_models` runs the `f3` route for each declared model.
- `make f4_models` runs the `f4` route for each declared model.
- `make f3lme_models` runs the `f3lme` route for each declared model.
- `make models` runs the full `all` pipeline for each declared model.

Model-level Venn output is disabled when its Makefile values are `NULL`. Set
`VENND_GLM_PHENOTYPES` or `VENND_LME_PHENOTYPES` to comma-separated phenotype
names to compare all coefficient p-value columns generated for those
phenotypes. A factor or interaction phenotype therefore contributes every
applicable coefficient column; the interaction variable is not repeated in the
selection. Optional `VENND_GLM_LABELS` and `VENND_LME_LABELS` values are applied
in that expanded coefficient order and retain their supplied letter case.

Omnibus p-values are selected independently with
`VENND_GLM_OMNIBUS_PHENOTYPES` or `VENND_LME_OMNIBUS_PHENOTYPES`, with optional
positional labels in the corresponding `*_OMNIBUS_LABELS` value. These options
require the relevant omnibus test to be enabled. Each requested analysis writes
nominal and genome-wide UCSC and release-labelled GENCODE figures in
`figures/<model>/methylationGLM` or `figures/<model>/methylationLME`. Nominal,
suggestive, and genome-wide membership tables are added to the annotated
workbook between `metadata` and the final `dictionary` sheet. Venn progress and
errors are retained in the corresponding model-analysis log. When four or more
sets are resolved, a ranked intersection figure accompanies each Venn diagram
to keep the overlap counts readable.

Manhattan plots and Venn membership use the same `9e-8` genome-wide
significance threshold. This threshold controls figure classifications and
membership tables; it does not alter model p-values or their FDR adjustment.

Set `GENCODE_HUB = TRUE` for a GRCh38 model when GENCODE output is required.
The R functions retrieve the internal AnnotationHub resource ID, validate its
assembly and gene-level schema, and derive the release label from the resource
metadata. Users do not provide a separate release value or local annotation
file. AnnotationHub uses its normal local cache after the first retrieval.
`GENCODE_HUB = FALSE` performs no GENCODE retrieval. Models using an hg19 array
annotation, including the 450K hg19 example, must keep this value `FALSE`.

Status and cleanup targets are available for routine use:

```bash
make status MODEL=model1
make clean MODEL=model1
make clean_models
```

`status` reports available outputs, `clean` removes one model, and
`clean_models` removes every model listed in `MODELS`. Cleanup removes the
corresponding directories under `data`, `rData`, `reports`, `logs`, `figures`,
and `results`, and removes empty output roots.

Each step writes outputs consumed by downstream targets. Step 2 creates
`data/<model>/svaEnmix/.sva_complete` only after the updated `phenoLC.csv` has
been validated and installed. Downstream targets depend on both files, so Make
reruns SVA when Step 1 replaces `phenoLC.csv` and skips SVA when the completed
inputs are unchanged.

## Report generation

The exported Makefile contains a report rule whose output is
`$(STEP6)/$(MODEL)/docs/index.html`. With the default directory settings, this
becomes `reports/<model>/docs/index.html`. Users usually do not need to call
that file target directly: the grouped targets above depend on it, so Make
refreshes the report automatically after the required pipeline outputs are
available.

For example, the `f3` route builds the report from the Data, Quality Control,
Batch Effect, Metrics, Report, and Logs inputs; Quality Control contains both
ENmix controls and methylation quality-control outputs. The `f4` route adds only
the GLM page and GLM log, the `f3lme` route adds only the LME page and LME log,
and `all` includes both modeling sections. Each report contains the model
sections defined by its route.

The report uses a responsive three-panel dashboard. The dark left panel holds
the persistent tab navigation and the dnaEPICORM logo, the centre panel uses
the available page width for the selected table or figure, and the right panel
shows section notes that update for the selected figure or workbook sheet. On
smaller displays the navigation becomes a compact top menu and the notes move
below the main content.

Figure-bearing tabs display one selected image at a time with previous/next,
mouse-controlled zoom, scrolling, one card expansion control, and a download
link outside the image region. Workbook tabs provide a sheet selector without
loading every sheet into one visible panel. Package-generated plots adapt point
density, labels, dimensions, and longitudinal traces to the number of samples
or CpGs, while minfi, limma, and ENmix plots retain their established plotting
methods.

The generated figures contain axis labels, legends, thresholds, and statistical
data labels where required, but no embedded title or descriptive caption. The
corresponding report card supplies the figure title, and explicit file names
identify the analysis, variable, diagnostic, annotation source, and version as
applicable. SVA technical-factor association figures are omitted when no
association can be estimated, with the condition retained in the analysis log.

The report target reads the same types of outputs generated in the examples
above, organised under the Makefile project layout: the phenotype table in
`data/`, quality-control and metric figures in `figures/`, the detection
P-value object in `rData/`, model annotation tables in `data/`, and workflow
logs in `logs/`.

The generated site is opened from the report target path:

```text
$(STEP6)/$(MODEL)/docs/index.html
```

Internally, the Makefile calls `dnamReport()` with one argument per tab, using
the project paths defined by the Makefile variables. The report target requires
the Quarto command line interface in the same environment where `make` is run.
If Quarto is not already available, install the Quarto command line interface
from the [Quarto get-started page](https://quarto.org/docs/get-started/), or
use the installation method provided by the local computing environment. Before
launching the report target, check:

```bash
quarto --version
```

If Quarto is installed outside `PATH`, export its executable path before running
`make`:

```bash
export QUARTO_BIN=/full/path/to/quarto
```

# Full exported Makefile

The full Makefile exported by `extractMake()` is shown below. Adapt this
template
for the project before running the pipeline in the target analysis environment.

```{r makefile-template-full, echo = FALSE, results = "asis"}
makefile_lines <- readLines(makefilePath, warn = FALSE)
cat("```make\n", paste(makefile_lines, collapse = "\n"), "\n```\n", sep = "")
```

# Makefile section guide

The exported Makefile is organised into sections so that project-specific
choices can be edited without changing the pipeline rules themselves.

## User configuration

- `MAKEFLAGS += --output-sync` keeps parallel Make output grouped by recipe,
    which makes logs easier to read.
- `RSCRIPT` identifies the R installation used by the workflow. Its default is
    recorded by `extractMake()` and can be overridden when Make is invoked.

## Models selection and parallel run

- `MODEL` defines a single pipeline configuration to run.
- `MODELS` defines a set of configurations that can be launched in parallel by
    the grouped Make targets.

In single-model use, commands such as `make f4 MODEL=model1` run one workflow.
In multi-model use, commands such as `make f4_models` evaluate the same template
for each name listed in `MODELS`.

## Per-model overrides

- `PHENO_FILE` is reassigned according to the selected `MODEL`.

This block is useful when different cohorts, case definitions, or phenotype
encodings should reuse the same analysis pipeline without duplicating the full
Makefile.

## Directories

- `LOGS_DIR`, `DATA_DIR`, `RDATA_DIR`, `RESULTS_DIR`, and `FIGURES_DIR` define
    the top-level folder structure.
- `STEP1` to `STEP6` define the canonical names of the major pipeline stages.
- `METRICS_DIR`, `IDAT_DIR`, `OBJ_DIR`, `MERGE_DIR`, `MODEL_DIR`, `CPG_DIR`,
    `SUMMARY_DIR`, `ENMIX_DIR`, `SVA_DIR`, and `QC_DIR` define the subdirectory
    names used by the rules file.
- `R_DIR` optionally points to a custom R library path on shared systems. Its
    default value is `NULL`, which uses the default library path.

These variables control where files are written and how paths are composed
across all stages of the pipeline.

## Global parameters

These parameters are shared across multiple workflow steps.

### Step 1 to Step 5 shared parameters

- `SEED`: seed used for reproducible operations in the workflow.
- `SEP_TYPE`: optional separator used when reading text files; `NULL` uses the
    default comma separator.
- `SAMPLE_ID`: phenotype column used to align samples.
- `N_SAMPLES`: maximum number of phenotype rows; `NA` uses all rows.
- `ARRAY_TYPE` and `ANNOTATION_VERSION`: array platform and annotation build.
- `IDAT_FORCE`: whether `minfi::read.metharray.exp()` should force reading
    selected IDAT files with different internal array sizes; the default is
    `FALSE`.
- `TIFF_WIDTH`, `TIFF_HEIGHT`, and `TIFF_RES`: default plot dimensions and
    resolution.
- `SEX_COLUMN`: phenotype column used for sex-based checks and prediction.
- `REMOVE_SEX_MISMATCH`: `FALSE` retains and reports confirmed sex mismatches;
    `TRUE` removes samples only when reported and methylation-predicted sex are
    both known and disagree. Samples with missing or unknown sex are retained.
- `SENTRIX_ID_COLUMN` and `SENTRIX_POSITION_COLUMN`: chip-position metadata
    used by the SVA stage.
- `BASENAME_COLUMN`: column holding the array basename.
- `TIME_VAR`: phenotype column used for repeated-measures or longitudinal
    workflows.
- `METHYLATION_SCALE`: methylation metric used in merged modeling tables;
    choose `beta`, `m`, or `cn`. The default is `beta`; Clock Foundation exports
    continue to use beta values.
- `PHENO_ORDER`: desired order of key phenotype columns in exported tables.

### Step 4 and Step 5 modeling parameters

- `CPG_PREFIX`: prefix used to identify CpG columns.
- `CPG_LIMIT`: optional cap on the number of CpGs processed; `NA` uses all.
- `PRS_MAP`: optional mapping from phenotype names to PRS variables; `NULL`
    omits PRS terms from the models.
- `SUMMARY_PVAL`: optional p-value cutoff for summary tables; `NA` keeps all
    summary rows.
- `N_CORES`: number of cores used within each R job.
- `SAVE_TXT_SUMMARIES`: whether plain-text summaries should be written.
- `RESUME_FROM_SUMMARY`: whether complete phenotype summaries generated from
    the same input files and model configurations may be reused after a model
    stage stops.
- `CHUNK_SIZE`: number of CpGs processed per chunk; `NULL` lets the R helpers
    choose the chunk size automatically. Chunks bound in-memory computation;
    they are not saved as restart checkpoints.
- `FDR_THRESHOLD`: false-discovery-rate threshold used for filtering.
- `PADJ_METHOD`: multiple-testing correction method.
- `ANNOTATION_PACKAGE`: array annotation package used for annotated summaries.
- `ANNOTATION_COLS`: annotation fields to merge into output tables.
- `GENCODE_HUB`: `TRUE` adds the package-managed release-aware GENCODE
    AnnotationHub resource to GRCh38 GLM and LME workbooks; `FALSE` leaves the
    array annotation unchanged.

## Step 1 parameters

- `QC_CUTOFF`: sample-quality threshold applied during preprocessing.
- `DET_PTYPE`: detection p-value method.
- `DET_PTHRESHOLD`: detection p-value cutoff.
- `PVAL_THRESHOLD`: probe-level p-value filter applied after preprocessing.
- `CHR_TO_REMOVE`: chromosomes to exclude from downstream matrices.
- `SNPS_TO_REMOVE`: SNP-related probe categories to remove.
- `PROBE_EXCLUSION_FILE`: semicolon-separated probe-exclusion reference files.
- `PROBE_EXCLUSION_ID_COLUMN`: optional column or semicolon-separated columns
    containing probe IDs; `NULL` auto-detects each reference file.
- `USE_EPICV2_MANIFEST`: whether to also remove EPICv2 probes flagged in the
    Peters et al. expanded manifest from AnnotationHub resource `AH116484`.
- `EPICV2_MANIFEST_CH_WGBS_EVIDENCE`, `EPICV2_MANIFEST_CH_BLAT`,
    `EPICV2_MANIFEST_MISSING_POS`, and `EPICV2_MANIFEST_MISMATCH_POS`: manifest
    flags included in the EPICv2 probe-exclusion set.
- `MAF_THRESHOLD`: minor-allele-frequency threshold used during SNP filtering.
- `PLOT_GROUP_VAR`: phenotype variable used for grouped QC plots.
- `LC_REF`: reference panel used for cell-type estimation.

These values control the main preprocessing and filtering choices in
`preprocessingMinfiEwasWater()`.

## Step 2 parameters

- `CTRL_SVA_PERC_VAR`: target proportion of control-probe variance explained by
    the retained surrogate variables.
- `CTRL_SVA_FLAG`: enables the control-based ENmix SVA workflow.

These values determine how `svaEnmix()` estimates and retains surrogate
variables. The completed step updates the Step 1 `phenoLC.csv` in place and
records successful completion with `.sva_complete`; it does not create a second
phenotype file.

## Step 3 parameters

- `TIMEPOINTS`: comma-separated timepoints exported separately.
- `COMBINE_TIMEPOINTS`: comma-separated timepoints combined into the
    longitudinal object, or `NULL` to skip the combined object.

These values determine how `preprocessingPheno()` splits and recombines the
phenotype-methylation data. The shared `METHYLATION_SCALE` value selects beta,
M-value, or copy-number inputs and is interpreted case-insensitively.

## Step 4 parameters

- `PHENOTYPES_GLM`: comma-separated phenotype variables that will be modelled
    one at a time by `methylationGLM()`.
- `COVARIATES_GLM`: comma-separated adjustment variables included in each GLM.
- `FACTOR_VARS_GLM`: GLM variables treated as categorical.
- `SCALE_VARS_GLM`: numeric GLM fixed-effect variables centred and divided by
    their sample standard deviations; `NULL` retains their original scales.
- `GLM_LIBS`: GLM implementation used by `methylationGLM()`.
- `INTERACTION_GLM`: optional interaction term added to the fixed-effect part
    of the GLM; `NULL` omits the interaction.
- `GLM_OMNIBUS_TEST`: whether one joint Wald F test is reported for the
    phenotype or phenotype-by-interaction term at each CpG.
- `SUMMARY_RESIDUAL_SD`: whether residual standard deviations are reported in
    summaries.
- `SAVE_SIGNIFICANT_CPGS`: whether tables of significant CpGs are exported.
- `SIGNIFICANT_CPG_PVAL`: p-value threshold used for those significant-CpG
    exports.
- `VENND_GLM_PHENOTYPES`: phenotype names whose coefficient p-value columns are
    compared in GLM Venn outputs; `NULL` disables coefficient Venn output.
- `VENND_GLM_LABELS`: optional labels applied in the expanded coefficient order.
- `VENND_GLM_OMNIBUS_PHENOTYPES`: phenotype names whose omnibus p-values are
    compared; `NULL` disables omnibus Venn output.
- `VENND_GLM_OMNIBUS_LABELS`: optional labels for the selected omnibus sets.
- Annotated GLM rows require at least one returned coefficient or omnibus
    p-value. Native glm2 and omnibus-test conditions are written to the
    phenotype-specific `Model.Message` column and aggregate availability counts
    are written to metadata.

## Step 5 parameters

- `PHENOTYPES_LME`: comma-separated phenotype variables that will be modelled
    one at a time by `methylationLME()`.
- `COVARIATES_LME`: comma-separated adjustment variables included in each LME.
- `FACTOR_VARS_LME`: LME variables treated as categorical.
- `SCALE_VARS_LME`: numeric LME fixed-effect variables centred and divided by
    their sample standard deviations; `NULL` retains their original scales.
- `PERSON_VAR`: participant identifier used as the random effect grouping
    factor.
- `LME_LIBS`: mixed-effects libraries used by `methylationLME()`.
- `LME_CORRELATION_STRUCTURE`: residual correlation structure for the `nlme`
    LME backend; use `none`, `AR1`, or `CAR1`.
- `LME_CORRELATION_VAR`: variable used to order repeated observations within
    participant for `AR1` or `CAR1`; required for those structures.
- `INTERACTION_LME`: optional interaction term used in the longitudinal model;
    `NULL` omits the interaction.
- `LME_OMNIBUS_TEST`: whether lmerTest/lme4 models calculate one joint test for
    the selected phenotype main effect or phenotype-by-interaction term.
- `LME_OMNIBUS_DDF`: denominator degrees-of-freedom method for the joint test;
    use `Satterthwaite` or `Kenward-Roger`.
- `SAVE_SIGNIFICANT_INTERACTIONS`: whether significant interaction tables are
    exported.
- `SIGNIFICANT_INTERACTION_PVAL`: p-value threshold for those exported
    interaction results.
- `VENND_LME_PHENOTYPES`: phenotype names whose coefficient p-value columns are
    compared in LME Venn outputs; `NULL` disables coefficient Venn output.
- `VENND_LME_LABELS`: optional labels applied in the expanded coefficient order.
- `VENND_LME_OMNIBUS_PHENOTYPES`: phenotype names whose omnibus p-values are
    compared; `NULL` disables omnibus Venn output.
- `VENND_LME_OMNIBUS_LABELS`: optional labels for the selected omnibus sets.
- Annotated LME rows require at least one returned coefficient or omnibus
    p-value. Native lmerTest/lme4 or nlme conditions are written to the
    phenotype-specific `Model.Message` column and aggregate availability counts
    are written to metadata.

## Template convention

The template uses `NA` for optional numeric limits or filters where the meaning
is "use all" or "keep all", and `NULL` when an optional separator, variable,
path, mapping, or model term should be absent. `CHUNK_SIZE` uses `NULL` for
automatic chunking because this disables a user-specified tuning value rather
than applying a filter.

## Include pipeline from dnaEPICO

- `DNAPIPE_MK` resolves the packaged `Makefile.rules.pipeline` file.
- `include $(DNAPIPE_MK)` imports the actual pipeline rules after the user
    configuration has been defined.

This keeps the template compact while the package maintains the rules in one
location.

# Summary

Use the file-based workflow when you need:

- a reproducible folder structure,
- files that can be reused by later workflow stages,
- Makefile execution, or
- repeated analyses across one or more models.

The interactive workflow supports direct inspection of returned objects; the
file-based workflow supports reproducible multi-step execution.

# Basics

Date the vignette was generated.

```{r vignette-date, echo = FALSE}
Sys.time()
```

Wallclock time spent generating the vignette.

```{r vignette-wallclock, echo = FALSE}
totalTime <- diff(c(startTime, Sys.time()))
round(totalTime, digits = 3)
```

`R` session information.

```{r vignette-session-info, echo = FALSE}
sessionInfo()
```

## Asking for help

Use the [Bioconductor support site](https://support.bioconductor.org/) for
package questions. Add the `dnaEPICO` tag, review
[previous posts](https://support.bioconductor.org/tag/dnaEPICO/), and include a
small reproducible example with session information.

# References
