%\VignetteIndexEntry{Growing Phylogenetic Trees in R with Treeline}
%\VignettePackage{DECIPHER}

\documentclass[10pt]{article}

\usepackage{times}
\usepackage{hyperref}
\usepackage{underscore}
\usepackage{enumerate}
\usepackage{graphics}
\usepackage{wrapfig}
\usepackage{cite}

\textwidth=6.5in
\textheight=8.5in
%\parskip=.3cm
\oddsidemargin=-.1in
\evensidemargin=-.1in
\headheight=-.3in

\newcommand{\scscst}{\scriptscriptstyle}
\newcommand{\scst}{\scriptstyle}

\newcommand{\R}{{\textsf{R}}}
\newcommand{\C}{{\textsf{C}}}
\newcommand{\code}[1]{{\texttt{#1}}}
\newcommand{\term}[1]{{\emph{#1}}}
\newcommand{\Rpackage}[1]{\textsf{#1}}
\newcommand{\Rfunction}[1]{\texttt{#1}}
\newcommand{\Robject}[1]{\texttt{#1}}
\newcommand{\Rclass}[1]{{\textit{#1}}}
\newcommand{\Rmethod}[1]{{\textit{#1}}}
\newcommand{\Rfunarg}[1]{{\textit{#1}}}

\bibliographystyle{plainnat}

\begin{document}
%\setkeys{Gin}{width=0.55\textwidth}

\title{Growing Phylogenetic Trees in R with Treeline}
\author{Erik S. Wright}
\date{\today}
\maketitle

\newenvironment{centerfig}
{\begin{figure}[htp]\centering}
{\end{figure}}
\renewcommand{\indent}{\hspace*{\tindent}}
\DefineVerbatimEnvironment{Sinput}{Verbatim} {xleftmargin=2em} \DefineVerbatimEnvironment{Soutput}{Verbatim}{xleftmargin=2em} \DefineVerbatimEnvironment{Scode}{Verbatim}{xleftmargin=2em} \fvset{listparameters={\setlength{\topsep}{0pt}}} \renewenvironment{Schunk}{\vspace{\topsep}}{\vspace{\topsep}}
\SweaveOpts{keep.source=TRUE}
<<echo=false>>=
options(continue=" ")
options(width=80)
@

\tableofcontents

%------------------------------------------------------------
\section{Introduction}
%------------------------------------------------------------

This document describes how to grow phylogenetic trees using the \Rfunction{Treeline} and \Rfunction{Zipline} functions in the \Rpackage{DECIPHER} package.  \Rfunction{Treeline} takes as input a set of aligned nucleotide or amino acid sequences and returns a phylogenetic tree (i.e., \Rclass{dendrogram} object) as output.  \Rfunction{Zipline} takes a list of trees and produces a single summary tree.  The overarching idea is to run \Rfunction{Treeline} on a set of multiple sequence alignments corresponding to genes from the same organisms and then give the output gene trees to \Rfunction{Zipline} for summarization into a species tree.

Why are the functions called \Rfunction{Treeline} and \code{Zipline}?  The goal of \Rfunction{Treeline} is to find the best tree according to an optimality criterion.  There are often many trees near the optimum.  Therefore, \Rfunction{Treeline} searches along the metaphorical treeline of high scoring trees to find the best one.  \Rfunction{Zipline} is named for its goal of zippy (fast) and accurate summary trees by using linear regression to connect trees on different treelines.

Why use \Rfunction{Treeline} and \Rfunction{Zipline} versus other programs?  Both functions are designed to return excellent phylogenetic trees with minimal user intervention.  Many tree building programs have a large set of complex options for niche applications.  In contrast, \Rfunction{Treeline} and \Rfunction{Zipline} simply build great trees by default.  \Rfunction{Treeline}'s unified optimization strategy also makes it easy to compare trees optimized under different optimality criteria.  This vignette is intended to get you started and introduce additional options/functions that might be useful.

This vignette focuses on optimizing balanced minimum evolution (ME), maximum likelihood (ML), and maximum parsimony (MP) phylogenetic trees starting from sequences.  Treeline uses a strategy of multi-start optimization followed by hill-climbing and grafting.  Since Treeline is a stochastic optimizer, it optimizes many trees to prevent chance from influencing the final result.  With some luck it'll find the highest scoring tree!

%------------------------------------------------------------
\section{Performance Considerations}
%------------------------------------------------------------

Finding an optimal tree is no easy feat.  \Rfunction{Treeline} systematically optimizes many candidate trees before returning the best one.  This takes time, but there are things you can do to make it go faster.

\begin{itemize}
\item Only use the sequences you need:  \Rfunction{Treeline}'s optimization runtime scales approximately quadratically with the number of sequences.  Hence, limiting the number of sequences is a worthwhile consideration.  In particular, always eliminate redundant sequences, as shown in the example below.
\item Compile with OpenMP support:  Significant speed-ups can be achieved with multi-threading using OpenMP, particularly for ML and MP \Rfunarg{method}s.  See the ``Getting Started DECIPHERing'' vignette for how to enable OpenMP on your computer.  Then you can set the argument \code{processors=NULL} and \Rfunction{Treeline} will use all available processors.
\item Compile for SIMD support:  \Rfunction{Treeline} is configured to make use of SIMD operations, which are available on most processors.  The easiest way to enable SIMD is to add a line with ``CFLAGS += -O3 -march=native'' to your $\sim$/.R/Makevars text file.  Then, after recompiling, there should be a modest speed-up on systems with SIMD support.  Note that enabling SIMD makes the compiled code non-portable, so the code always needs to be compiled on the hardware being used.
\item Set a timeout:  The \Rfunarg{maxTime} argument specifies the (approximate) maximum number of hours you are willing to let \Rfunction{Treeline} run.  If you are concerned about the code running for too long then simply set this argument.
\item Limit iterations:  \Rfunction{Treeline} will converge after \Rfunarg{minIterations} when the score is expected to change less than \Rfunarg{tolerance} per iteration, unless \Rfunarg{maxIterations} is met before convergence.  A reasonable way to converge early is to set \Rfunarg{minIterations} and \Rfunarg{maxIterations} to lower values.  There is evidence supporting the notion that exhaustive searching is unlikely to result in a significantly more correct tree \cite{Kumar:2023}, even as the score continues to improve.
\item For ML, choose a model:  Automatic model selection is a useful feature, but frequently this time-consuming step can be skipped.  For many nucleotide sequences the \code{"GTR+G4"} model will be automatically selected.  Typical amino acid sequences will pick the \code{"LG+G4"} or \code{"WAG+G4"} models, unless the sequences are from a particular origin (e.g., mitochondria).  Pre-selecting a subset of the available \Robject{MODELS} and supplying this as the \Rfunarg{model} argument can save time.
\end{itemize}

Accuracy is another performance consideration.  \Rfunction{Treeline} is a stochastic optimizer, so it will continue searching the space of possible trees until convergence.  It is possible to find the best tree on the first iteration, but most of the time additional iterations will yield a better scoring tree.  If you are feeling unlucky, you can simply increase the number of iterations (i.e., \Rfunarg{minIterations} and \Rfunarg{maxIterations}) to ensure a good tree is found.  There is a decreasing marginal return to more iterations, and it's probably not worth searching (almost) endlessly for a slightly better tree.  \Rfunction{Treeline}'s default settings are designed to balance runtime versus the reward of better scoring trees.

To largely remove luck from the equation, it is best to use an input alignment with many more sites than sequences.  A good rule of thumb is that the input alignment should have a \Rfunarg{width} at least four times its \Rfunarg{length} for nucleotides and two times for amino acids.  Otherwise the optimization space may have a rugged topology with many similarly scoring peaks, and it might be impossible to find the treeline of highest scoring trees.  More importantly than the difficulty of optimization is the lack of robustness, which indicates weak confidence in the resulting tree.  A good way to verify robustness is comparing the score across multiple runs starting from different random seeds or using bootstrapping.

%------------------------------------------------------------
\section{Preparing the Input Data}
%------------------------------------------------------------

\Rfunction{Treeline} takes as input a multiple sequence alignment and/or a distance matrix.  All distance-based methods (including ME) only require specification of \code{myDistMatrix} but will generate a distance matrix using \Rfunction{DistanceMatrix} if \code{myXStringSet} is provided instead.  The character-based methods (i.e., ML and MP) require a multiple sequence alignment and will generate a distance matrix to construct the first candidate tree unless one is provided.

Multiple sequence alignments can be constructed from a set of (unaligned) sequences using \Rfunction{AlignSeqs} or related functions.  \Rfunction{Treeline} will optimize trees for amino acid (i.e., \code{AAStringSet}) or nucleotide (i.e., \code{DNAStringSet} or \code{RNAStringSet}) sequences.  For coding sequences, it is intuitive to assume that nucleotide data would better resolve close taxa, whereas amino acid data would be preferable to determine the branching order of deep taxa.  However, studies have shown that nucleotide data is adequate for determining distant relationships \cite{Kapli:2023}.  A good bet is to use nucleotide sequences with the \code{"ME"} \Rfunarg{method}, possibly specifying a model (e.g., \code{"F81+F"} that corrects for multiple substitutions per site.

Here, we are going to use a set of sequences that is included with \Rpackage{DECIPHER}.  These sequences are from the internal transcribed spacer (ITS) between the 16S and 23S ribosomal RNA genes in several \textit{Streptomyces} species.  To avoid letting the result come down to good old-fashioned luck, it is always best to compare multiple trees optimized for different objectives (ME, ML, and MP) and alternative models of evolution.  \Rfunction{Treeline} is designed to facilitate this type of comparison, ideally across multiple loci.

<<expr1,eval=TRUE,echo=TRUE>>=
library(DECIPHER)

# specify the path to your sequence file:
fas <- "<<path to FASTA file>>"
# OR find the example sequence file used in this tutorial:
fas <- system.file("extdata", "Streptomyces_ITS_aligned.fas", package="DECIPHER")

seqs <- readDNAStringSet(fas) # use readAAStringSet for amino acid sequences
seqs # the aligned sequences
@

Many of these sequences are redundant or from the same genome.  We can de-replicate the sequences to accelerate tree building and simplify analyses:

<<expr2,eval=TRUE,echo=TRUE>>=
seqs <- unique(seqs) # remove duplicated sequences

ns <- gsub("^.*Streptomyces( subsp\\. | sp\\. | | sp_)([^ ]+).*$",
	"\\2",
	names(seqs))
names(seqs) <- ns # name by species (or any other preferred names)
seqs <- seqs[!duplicated(ns)] # remove redundant sequences from the same species
seqs
@

%------------------------------------------------------------
\section{Choosing a Method and Model of Evolution}
%------------------------------------------------------------

Before choosing a \code{model} of evolution, it is necessary to choose a \Rfunarg{method} for optimizing the tree.  The default \Rfunarg{method} is \code{"ME"} because it is fast and performs best on empirical datasets \cite{Gonnet:2012,Spirin:2024}.  The ME \Rfunarg{method} accepts \Rfunarg{myDistMatrix} as input, or \Rfunarg{myXStringSet} can be given with or without a \Rfunarg{model} to use with \Rfunction{DistanceMatrix} for building a distance matrix.  For maximum likelihood, set \Rfunarg{method} to \code{"ML"}, which requires a \Rfunarg{model} of sequence evolution.  For maximum parsimony, set \Rfunarg{method} to \code{"MP"} and (optionally) specify a \Rfunarg{costMatrix}.

\begin{figure}
\begin{center}
\includegraphics[width=1\textwidth]{NucleotideModels}
\caption{\label{f1} Free rates and frequencies in nucleotide models.}
\end{center}
\end{figure}

\Rfunction{Treeline} supports many \Robject{MODELS} of evolution.  In many cases, these \Robject{MODELS} can be extended by appending the model with \code{"+F"}, \code{"+G\#"}, or \code{"+Indels"}.  Here is the list of built-in \Robject{MODELS}:

<<expr3,eval=TRUE,echo=TRUE>>=
MODELS
@

The nucleotide models each have different numbers of free parameters (Fig. \ref{f1}).  The \Robject{MODELS} with few free parameters are supported by \Rfunction{DistanceMatrix} and, therefore, \Rfunarg{method} \code{"ME"}.  This is because distance for few-parameter models can be analytically estimated from the sequences with relatively little error.  High-parameter models, such as \code{"GTR"}, must be optimized and are only supported by \Rfunction{Treeline} \Rfunarg{method} \code{"ML"}.  All base built-in amino acid \Robject{MODELS} have no free parameters and are supported by \Rfunction{DistanceMatrix} and \Rfunction{Treeline}.  See \code{?MODELS} for more information.

\subsection{Minimum Evolution}

Empirical benchmarks suggest ME with p-distance (i.e., length-normalized Hamming distance) results in impressively accurate trees.  Therefore, this is the default configuration when \Rfunarg{myXStringSet} is supplied without \code{myDistMatrix}, which returns branch lengths in units of differences per site.  If you would prefer to have branch lengths in units of substitutions per site, it is possible correct for multiple substitutions (e.g., \textit{A} to \textit{G} back to \textit{A}) by setting \Rfunarg{model} to any of the \Robject{MODELS} of evolution supported by \Rfunction{DistanceMatrix} (e.g., \code{"JC"} or \code{"F81+F"} for nucleotides, and \code{"WAG"} or \code{"WAG+F"} for amino acids).  See Figure \ref{f1} for a list of models supported by \Rfunction{DistanceMatrix}.  When \Rfunarg{method} is \code{"ME"}, maximum control is gained by supplying \Rfunarg{myDistMatrix}, which can be calculated with \Rfunction{DistanceMatrix} beforehand.

For example, a standard \Rfunarg{model} to select for nucleotide sequences would be \code{"TN93+F"} and for amino acid sequences would be \code{"WAG"}.  These models return trees with branch lengths in units of substitutions per site.

\subsection{Maximum Likelihood}

For ML trees, \Rfunction{Treeline} will automatically select an appropriate \Rfunarg{model} according to Akaike information criterion (by default).  It is possible to choose specific model(s) (e.g., \code{model="GTR+G4"}) to limit the possible selections and test your luck with fewer options.  There is evidence that the choice of nucleotide model does not substantially alter tree accuracy \cite{Ripplinger:2008,Hoff:2016,Abadi:2019}, and picking the most complex model every time is a reasonable decision.  All \Rfunarg{model}s can be used with fixed (empirical) letter frequencies (i.e., by appending with \code{+F}) and/or gamma rate variation across sites (e.g., \code{+G4}).  Note \Rfunction{Treeline} supports two discretizations of the gamma distribution: the default of equal binning, or the Laguerre quadrature if \Rfunarg{quadrature} is set to \code{TRUE}.  The former will give likelihoods comparable with other programs, but the latter is more accurate at representing the gamma distribution with limited bins.

For example, a standard \Rfunarg{model} to select for nucleotide sequences would be \code{"GTR+G4+F"} and for amino acid sequences would be \code{"WAG+G4"}, with \Rfunarg{quadrature} set to \code{TRUE} in both cases.  These models return trees with branch lengths in units of substitutions per site.

\subsection{Maximum Parsimony}

For MP trees, the best results are typically obtained by providing a \Rfunarg{costMatrix} rather than relying on the default of binary costs.  The choice of \Rfunarg{costMatrix} is up to you, and several rational options are provided in the examples section of the \Rfunction{Treeline} manual page (see \code{?Treeline}).  A systematic approach to deriving a substitution matrix is also provided as an example below.

\subsection{Treatment of gaps}

The standard models of evolution described above all ignore gap (``-'' and ``.'') characters representing insertions or deletions (indels).  But you're in luck --- \Rfunction{Treeline} has the ability to incorporate gaps into all \Rfunarg{method}s.  For ME trees, \Rfunction{DistanceMatrix} allows gaps to be penalized in p-distance or added to any distance corrected for multiple substitutions per site.  You can either specify a model with \code{"+Indels"} in \Rfunction{Treeline}, or supply \Rfunarg{myDistMatrix} after setting \Rfunarg{penalizeGapLetterMatches} to \code{TRUE} or \code{NA} (see \code{?DistanceMatrix}).  For ML trees, gaps can be added into any model of evolution as an additional state by specifying a model \code{"+Indels"}, which adds two free parameters (Fig. \ref{f1}).  Incorporating gaps results in branch lengths in units of \textit{changes} per site, since both substitutions and indels contribute to distance.  For MP trees, gaps can be added as a character to the \Rfunarg{costMatrix}.  As luck would have it, incorporating gaps tends to result in \textit{slightly} better trees on empirical datasets, although the average improvement is typically very small.

\subsection{Missing models}

There exists a plethora of published models representing sequence evolution, not all of which are supported.  Two notably absent \Robject{MODELS} are invariant and codon models.  Models with a fraction of invariant sites, often represented as \code{+I}, are biologically unrealistic and effectively captured by gamma rate variation across sites (e.g., \code{+G4}).  Including both \code{+I} and \code{+G} creates unnecessary over-parameterization.  Similarly, empirical codon models are not offered because they contain too many (2080) free parameters.  It is hard to believe a single codon substitution matrix can adequately capture variation in codon usage across organisms, when it is known generic amino acid matrices (210 parameters) insufficiently represent many proteins.  Nucleotide models have the advantage that their relatively low number of parameters can be estimated from the data, and there is evidence nucleotide models can even be used for distant relationships where amino acid models were traditionally thought to have an advantage \cite{Kapli:2023}.

%------------------------------------------------------------
\section{Minimum Evolution Phylogenetic Trees}
%------------------------------------------------------------

Now, it's time to try our luck at finding the most likely tree.  We will use the default settings, which returns a minimum evolution tree based on a p-distance matrix.  Simply specify a \Rfunarg{model} to correct for multiple substitutions (e.g., \code{"TN93+F"} or \code{"WAG"}).

Since \Rfunction{Treeline} is a stochastic optimizer, it is critical to always set the random number seed for reproducibility.  This will result in the same sequence of random numbers every time and, therefore, reproducibility.  You can pick any lucky number, and if you ever wonder how much you pushed your luck, you can try running again from a different random number seed to see how much the result came down to luck of the draw.  Note that setting a time limit, as done below with \Rfunarg{maxTime}, negates the purpose of setting a seed -- never set a time limit if reproducibility is desired or you'll have no such luck.

<<expr4,eval=TRUE,echo=TRUE>>=
set.seed(123) # set the random number seed

treeME <- Treeline(seqs, verbose=FALSE, processors=1)

set.seed(NULL) # reset the seed
@

\Rfunction{Treeline} returns an object of class \Rclass{dendrogram} that stores the tree in a nested list structure.  We can take an initial look at the tree and its attributes.

<<expr5,eval=TRUE,echo=TRUE>>=
treeME

attributes(treeME)

str(treeME, max.level=4)
@

%------------------------------------------------------------
\section{Maximum Likelihood Phylogenetic Trees}
%------------------------------------------------------------

For the next example, we will grow a maximum likelihood phylogenetic tree, which is the most computationally demanding optimization objective that is supported.  We will set a stringent time limit (0.01 hours) to make this example faster, although longer time limits (e.g., 24 hours) are advised because setting very short time limits leaves the result partly up to luck.  Again, note that setting a time limit negates the purpose of setting a seed, so omitting a time limit is required for reproducibility.

\begin{centerfig}
<<expr6,eval=TRUE,echo=TRUE,fig=TRUE,height=3.5,width=6>>=
set.seed(123) # set the random number seed

tree <- Treeline(seqs,
	method="ML",
	model="GTR+G4",
	maxTime=0.01,
	verbose=FALSE,
	processors=1)

set.seed(NULL) # reset the seed

plot(tree)
@
\caption{\label{f2} ML tree showing the relationships between \textit{Streptomyces} species.}
\end{centerfig}

\clearpage

\subsection{Plotting Branch Support Values}

Maybe it was just beginner's luck, but we already have a good looking tree!  \Rfunction{Treeline} automatically returns a variety of information about the tree that can be accessed with the \Rfunction{attributes} and \Rfunction{attr} functions:

<<expr7,eval=TRUE,echo=TRUE>>=
attr(tree, "members") # number of leaves below this (root) node
attr(tree, "height") # height of the node (in this case, the midpoint root)
attr(tree, "score") # best score (in this case, the -LnL)
attr(tree, "model") # either the specified or automatically select transition model
attr(tree, "parameters") # the free model parameters (or NA if unoptimized)
attr(tree, "midpoint") # center of the edge (for plotting)
@

The tree is rooted at its midpoint by default.  For maximum likelihood trees, all internal nodes include aBayes branch support values \cite{Anisimova:2011}.  These are given as probabilities that can be used in plotting on top of each edge.  We can also italicize the leaf labels (species names) and add a scale bar.

\begin{centerfig}
<<expr8,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
plot(dendrapply(tree,
	function(x) {
		s <- attr(x, "probability") # choose "probability" (aBayes)
		if (!is.null(s) && !is.na(s)) {
			s <- formatC(as.numeric(s), digits=2, format="f")
			attr(x, "edgetext") <- paste(s, "\n")
		}
		attr(x, "edgePar") <- list(p.col=NA, p.border=NA, t.col="#CC55AA", t.cex=0.7)
		if (is.leaf(x))
			attr(x, "nodePar") <- list(lab.font=3, pch=NA)
		x
	}),
	horiz=TRUE,
	yaxt='n')
# add a scale bar (placed manually)
arrows(2, 0, 2.4, 0, code=3, angle=90, len=0.05, xpd=TRUE)
text(2.2, 0, "0.4 subs./site", pos=3, xpd=TRUE)
@
\caption{\label{f3} ML tree with aBayes probabilities at each internal node.}
\end{centerfig}

\clearpage

%------------------------------------------------------------
\section{Maximum Parsimony Phylogenetic Trees}
%------------------------------------------------------------

While ME and ML trees are based on models of evolution, MP relies on a cost matrix giving the penalty for switching characters along a branch.  The default \Rfunarg{costMatrix} is binary, which is biologically implausible and may invite bad luck.  Hence, we will construct a binary tree and use the result to infer a more appropriate \Rfunarg{costMatrix}.

<<expr9,eval=TRUE,echo=TRUE>>=
set.seed(123) # set the random number seed

tree_UniformCosts <- Treeline(seqs,
	method="MP",
	reconstruct=TRUE,
	verbose=FALSE,
	processors=1)

set.seed(NULL) # reset the seed
@

Since we set \Rfunarg{reconstruct} to \code{TRUE}, \Rfunction{Treeline} output the state transition matrix as an attribute of the tree.  We will use this to make our own luck by deriving a more biologically plausible \Rfunarg{costMatrix}.  It is apparent that transitions are more frequent than transversions and, therefore, are presumably less costly.

<<expr10,eval=TRUE,echo=TRUE>>=
mat <- attr(tree_UniformCosts, "transitions")
mat # count of state transitions

mat <- mat + t(mat) # make symmetric
mat <- mat/(sum(mat)/2) # normalize
mat <- -log2(mat) # convert to bits
diag(mat) <- 0 # reset diagonal
mat # a derived cost matrix
@

Now we can compare the two trees to see whether specifying a non-uniform cost matrix made a difference.  We will highlight different partitions between the trees with dashed edges.  Ideally the two tree topologies would be identical, implying the tree is robust to the specification of the cost matrix.  The fact that this isn't the case suggests the cost matrix has a substantial influence over the tree, as might be expected.  Note the scale of the two trees is different, because branch lengths are in units of average cost (per site) according to each \Rfunarg{costMatrix}.

\begin{centerfig}
<<expr11,eval=TRUE,echo=TRUE,fig=TRUE,height=4,width=6>>=
set.seed(123) # set the random number seed

tree_NonUniformCosts <- Treeline(seqs,
	method="MP",
	costMatrix=mat,
	reconstruct=TRUE,
	verbose=FALSE,
	processors=1)

set.seed(NULL) # reset the seed

splits <- function(x) {
	y <- sapply(x, function(x) paste(sort(unlist(x)), collapse=" "))
	if (!is.leaf(x))
		y <- c(y, splits(x[[1]]), splits(x[[2]]))
	y
}
splits_UniformCosts <- splits(tree_UniformCosts)
splits_NonUniformCosts <- splits(tree_NonUniformCosts)

dashEdges <- function(x, splits) {
	y <- paste(sort(unlist(x)), collapse=" ")
	if (!y %in% splits)
		attr(x, "edgePar") <- list(lty=2)
	x
}

layout(matrix(1:2, nrow=1))
plot(dendrapply(tree_UniformCosts, dashEdges, splits_NonUniformCosts),
	main="MP uniform costs")
plot(dendrapply(tree_NonUniformCosts, dashEdges, splits_UniformCosts),
	main="MP non-uniform costs")
@
\caption{\label{f4} Comparison of MP trees built with different cost matrices.}
\end{centerfig}

\clearpage

\subsection{Ancestral State Reconstruction}

We're in luck \textemdash when \Rfunarg{reconstruct} is \code{TRUE}, \Rfunction{Treeline} infers ancestors for each internal node on the tree \cite{Joy:2016}.  These character states can be used by the function \Rfunction{MapCharacters} to determine state transitions along each edge of the tree. This information enables us to plot the total number of substitutions occurring along each edge.  The state transitions can be accessed along each edge by querying a new ``change'' attribute.

\begin{centerfig}
<<expr12,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
new_tree <- MapCharacters(tree_NonUniformCosts, labelEdges=TRUE)
plot(new_tree, edgePar=list(p.col=NA, p.border=NA, t.col="#55CC99", t.cex=0.7))
attr(new_tree[[1]], "change") # state changes on first branch left of (virtual) root
@
\caption{\label{f5} Edges labeled with the number of state transitions.}
\end{centerfig}

\clearpage

%------------------------------------------------------------
\section{Calculating Bootstrap Support Values}
%------------------------------------------------------------

Phylogenetic trees output by \Rfunction{Treeline} contain information in both their topology and branch lengths.  The goal of phylogenetics is often to determine the branching order of a set of taxa, but this requires a test for statistical significance.  It is usually best to compare trees across different genes, such as how often trees constructed from different genes support the same hypothesis.  In the absence of multiple genes, another option is to quantify the amount of statistical support for each branch separating two sets of taxa.

The aBayes probabilities are a good proxy for whether a partition in the tree is correct \cite{Ecker:2024}, but they are only available for maximum likelihood trees.  For the other trees we need to bootstrap the alignment.  The idea behind bootstrapping is to resample columns (sites) of the alignment with replacement and determine whether each partition was found in the original tree.  Repeating this process allows us to measure the level of statistical support for each branch.

<<expr13,eval=TRUE,echo=TRUE>>=
reps <- 100 # number of bootstrap replicates

tree1 <- Treeline(seqs, verbose=FALSE, processors=1)

partitions <- function(x) {
	if (is.leaf(x))
		return(NULL)
	x0 <- paste(sort(labels(x)), collapse=" ")
	x1 <- partitions(x[[1]])
	x2 <- partitions(x[[2]])
	return(list(x0, x1, x2))
}

pBar <- txtProgressBar()
bootstraps <- vector("list", reps)
for (i in seq_len(reps)) {
	r <- sample(width(seqs)[1], replace=TRUE)
	at <- IRanges(r, width=1)
	seqs2 <- extractAt(seqs, at)
	seqs2 <- lapply(seqs2, unlist)
	seqs2 <- DNAStringSet(seqs2)
	
	temp <- Treeline(seqs2, verbose=FALSE)
	bootstraps[[i]] <- unlist(partitions(temp))
	setTxtProgressBar(pBar, i/reps)
}
close(pBar)
@

Now we can label edges by the percentage of times each partition appeared among the bootstrap replicates.

\begin{centerfig}
<<expr14,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
bootstraps <- table(unlist(bootstraps))
original <- unlist(partitions(tree1))
hits <- bootstraps[original]
names(hits) <- original
w <- which(is.na(hits))
if (length(w) > 0)
	hits[w] <- 0
hits <- round(hits/reps*100)

labelEdges <- function(x) {
	if (is.null(attributes(x)$leaf)) {
		part <- paste(sort(labels(x)), collapse=" ")
		attr(x, "edgetext") <- as.character(hits[part])
	}
	return(x)
}
tree2 <- dendrapply(tree1, labelEdges)
attr(tree2, "edgetext") <- NULL # remove text from (virtual) root branch

plot(tree2, edgePar=list(t.cex=0.5), nodePar=list(lab.cex=0.7, pch=NA))
@
\caption{\label{f6} Tree with bootstrap support probabilities at each internal node.}
\end{centerfig}

\clearpage

%------------------------------------------------------------
\section{More Examples of Manipulating Dendrograms}
%------------------------------------------------------------

It is sometimes useful to alter \Rclass{dendrogram} objects output by \Rfunction{Treeline}.  There are three main ways for working with \Rclass{dendrogram}s:  apply a function to each leaf with \Rfunction{rapply}, apply a function to every node with \Rfunction{dendrapply}, or apply your own function recursively.  The next examples will illustrate each of these approaches with increasing complexity.

In the first example, we will use \Rfunction{rapply} to query and set attributes of each leaf.

<<expr15,eval=TRUE,echo=TRUE>>=
rapply(tree, attr, which="label") # label of each leaf (left to right)
labels(tree) # alternative
rapply(tree, attr, which="height") # height of each leaf (left to right)
italicize <- function(x) {
	if(is.leaf(x)) 
		attr(x, "label") <- as.expression(substitute(italic(leaf),
			list(leaf=attr(x, "label"))))
	x
}
rapply(tree, italicize, how="replace") # italicize leaf labels
@

In the second example, we will use \Rfunction{dendrapply} to identify exclusive groups wherein the members of each group are more similar to each other than they are to those outside the group \cite{Wright:2018}.

\begin{centerfig}
<<expr16,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
d <- DistanceMatrix(seqs, correction="F81+F", verbose=FALSE, processors=1)
exclusive <- function(x) {
	if (!is.leaf(x)) { # leaves are trivially exclusive
		leaves <- unlist(x)
		max_dist <- max(d[leaves, leaves]) # max within group
		if (all(max_dist < d[-leaves, leaves]))
			attr(x, "edgePar") <- list(col="orange")
	}
	x
}
plot(dendrapply(tree, exclusive))
@
\caption{\label{f7} Tree with colored branches above exclusive groups.}
\end{centerfig}

\clearpage

In the third example, we will extract the branching order of five species of interest using a recursive function.  This might be useful if we wanted to count how many times different topologies occurred among a set of trees.  Recursion is the most flexible approach and can be applied with more sophisticated functions to accomplish goals beyond what is possible with \Rfunction{dendrapply}.

<<expr17,eval=TRUE,echo=TRUE>>=
Spp <- c("coelicolor", "lividans", "AA4", "Mg1", "scabiei") # species to retain

extractClade <- function(x) {
	if (is.leaf(x)) {
		if (sum(Spp %in% labels(x)) > 0L) {
			labels(x)
		} else {
			NULL
		}
	} else {
		x <- lapply(x, extractClade)
		x <- x[lengths(x) > 0]
		if (length(x) == 1)
			x <- x[[1]]
		x
	}
}

extractClade(tree)
@

%------------------------------------------------------------
\section{Inspecting the Inputs and Outputs}
%------------------------------------------------------------

If you are feeling down on your luck, you might want to double-check the inputs and outputs for any issues.  First, we can check for any input sequences with unexpectedly few or many characters by comparing character frequencies across all input sequences.  Next, we can look for input sequences that significantly deviate from the expected background frequencies using Pearson's chi-squared test.  We can also check for sequences with extreme distances that might be incorrectly aligned.  Outliers in any of these checks may point to spurious sequences that should be double-checked for correctness or completion.

<<expr18,eval=TRUE,echo=TRUE>>=
freqs <- alphabetFrequency(seqs, baseOnly=TRUE)
head(freqs)
# summarize the number of non-base characters (gaps/ambiguities)
summary(freqs) # "other" represents non-base characters

# index of sequence with the most non-base characters
which.max(freqs[, "other"])

freqs <- freqs[, DNA_BASES]
background <- colMeans(freqs)
background

# look for sequences deviating from background frequencies
chi2 <- colSums((t(freqs) - background)^2/background)
pval <- pchisq(chi2, length(background) - 1, lower.tail=FALSE)
w <- which(pval < 0.05)
seqs[w] # outlier sequences
freqs[w,] # frequencies of outliers

# get sequence index of any very distant outlier sequences
D <- DistanceMatrix(seqs, verbose=FALSE, processors=1)
t <- table(which(D > 0.9, arr.ind=TRUE)) # choose a cutoff
head(sort(t, decreasing=TRUE)) # index of top outliers, if any
@

It is also possible to check whether the output tree reasonably represents the distances between sequences.  For ME trees, the tree should explain greater than \code{0.9} of the variance in the distance matrix used to construct the tree.  We can use Pearson's correlation for trees with branch lengths in different units than the distance matrix (i.e., ML or MP).  Lower correlations may result from alignments with sites having different genealogies, such as concatenated alignments or non-orthologous sequences.

\begin{centerfig}
<<expr19,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
P <- Cophenetic(treeME) # patristic distances
D <- as.dist(D) # conver to 'dist' object

plot(D, P, xlab="Pairwise distance", ylab="Patristic distance", log="xy")
abline(a=0, b=1)

# for ME trees we want explained variance > 0.9
V <- 1 - sum((P - D)^2)/sum((D - mean(D))^2)
V # check the input data if V << 1

cor(P, D) # should be >> 0
cor(log(P), log(D)) # should be >> 0
@
\caption{\label{f8} Confirming correlation between input distances and output patristic distances.}
\end{centerfig}

\clearpage

%------------------------------------------------------------
\section{Exporting the Tree}
%------------------------------------------------------------

We've had a run of good luck with this tree, so we'd better save it before our luck runs out!  The functions \Rfunction{ReadDendrogram} and \Rfunction{WriteDendrogram} will import and export trees in Newick file format.  If we leave the \Rfunarg{file} argument blank then it will print the output to the console for our viewing:

<<expr20,eval=TRUE,echo=TRUE>>=
WriteDendrogram(tree, file="")
@

To keep up our lucky streak, we should probably include any model parameters in the output along with the tree.  Luckily, Newick format supports square brackets (i.e., ``[]'') for comments, which we can append to the end of the file:

<<expr21,eval=TRUE,echo=TRUE>>=
params <- attr(tree, "parameters")
cat("[", paste(names(params), params, sep="=", collapse=","), "]",
	sep="", append=TRUE, file="")
@

%------------------------------------------------------------
\section{Generating a Summary Tree}
%------------------------------------------------------------

So far, we've seen how variability can arise from different methodological decisions, but the choice of gene (or protein) also has a substantial influence on the tree.  In particular, genes within a genome can have discordant genealogies due to biological processes such as incomplete lineage sorting, gene duplication/loss, and horizontal gene transfer \cite{Steenwyk:2023}.  Accounting for the inconsistencies among gene trees is important for inferring correct species phylogenies.

Lucky for us, \Rpackage{DECIPHER} has a function, named \Rfunction{Zipline}, for summarizing trees.  \Rfunction{Zipline} takes a list of \Rclass{dendrogram}s as input and returns a single summary tree as output.  The \Rfunction{Zipline} algorithm uses linear regression to generate a distance matrix representing relationships among the complete set of leaf labels.  Similarly to regression, the \Rfunction{Zipline} function seamlessly handles missing data so that trees don't all need the same set of leaf labels.

Since we've lucked out so far, let's try building a species tree from a set of single copy gene families previously collated from plant genomes \cite{Morel:2023}.  These gene trees were built under the maximum likelihood ``LG+G4'' model with 100 bootstrap replicates.  To begin, we can compute some summary statistics about the plant dataset, which is provided along with the \Rpackage{DECIPHER} package.

<<expr22,eval=TRUE,echo=TRUE>>=
newick <- system.file("extdata", "Plant_gene_trees.newick.gz", package="DECIPHER")
trees <- ReadDendrogram(newick)
length(trees)
head(trees)
tabulate(sapply(trees, attr, which="members"))
summary(sapply(trees, attr, which="height"))
table(unlist(lapply(trees, labels)))
@

Note how \Rfunction{ReadDendrogram} imported 1,547 \Rclass{dendrogram}s from the same Newick file as a \Rclass{list} object.  These trees have between 4 and 21 leaves and vary a lot in their height, which may indicate a problem with orthology inference in some cases.  Several of the species are only found in a few trees, while others are present in hundreds of trees.  The bootstrap values of the trees are imported as ``edgetext'' attributes and range between \code{0} and \code{100}.  We can add a new ``bootstrap'' attribute to normalize the values between \code{0} and \code{1}.

<<expr23,eval=TRUE,echo=TRUE>>=
trees <- lapply(trees,
	dendrapply,
	function(x) {
		b <- as.numeric(attr(x, "edgetext"))
		if (length(b) == 0L) {
			attr(x, "bootstrap") <- 0 # leaf
		} else {
			attr(x, "bootstrap") <- b/100
		}
		x
	})
@

Bootstrap values are positively correlated with branch length, and normalizing them allows us to easily construct a hybrid distance measure to use in summarizing the gene trees.  \Rfunction{Zipline} supports \Rfunarg{distance}s of \code{"length"} (path length between leaves), \code{"edges"} (also known as ``internode'' distance), and custom attributes or their combinations.  Here we will try our chances at using the sum of edge length multiplied by our normalized bootstrap score as the measure of leaf-to-leaf distance.

\begin{centerfig}
<<expr24,eval=TRUE,echo=TRUE,fig=TRUE,height=6,width=6>>=
species_tree <- Zipline(trees,
	bootstraps=10, # typically 100 or more
	method="NJ", # typically "ME" (the default)
	distance=c("length", "bootstrap"))
plot(species_tree)
@
\caption{\label{f9} Summary tree of plant species with bootstrap confidences on each internal edge.}
\end{centerfig}

\Rfunction{Zipline} constructs a distance matrix with linear regression and then calls \Rfunction{Treeline} to build a tree.  Here we asked \Rfunction{Zipline} to perform a very small number \Rfunarg{bootstrap} replicates with the neighbor joining (\code{"NJ"}) method to speed up the example.  Bootstrap values are supplied as ``edgetext'' attributes in percentages on the output \Rclass{dendrogram}.  Our preliminary species tree has relatively low support (Fig. \ref{f9}), suggesting more data are needed to resolve the true species tree.  Hopefully we'll have better luck next time!

Different ways of parameterizing \Rfunarg{distance} may return alternative trees, especially in the absence of sufficient data.  This raises the question:  which measure of \Rfunarg{distance} is best?  Averaging output bootstrap values offers a means of comparing trees for their overall level of support.  Luckily, \Rfunction{Zipline} is fast enough to compare different \Rfunarg{distance} measures by generating alternative species trees and calculating their average support.

<<expr25,eval=TRUE,echo=TRUE>>=
support <- 0
count <- 0
dendrapply(species_tree,
	function(x) {
		s <- attr(x, "edgetext")
		if (!is.null(s)) {
			support <<- support + as.numeric(s)
			count <<- count + 1
		}
		x
	})
support/count # average bootstrap support
@

Using \Rfunction{Treeline} in combination with \Rfunction{Zipline} facilitates the exploration of alternative methods, models, and \Rfunarg{distance} measures.  Generating summary trees with \Rfunction{Zipline} can be a lot of fun, especially when they represent familiar species as in the example above.  May you have the best of luck with growing your own trees!

%------------------------------------------------------------
\section{Session Information}
%------------------------------------------------------------

All of the output in this vignette was produced under the following
conditions:

<<sessinfo,echo=FALSE,results=tex>>=
toLatex(sessionInfo(), locale=FALSE)
@

\begin{thebibliography}{}

\bibitem{Abadi:2019}
{Abadi, S.}, {Azouri, D.}, {Pupko, T.}, \& {Mayrose, I.}
\newblock Model selection may not be a mandatory step for phylogeny reconstruction.
\newblock {\em Nat. Comm.}, 10(1).

\bibitem{Anisimova:2011}
{Anisimova, M.}, {Gil, M.}, {Dufayard, J.}, {Dessimoz, C.}, \& {Gascuel, O.}
\newblock Survey of branch support methods demonstrates accuracy, power, and robustness of fast likelihood-based approximation schemes.
\newblock {\em Syst. Biol.}, 60(5), 685-699.

\bibitem{Ecker:2024}
{Ecker, N.}, {Huchon, D.}, {Mansour, Y.}, {Mayrose, I.}, \& {Pupko, T.}
\newblock A machine-learning-based alternative to phylogenetic bootstrap.
\newblock {\em Bioinformatics}, 40, i208-i217.

\bibitem{Gonnet:2012}
{Gonnet, G.}
\newblock Surprising results on phylogenetic tree building methods based on molecular sequences.
\newblock {\em BMC Bioinformatics}, 13(148).

\bibitem{Hoff:2016}
{Hoff, M.}, {Orf, S.}, {Riehm, B.}, {Darriba, D.}, \& {Stamatakis, A.}
\newblock Does the choice of nucleotide substitution models matter topologically?
\newblock {\em BMC Bioinformatics}, 17(143).

\bibitem{Joy:2016}
{Joy, J.}, {Liang, R.}, {McCloskey, R.}, {Nguyen, T.}, \& {Poon, A.}
\newblock Ancestral Reconstruction.
\newblock {\em PLoS Comp. Biol.}, 12(7), e1004763.

\bibitem{Kapli:2023}
{Kapli, P.}, {Kotari, I.}, {Telford, M.}, {Goldman, N.}, \& {Yang, Z.}
\newblock DNA Sequences Are as Useful as Protein Sequences for Inferring Deep Phylogenies.
\newblock {\em Syst. Biol.}, 72(5), 1119-1135.

\bibitem{Kumar:2023}
{Kumar, S.}, {Tao, Q.}, {Lamarca, A.}, \& {Tamura, K.}
\newblock Computational Reproducibility of Molecular Phylogenies.
\newblock {\em Mol. Biol. Evol.}, 40(7).

\bibitem{Morel:2023}
{Morel, B.}, {Williams, T.}, \& {Stamatakis, A.}
\newblock Asteroid: a new algorithm to infer species trees from gene trees under high proportions of missing data.
\newblock {\em Bioinformatics}, 39(1).

\bibitem{Ripplinger:2008}
{Ripplinger, J.} \& {Sullivan, J.}
\newblock Does Choice in Model Selection Affect Maximum Likelihood Analysis?
\newblock {\em Syst. Biol.}, 57(1), 76-85.

\bibitem{Spirin:2024}
{Ripplinger, S.}, {Sigorskikh, A.}, {Efremov, A.}, {Penzar, D.}, \& {Karyagina, A.}
\newblock PhyloBench: A Benchmark for Evaluating Phylogenetic Programs.
\newblock {\em Mol. Biol. Evol.}, 41(6).

\bibitem{Steenwyk:2023}
{Steenwyk, J.}, {Li, Y.}, {Zhou, X.}, {Shen, X.}, \& {Rokas, A.}
\newblock Incongruence in the phylogenomics era.
\newblock {\em Nat. Rev. Gen.}, 24(12), 834-850.

\bibitem{Wright:2018}
{Wright, E.} \& {Baum, D.}
\newblock Exclusivity offers a sound yet practical species criterion for bacteria despite abundant gene flow.
\newblock {\em BMC Genomics}, 19(724).

\end{thebibliography}

\end{document}
