Relaxed Agreement Forests of Phylogenetic Trees

The phylogenetic inference process can produce, for multiple reasons, conflicting hypotheses of the evolutionary history of a set X of biological entities, i.e., phylogenetic trees with the same set of leaf labels X but with distinct topologies. It is natural to wish to quantify the difference between two such trees T<inf>1</inf>and T<inf>2</inf>. We introduce the problem of computing a maximum relaxed agreement forest (MRAF) and use this as a proxy for the dissimilarity of T<inf>1</inf>and T<inf>2</inf>, which in this article we assume to be unrooted and binary. MRAF asks for a partition of the leaf labels X into a minimum number of blocks S<inf>1</inf>, …, S<inf>k</inf>such that the two subtrees induced in T<inf>1</inf>and T<inf>2</inf>by every S<inf>i</inf>are isomorphic up to suppression of degree-2 nodes and taking the labels X into account. Unlike the earlier introduced maximum agreement forest (MAF) model, the subtrees induced by the S<inf>i</inf>are allowed to overlap. We prove that it is NP-hard to compute MRAF, by reducing from the problem of partitioning a permutation into a minimum number of monotonic subsequences (PIMS). We further show that MRAF has a O(log n)-approximation algorithm where n=|X| and permits exact algorithms with single-exponential running time. When one of the trees is a caterpillar, we prove that testing whether an MRAF has size at most k can be answered in polynomial time when k is fixed. We also note that on two caterpillars the approximability of MRAF is related to that of PIMS. Finally, we establish a number of bounds on MRAF, compare its behaviour to MAF both theoretically and experimentally and discuss a number of open problems.