Evaluation of Genomic Island Predictors Using a Comparative Genomics Approach

Background: Genomic islands (GIs) are clusters of genes in prokaryotic genomes of probablehorizontal origin. GIs are disproportionately associated with microbial adaptations of medical orenvironmental interest. Recently, multiple programs for automated detection of GIs have beendeveloped that utilize sequence composition characteristics, such as G+C ratio and dinucleotidebias. To robustly evaluate the accuracy of such methods, we propose that a dataset of GIs beconstructed using criteria that are independent of sequence composition-based analysisapproaches.Results: We developed a comparative genomics approach (IslandPick) that identifies both veryprobable islands and non-island regions. The approach involves 1) flexible, automated selection ofcomparative genomes for each query genome, using a distance function that picks appropriategenomes for identification of GIs, 2) identification of regions unique to the query genome,compared with the chosen genomes (positive dataset) and 3) identification of regions conservedacross all genomes (negative dataset). Using our constructed datasets, we investigated the accuracyof several sequence composition-based GI prediction tools.Conclusion: Our results indicate that AlienHunter has the highest recall, but the lowest measuredprecision, while SIGI-HMM is the most precise method. SIGI-HMM and IslandPath/DIMOB havecomparable overall highest accuracy. Our comparative genomics approach, IslandPick, was themost accurate, compared with a curated list of GIs, indicating that we have constructed suitabledatasets. This represents the first evaluation, using diverse and, independent datasets that were notartificially constructed, of the accuracy of several sequence composition-based GI predictors. Thecaveats associated with this analysis and proposals for optimal island prediction are discussed.