Utilización del Weka-Spark para la clasificación de genes ortólogos
Archivos
Fecha
2016-06-28
Autores
Zunda Herrera, Lázaro Antonio
Título de la revista
ISSN de la revista
Título del volumen
Editor
Universidad Central “Marta Abreu” de Las Villas
Resumen
Las técnicas de Minería de Datos han sido adaptadas para manejar grandes volúmenes de datos mediante modelos de programación como MapReduce y Spark. Ejemplo de estas adaptaciones son las librerías Mahout implementada en MapReduce y MLlib de Spark. Alternativamente aparece una implementación Weka para Spark que resulta de utilidad para la clasificación binaria de genes ortólogos mediante algoritmos supervisados que manejan el desbalance extremo entre los pares de ortólogos y los no ortólogos. La implementación Weka-Spark soporta diversos filtros para pre-procesamiento de casos y de atributos que pueden mejorar la clasificación en el problema planteado al ser configurada su ejecución con métodos ensamblados y una conexión a un clúster de Hadoop para manejar los datos distribuidos en el sistema de archivos HDFS. Algunos experimentos de comparación de genomas de levaduras Saccharomycete para detectar pares de ortólogos fueron realizados para corroborar tales afirmaciones. Los resultados obtenidos en cuanto a la eficacia en la clasificación y a los tiempos de ejecución son comparables con resultados publicados en esta disciplina de la Bioinformática.
Data mining techniques have been adapted to manage big amount of data by using programing models such as MapReduce and Spark. Some of these adaptations are the Mahout library implemented in MapReduce and the Spark MLlib library. Alternatively, there is an implementation of Weka based on Spark that is useful in the binary classification of ortholog genes by means of supervised algorithms managing extreme imbalance between ortholog and non-ortholog pairs. The Weka-Spark implementation support different filters to pre-process the cases and the attributes in order to improve classification in the stated problem. To achieve this purpose, it should be configured to execute ensemble methods and a connection to a Hadoop cluster to manage distributed data in the HDFS file system. Some experiments of genome comparison in Saccharomycete yeast genomes have been carried out to detect ortholog pairs and to corroborate all these statements. The obtained results in terms of efficacy and runtime are comparable with those published in this Bioinformatics discipline.
Data mining techniques have been adapted to manage big amount of data by using programing models such as MapReduce and Spark. Some of these adaptations are the Mahout library implemented in MapReduce and the Spark MLlib library. Alternatively, there is an implementation of Weka based on Spark that is useful in the binary classification of ortholog genes by means of supervised algorithms managing extreme imbalance between ortholog and non-ortholog pairs. The Weka-Spark implementation support different filters to pre-process the cases and the attributes in order to improve classification in the stated problem. To achieve this purpose, it should be configured to execute ensemble methods and a connection to a Hadoop cluster to manage distributed data in the HDFS file system. Some experiments of genome comparison in Saccharomycete yeast genomes have been carried out to detect ortholog pairs and to corroborate all these statements. The obtained results in terms of efficacy and runtime are comparable with those published in this Bioinformatics discipline.
Descripción
Palabras clave
Spark, Weka-Spark, Clasificación Supervisada, Detección de Ortólogos, Spark, Weka-Spark, Supervised Classification, Ortholog Detection