Multitask learning for Transformers with application to large-scale single-cell transcriptomes
KAUST DepartmentBiological and Environmental Sciences and Engineering (BESE) Division
Permanent link to this recordhttp://hdl.handle.net/10754/666320
MetadataShow full item record
AbstractAbstractRecent progress in machine learning provides competitive methods for bioinformatics in many traditional topics, such as transcriptomes sequence and single-cell analysis. However, discovering biomedical correlation of cells that are present across large-scale data sets remains challenging. Our attention-based neural network module with 300 million parameters is able to capture biological knowledge in a data-driven way. The module contains high-quality embedding, taxonomy analysis and similarity measurement. We tested the model on Mouse Brain Atlas, which consists of 160,000 cells and 25,000 genes. Our module obtained some interesting findings that have been verified by biologists and got better performance when benchmarked against autoencoder and principal components analysis.
CitationPang, M., & Tegnér, J. (2020). Multitask learning for Transformers with application to large-scale single-cell transcriptomes. doi:10.1101/2020.02.05.935239
PublisherCold Spring Harbor Laboratory
Except where otherwise noted, this item's license is described as http://creativecommons.org/licenses/by-nc-nd/4.0/