Multitask learning for Transformers with application to large-scale single-cell transcriptomes
Type
PreprintAuthors
Pang, MinxingTegner, Jesper

KAUST Department
Biological and Environmental Sciences and Engineering (BESE) DivisionBioscience Program
Date
2020-02-06Permanent link to this record
http://hdl.handle.net/10754/666320
Metadata
Show full item recordAbstract
AbstractRecent progress in machine learning provides competitive methods for bioinformatics in many traditional topics, such as transcriptomes sequence and single-cell analysis. However, discovering biomedical correlation of cells that are present across large-scale data sets remains challenging. Our attention-based neural network module with 300 million parameters is able to capture biological knowledge in a data-driven way. The module contains high-quality embedding, taxonomy analysis and similarity measurement. We tested the model on Mouse Brain Atlas, which consists of 160,000 cells and 25,000 genes. Our module obtained some interesting findings that have been verified by biologists and got better performance when benchmarked against autoencoder and principal components analysis.Citation
Pang, M., & Tegnér, J. (2020). Multitask learning for Transformers with application to large-scale single-cell transcriptomes. doi:10.1101/2020.02.05.935239Publisher
Cold Spring Harbor LaboratoryAdditional Links
http://biorxiv.org/lookup/doi/10.1101/2020.02.05.935239ae974a485f413a2113503eed53cd6c53
10.1101/2020.02.05.935239
Scopus Count
Except where otherwise noted, this item's license is described as http://creativecommons.org/licenses/by-nc-nd/4.0/