Multitask learning for Transformers with application to large-scale single-cell transcriptomes
dc.contributor.author | Pang, Minxing | |
dc.contributor.author | Tegner, Jesper | |
dc.date.accessioned | 2020-12-09T13:28:51Z | |
dc.date.available | 2020-12-09T13:28:51Z | |
dc.date.issued | 2020-02-06 | |
dc.identifier.citation | Pang, M., & Tegnér, J. (2020). Multitask learning for Transformers with application to large-scale single-cell transcriptomes. doi:10.1101/2020.02.05.935239 | |
dc.identifier.doi | 10.1101/2020.02.05.935239 | |
dc.identifier.uri | http://hdl.handle.net/10754/666320 | |
dc.description.abstract | AbstractRecent progress in machine learning provides competitive methods for bioinformatics in many traditional topics, such as transcriptomes sequence and single-cell analysis. However, discovering biomedical correlation of cells that are present across large-scale data sets remains challenging. Our attention-based neural network module with 300 million parameters is able to capture biological knowledge in a data-driven way. The module contains high-quality embedding, taxonomy analysis and similarity measurement. We tested the model on Mouse Brain Atlas, which consists of 160,000 cells and 25,000 genes. Our module obtained some interesting findings that have been verified by biologists and got better performance when benchmarked against autoencoder and principal components analysis. | |
dc.publisher | Cold Spring Harbor Laboratory | |
dc.relation.url | http://biorxiv.org/lookup/doi/10.1101/2020.02.05.935239 | |
dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/ | |
dc.title | Multitask learning for Transformers with application to large-scale single-cell transcriptomes | |
dc.type | Preprint | |
dc.contributor.department | Biological and Environmental Sciences and Engineering (BESE) Division | |
dc.contributor.department | Bioscience Program | |
dc.eprint.version | Pre-print | |
dc.contributor.institution | Nankai University. | |
dc.contributor.institution | Karolinska Institutet. | |
kaust.person | Tegner, Jesper | |
refterms.dateFOA | 2020-12-09T13:29:41Z |
Files in this item
This item appears in the following Collection(s)
-
Biological and Environmental Sciences and Engineering (BESE) Division
For more information visit: https://bese.kaust.edu.sa/ -
Preprints
-
Bioscience Program
For more information visit: https://bese.kaust.edu.sa/study/Pages/Bioscience.aspx