Show simple item record

dc.contributor.authorKrueger, Robert
dc.contributor.authorBeyer, Johanna
dc.contributor.authorJang, Won-Dong
dc.contributor.authorKim, Nam Wook
dc.contributor.authorSokolov, Artem
dc.contributor.authorSorger, Peter K.
dc.contributor.authorPfister, Hanspeter
dc.date.accessioned2021-02-07T06:42:44Z
dc.date.available2021-02-07T06:42:44Z
dc.date.issued2020-01
dc.identifier.citationKrueger, R., Beyer, J., Jang, W.-D., Kim, N. W., Sokolov, A., Sorger, P. K., & Pfister, H. (2020). Facetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data. IEEE Transactions on Visualization and Computer Graphics, 26(1), 227–237. doi:10.1109/tvcg.2019.2934547
dc.identifier.issn1077-2626
dc.identifier.issn1941-0506
dc.identifier.issn2160-9306
dc.identifier.doi10.1109/tvcg.2019.2934547
dc.identifier.urihttp://hdl.handle.net/10754/667228
dc.description.abstractFacetto is a scalable visual analytics application that is used to discover single-cell phenotypes in high-dimensional multi-channel microscopy images of human tumors and tissues. Such images represent the cutting edge of digital histology and promise to revolutionize how diseases such as cancer are studied, diagnosed, and treated. Highly multiplexed tissue images are complex, comprising 109 or more pixels, 60-plus channels, and millions of individual cells. This makes manual analysis challenging and error-prone. Existing automated approaches are also inadequate, in large part, because they are unable to effectively exploit the deep knowledge of human tissue biology available to anatomic pathologists. To overcome these challenges, Facetto enables a semi-automated analysis of cell types and states. It integrates unsupervised and supervised learning into the image and feature exploration process and offers tools for analytical provenance. Experts can cluster the data to discover new types of cancer and immune cells and use clustering results to train a convolutional neural network that classifies new cells accordingly. Likewise, the output of classifiers can be clustered to discover aggregate patterns and phenotype subsets. We also introduce a new hierarchical approach to keep track of analysis steps and data subsets created by users; this assists in the identification of cell types. Users can build phenotype trees and interact with the resulting hierarchical structures of both high-dimensional feature and image spaces. We report on use-cases in which domain scientists explore various large-scale fluorescence imaging datasets. We demonstrate how Facetto assists users in steering the clustering and classification process, inspecting analysis results, and gaining new scientific insights into cancer biology.
dc.description.sponsorshipWe thank Rumana Rashid, Shannon Coy, Sandro Santagata, Anniina Farkkila, and Julia Casado for their valuable input and feedback. We thank Benjamin Izar, Jia-Ren Lin and YuAn Chen for data and advice on the manuscript, and Denis Schapiro, Clarence Yapp, Jeremy Muhlich for image stitching and segmentation. This work is supported by the Ludwig Center at Harvard Medical School, by NCI Grant U54-CA225088, by King Abdullah University of Science and Technology (KAUST) and the KAUST Office of Sponsored Research (OSR) award OSR-2015-CCF-2533-0.
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.relation.urlhttps://ieeexplore.ieee.org/document/8827951/
dc.rights(c) 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
dc.titleFacetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data
dc.typeArticle
dc.identifier.journalIEEE Transactions on Visualization and Computer Graphics
dc.eprint.versionPost-print
dc.contributor.institutionHarvard University. School of Engineering and Applied Sciences Cambridge, MA, USA.
dc.contributor.institutionSchool of Engineering and Applied Sciences, Harvard University, Cambridge, USA.
dc.contributor.institutionLaboratory of Systems Pharmacology, Harvard Medical School, Boston, USA.
dc.identifier.volume26
dc.identifier.issue1
dc.identifier.pages227-237
kaust.grant.numberOSR-2015-CCF-2533
kaust.acknowledged.supportUnitCCF
kaust.acknowledged.supportUnitKAUST Office of Sponsored Research (OSR)


This item appears in the following Collection(s)

Show simple item record