I am a first-year Ph.D. student in Computer Science at Harvard University. Prior to starting my Ph.D. I obtained my M.S. in Computational Biology and Quantitative Genetics at Harvard, advised by Prof. Faisal Mahmood. Before that, I received my B.S. in Applied Biology with a minor in Applied Mathematics at the Hong Kong Polytechnic University, where I was fortunate to work with Prof. Lawrence Chan, and Prof. Songlin Chua.
My research interests include gigapixel computational pathology, representation learning, and multimodal integration.
Contrastive visual language pretraining has emerged as a powerful method for either training new language-aware image encoders or augmenting existing pretrained models with zero-shot visual recognition capabilities. However, existing works typically train on large datasets of image-text pairs and have been designed to perform downstream tasks involving only small to medium sized-images, neither of which are applicable to the emerging field of computational pathology where there are limited publicly available paired image-text datasets and each image can span up to 100,000 x 100,000 pixels in dimensions. In this paper we present MI-Zero, a simple and intuitive framework for unleashing the zero-shot transfer capabilities of contrastively aligned image and text models to gigapixel histopathology whole slide images, enabling multiple downstream diagnostic tasks to be carried out by pretrained encoders without requiring any additional labels. MI-Zero reformulates zero-shot transfer under the framework of multiple instance learning to overcome the computational challenge of inference on extremely large images. We used over 550k pathology reports and other available in-domain text corpora to pretrain our text encoder. By effectively leveraging strong pretrained encoders, our best model pretrained on over 33k histopathology image-caption pairs achieves an average median zero-shot accuracy of 70.2% across three different real-world cancer subtyping tasks. Our code is available at: https://github.com/mahmoodlab/MI-Zero.
@inproceedings{lu2023visual,title={Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images},author={Lu*, Ming Y and Chen*, Bowen and Zhang, Andrew and Williamson, Drew FK and Chen, Richard J and Ding, Tong and Le, Long Phi and Chuang, Yung-Sung and Mahmood, Faisal},booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},pages={19764--19775},year={2023},}
