Chau Pham

I’m a Ph.D. student at Boston University, advised by Prof. Bryan Plummer.

Research Interests:
My research focuses on Machine Learning, with a particular emphasis on efficient deep learning, computer vision, the intersection of vision and language, and large language models.

Previously:
I earned my bachelor’s degree in Computer Science (Honors Program) from HCMUT, Vietnam. After graduation, I worked as a research data scientist at Zalo R&D and then research assistant at Texas Tech University.

You can reach me at chaupham [at] bu [dot] edu.

news

May 2024	I join TikTok Machine Learning team as a Research Scientist Intern this summer.
May 2024	New preprint: Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers
Jan 2024	Our paper on LLM communication via raw transformer output embeddings has been accepted at ICLR 2024.
Oct 2023	Our paper on growing neural networks was accepted at WACV 2024.
Sep 2023	Our paper introducing a new research topic on channel-adaptive imaging models was accepted at NeurIPS 2023.

selected publications

* denotes equal contribution

Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers

Chau Pham, and Bryan A. Plummer

preprint

Abstract PDF Bibtex

Multi-Channel Imaging (MCI) contains an array of challenges for encoding useful feature representations not present in traditional images. For example, images from two different satellites may both contain RGB channels, but the remaining channels can be different for each imaging source. Thus, MCI models must support a variety of channel configurations at test time. Recent work has extended traditional visual encoders for MCI, such as Vision Transformers (ViT), by supplementing pixel information with an encoding representing the channel configuration. However, these methods treat each channel equally, i.e., they do not consider the unique properties of each channel type, which can result in needless and potentially harmful redundancies in the learned features. For example, if RGB channels are always present, the other channels can focus on extracting information that cannot be captured by the RGB channels. To this end, we propose DiChaViT, which aims to enhance the diversity in the learned features of MCI-ViT models. This is achieved through a novel channel sampling strategy that encourages the selection of more distinct channel sets for training. Additionally, we employ regularization and initialization techniques to increase the likelihood that new information is learned from each channel. Many of our improvements are architecture agnostic and could be incorporated into new architectures as they are developed. Experiments on both satellite and cell microscopy datasets, CHAMMI, JUMP-CP, and So2Sat, report DiChaViT yields a 1.5-5.0% gain over the state-of-the-art.

@article{pham2024enhancing,
title={Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers},
author={Chau Pham and Bryan A. Plummer},
journal={arXiv preprint arXiv:2405.16419},
year={2024}}
ICLR

Let Models Speak Ciphers: Multiagent Debate through Embeddings

Chau Pham*, Boyi Liu*, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan A. Plummer, Zhaoran Wang, and Hongxia Yang

In International Conference on Learning Representations (ICLR) 2024

Abstract PDF Bibtex Code

Discussion and debate among Large Language Models (LLMs) have gained considerable attention due to their potential to enhance the reasoning ability of LLMs. Although natural language is an obvious choice for communication due to LLM’s language understanding capability, the token sampling step needed when generating natural language poses a potential risk of information loss, as it uses only one token to represent the model’s belief across the entire vocabulary. In this paper, we introduce a communication regime named CIPHER (Communicative Inter-Model Protocol Through Embedding Representation) to address this issue. Specifically, we remove the token sampling step from LLMs and let them communicate their beliefs across the vocabulary through the expectation of the raw transformer output embeddings. Remarkably, by deviating from natural language, CIPHER offers an advantage of encoding a broader spectrum of information without any modification to the model weights, outperforming the state-of-the-art LLM debate methods using natural language by 0.5-5.0% across five reasoning tasks and multiple open-source LLMs of varying sizes. This showcases the superiority and robustness of embeddings as an alternative "language" for communication among LLMs. We anticipate that CIPHER will inspire further exploration for the design of interactions within LLM agent systems, offering a new direction that could significantly influence future developments in the field.

@InProceedings{phamCIPHER2024,
author={Chau Pham and Boyi Liu and Yingxiang Yang and Zhengyu Chen and Tianyi Liu and Jianbo Yuan and Bryan A. Plummer and Zhaoran Wang and Hongxia Yang},
title={Let Models Speak Ciphers: Multiagent Debate through Embeddings},
booktitle={International Conference on Learning Representations (ICLR)},
year={2024}}
WACV

MixtureGrowth: Growing Neural Networks by Recombining Learned Parameters

Chau Pham*, Piotr Teterwak*, Soren Nelson*, and Bryan A. Plummer

In IEEE Winter Conference on Applications of Computer Vision (WACV) 2024

Abstract PDF Bibtex Code

Most deep neural networks are trained under fixed network architectures and require retraining when the architecture changes. If expanding the network’s size is needed, it is necessary to retrain from scratch, which is expensive. To avoid this, one can grow from a small network by adding random weights over time to gradually achieve the target network size. However, this naive approach falls short in practice as it brings too much noise to the growing process. Prior work tackled this issue by leveraging the already learned weights and training data for generating new weights through conducting a computationally expensive analysis step. In this paper, we introduce MixtureGrowth, a new approach to growing networks that circumvents the initialization overhead in prior work. Before growing, each layer in our model is generated with a linear combination of parameter templates. Newly grown layer weights are generated by using a new linear combination of existing templates for a layer. On one hand, these templates are already trained for the task, providing a strong initialization. On the other, the new coefficients provide flexibility for the added layer weights to learn something new. We show that our approach boosts top-1 accuracy over the state-of-the-art by 2-2.5% on CIFAR-100 and ImageNet datasets, while achieving comparable performance with fewer FLOPs to a larger network trained from scratch. Code is available at https://github.com/chaudatascience/mixturegrowth.

@InProceedings{phamMixtureGrowth2024,
author={Chau Pham and Piotr Teterwak and Soren Nelson and Bryan A. Plummer},
title={MixtureGrowth: Growing Neural Networks by Recombining Learned Parameters},
booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)},
year={2024}}
NeurIPS

CHAMMI: A benchmark for channel-adaptive models in microscopy imaging

Zitong Chen*, Chau Pham*, Siqi Wang, Michael Doron, Nikita Moshkov, Bryan A. Plummer, and Juan C. Caicedo

In Advances in Neural Information Processing Systems (NeurIPS) 2023

Abstract PDF Bibtex Code Data

Most neural networks assume that input images have a fixed number of channels (three for RGB images). However, there are many settings where the number of channels may vary, such as microscopy images where the number of channels changes depending on instruments and experimental goals. Yet, there has not been a systemic attempt to create and evaluate neural networks that are invariant to the number and type of channels. As a result, trained models remain specific to individual studies and are hardly reusable for other microscopy settings. In this paper, we present a benchmark for investigating channel-adaptive models in microscopy imaging, which consists of 1) a dataset of varied-channel single-cell images, and 2) a biologically relevant evaluation framework. In addition, we adapted several existing techniques to create channel-adaptive models and compared their performance on this benchmark to fixed-channel, baseline models. We find that channel-adaptive models can generalize better to out-of-domain tasks and can be computationally efficient. We contribute a curated dataset and an evaluation API to facilitate objective comparisons in future research and applications.

@InProceedings{ChenCHAMMI2023,
author={Zitong Chen and Chau Pham and Siqi Wang and Michael Doron and Nikita Moshkov and Bryan A. Plummer and Juan C Caicedo},
title={CHAMMI: A benchmark for channel-adaptive models in microscopy imaging},
booktitle={Advances in Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks},
year={2023}}
IEEE Bigdata

Graph Adversarial Attacks and Defense: An Empirical Study on Citation Graph

Chau Pham, Vung Pham, and Tommy Dang

In 2020 IEEE International Conference on Big Data (Big Data) 2020

Abstract

This paper details the methodologies and decisions making processes used while developing the attacking and defending models for the Graph Adversarial Attacks and Defense applied to a large citation graph. To handle the large graphs, our attack strategy is twofold: 1) randomly attack the structure first, 2) keep the structure unchanged, then continue the attack on the features using the gradient-based method. On the other hand, the defender is based on 1) filtering and normalizing the feature data, 2) applying the Graph Convolutional Network model, and 3) selecting the models with the highest accuracy and robustness based on our own attacking data. We applied these strategies in KDD Cup 2020 on Graph Adversarial Attacks and Defense dataset. The attacker can drop the accuracy of a surrogate 2-layer Graph Convolutional Network model from 60% to 30% on the test set. Our defending model has 68% accuracy on the validated data and has 89% of the target labels remained the same while adding fake nodes, generated by our attacking method, to the graph.
IEEE Bigdata

Road Damage Detection and Classification with Detectron2 and Faster R-CNN

Vung Pham, Chau Pham, and Tommy Dang

In 2020 IEEE International Conference on Big Data (Big Data) 2020

Abstract

The road is vital for many aspects of life, and road maintenance is crucial for human safety. One of the critical tasks to allow timely repair of road damages is to quickly and efficiently detect and classify them. This work details the strategies and experiments evaluated for these tasks. Specifically, we evaluate Detectron2’s implementation of Faster R-CNN using different base models and configurations. We also experiment with these approaches using the Global Road Damage Detection Challenge 2020, A Track in the IEEE Big Data 2020 Big Data Cup Challenge dataset. The results show that the X101-FPN base model for Faster R-CNN with Detectron2’s default configurations is efficient and general enough to be transferable to different countries in this challenge. This approach results in F1 scores of 51.0% and 51.4% for the test1 and test2 sets of the challenge, respectively. Though the visualizations show good prediction results, the F1 scores are low. Therefore, we also evaluate the prediction results against the existing annotations and discover some discrepancies. Thus, we also suggest strategies to improve the labeling process for this dataset.