Wen-jie Lu (Ant Group), Zhicong Huang (Ant Group), Zhen Gu (Alibaba Group), Jingyu Li (Ant Group & Zhejiang University), Jian Liu (Zhejiang University), Cheng Hong (Ant Group), Kui Ren (Zhejiang University), Tao Wei (Ant Group), WenGuang Chen (Ant Group)

Large transformer-based models have realized state-of-the-art performance on lots of real-world tasks such as natural language processing and computer vision.
However, with the increasing sensitivity of the data and tasks they handle, privacy has become a major concern during model deployment.
In this work, we focus on private inference in two-party settings, where one party holds private inputs and the other holds the model.
We introduce BumbleBee, a fast and communication-friendly two-party private transformer inference system.
Our contributions are three-fold:
First, we propose optimized protocols for matrix multiplication, which significantly reduce communication costs by 80% -- 90% compared to previous techniques.
Secondly, we develop a methodology for constructing efficient protocols tailored to the non-linear activation functions employed in transformer models.
The proposed activation protocols have realized a significant enhancement in processing speed, alongside a remarkable reduction in communication costs by 80% -- 95% compared with two prior methods.
Lastly, we have performed extensive benchmarks on five transformer models.
BumbleBee demonstrates its capability by evaluating the LLaMA-7B model, generating one token in approximately 8 minutes using CPUs.
Our results further reveal that BumbleBee outperforms Iron (NeurIPS22) by over an order of magnitude and is three times faster than BOLT (Oakland24) with one-tenth communication.

View More Papers

The Forking Way: When TEEs Meet Consensus

Annika Wilde (Ruhr University Bochum), Tim Niklas Gruel (Ruhr University Bochum), Claudio Soriente (NEC Laboratories Europe), Ghassan Karame (Ruhr University Bochum)

Read More

Delay-allowed Differentially Private Data Stream Release

Xiaochen Li (University of Virginia), Zhan Qin (Zhejiang University), Kui Ren (Zhejiang University), Chen Gong (University of Virginia), Shuya Feng (University of Connecticut), Yuan Hong (University of Connecticut), Tianhao Wang (University of Virginia)

Read More

Do (Not) Follow the White Rabbit: Challenging the Myth...

Soheil Khodayari (CISPA Helmholtz Center for Information Security), Kai Glauber (Saarland University), Giancarlo Pellegrino (CISPA Helmholtz Center for Information Security)

Read More

Towards LLM-Assisted Vulnerability Detection and Repair for Open-Source 5G...

Rupam Patir (University at Buffalo), Qiqing Huang (University at Buffalo), Keyan Guo (University at Buffalo), Wanda Guo (Texas A&M University), Guofei Gu (Texas A&M University), Haipeng Cai (University at Buffalo), Hongxin Hu (University at Buffalo)

Read More