Peizhuo Lv (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Pan Li (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Shenchen Zhu (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Shengzhi Zhang (Department of Computer Science, Metropolitan College, Boston University, USA), Kai Chen (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Ruigang Liang (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Chang Yue (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Fang Xiang (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Yuling Cai (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Hualong Ma (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Yingjun Zhang (Institute of Software, Chinese Academy of Sciences, China), Guozhu Meng (Institute of Information Engineering, Chinese Academy of Sciences, China; School of Cyber Security, University of Chinese Academy of Sciences, China)

Recent years have witnessed tremendous success in Self-Supervised Learning (SSL), which has been widely utilized to facilitate various downstream tasks in Computer Vision (CV) and Natural Language Processing (NLP) domains. However, attackers may steal such SSL models and commercialize them for profit, making it crucial to verify the ownership of the SSL models. Most existing ownership protection solutions (e.g., backdoor-based watermarks) are designed for supervised learning models and cannot be used directly since they require that the models' downstream tasks and target labels be known and available during watermark embedding, which is not always possible in the domain of SSL. To address such a problem, especially when downstream tasks are diverse and unknown during watermark embedding, we propose a novel black-box watermarking solution, named SSL-WM, for verifying the ownership of SSL models. SSL-WM maps watermarked inputs of the protected encoders into an invariant representation space, which causes any downstream classifier to produce expected behavior, thus allowing the detection of embedded watermarks. We evaluate SSL-WM on numerous tasks, such as CV and NLP, using different SSL models both contrastive-based and generative-based. Experimental results demonstrate that SSL-WM can effectively verify the ownership of stolen SSL models in various downstream tasks. Furthermore, SSL-WM is robust against model fine-tuning, pruning, and input preprocessing attacks. Lastly, SSL-WM can also evade detection from evaluated watermark detection approaches, demonstrating its promising application in protecting the ownership of SSL models.

View More Papers

On the Feasibility of CubeSats Application Sandboxing for Space...

Gabriele Marra (CISPA Helmholtz Center for Information Security), Ulysse Planta (CISPA Helmholtz Center for Information Security and Saarbrücken Graduate School of Computer Science), Philipp Wüstenberg (Chair of Space Technology, Technische Universität Berlin), Ali Abbasi (CISPA Helmholtz Center for Information Security)

Read More

Security Attacks to the Name Management Protocol in Vehicular...

Sharika Kumar (The Ohio State University), Imtiaz Karim, Elisa Bertino (Purdue University), Anish Arora (Ohio State University)

Read More

You Can Use But Cannot Recognize: Preserving Visual Privacy...

Qiushi Li (Tsinghua University), Yan Zhang (Tsinghua University), Ju Ren (Tsinghua University), Qi Li (Tsinghua University), Yaoxue Zhang (Tsinghua University)

Read More

LDR: Secure and Efficient Linux Driver Runtime for Embedded...

Huaiyu Yan (Southeast University), Zhen Ling (Southeast University), Haobo Li (Southeast University), Lan Luo (Anhui University of Technology), Xinhui Shao (Southeast University), Kai Dong (Southeast University), Ping Jiang (Southeast University), Ming Yang (Southeast University), Junzhou Luo (Southeast University, Nanjing, P.R. China), Xinwen Fu (University of Massachusetts Lowell)

Read More