Andrew Searles (University of California Irvine), Renascence Tarafder Prapty (University of California Irvine), Gene Tsudik (University of California Irvine)

Since 2003, CAPTCHAS have been widely used as a barrier against bots, while simultaneously annoying great multitudes of users worldwide. As the use of CAPTCHAS grew, techniques to defeat or bypass them kept improving. In response, CAPTCHAS themselves evolved in terms of sophistication and diversity, becoming increasingly difficult to solve for both bots and humans. Given this long-standing and still-ongoing arms race, it is important to investigate usability, solving performance, and user perceptions of modern CAPTCHAS. In this work, we do so via a large scale (over 3,600 distinct users) 13-month realworld user study and post-study survey. The study, conducted at a large public university, is based on a live account creation and password recovery service with currently prevalent CAPTCHA type: reCAPTCHAv2.

Results show that, with more attempts, users improve in solving checkbox CAPTCHAS. For website developers and user study designers, results indicate that the website context, i.e., whether the service is password recovery or account creation, directly influences (with statistically significant differences) CAPTCHA solving times. We consider the impact of participants’ major and education level, showing that certain majors exhibit better performance, while, in general, education level has a direct impact on solving time. Unsurprisingly, we discover that participants find image CAPTCHAS to be annoying, while checkbox CAPTCHAS are perceived as easy. We also show that, rated via System Usability Scale (SUS), image CAPTCHAS are viewed as “OK”, while checkbox CAPTCHAS are viewed as “good”.

Finally, we also explore the cost and security of reCAPTCHAv2 and conclude that it comes at an immense cost and offers practically no security. Overall, we believe that this study’s results prompt a natural conclusion: reCAPTCHAv2 and similar reCAPTCHA technology should be deprecated.

View More Papers

Towards Understanding Unsafe Video Generation

Yan Pang (University of Virginia), Aiping Xiong (Penn State University), Yang Zhang (CISPA Helmholtz Center for Information Security), Tianhao Wang (University of Virginia)

Read More

SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in...

Phillip Rieger (Technical University of Darmstadt), Alessandro Pegoraro (Technical University of Darmstadt), Kavita Kumari (Technical University of Darmstadt), Tigist Abera (Technical University of Darmstadt), Jonathan Knauer (Technical University of Darmstadt), Ahmad-Reza Sadeghi (Technical University of Darmstadt)

Read More

TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents

Chen Gong (University of Vriginia), Kecen Li (Chinese Academy of Sciences), Jin Yao (University of Virginia), Tianhao Wang (University of Virginia)

Read More

The Midas Touch: Triggering the Capability of LLMs for...

Yi Yang (Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Jinghua Liu (Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China; School of Cyber Security, University of Chinese Academy of Sciences, China), Kai Chen (Institute of Information Engineering, Chinese Academy of…

Read More