Rei Yamagishi, Shinya Sasa, and Shota Fujii (Hitachi, Ltd.)

Codes automatically generated by large-scale language models are expected to be used in software development. A previous study verified the security of 21 types of code generated by ChatGPT and found that ChatGPT sometimes generates vulnerable code. On the other hand, although ChatGPT produces different output depending on the input language, the effect on the security of the generated code is not clear. Thus, there is concern that non-native English-speaking developers may generate insecure code or be forced to bear unnecessary burdens. To investigate the effect of language differences on code security, we instructed ChatGPT to generate code in English and Japanese, each with the same content, and generated a total of 450 codes under six different conditions. Our analysis showed that insecure codes were generated in both English and Japanese, but in most cases they were independent of the input language. In addition, the results of validating the same content in different programming languages suggested that the security of the code tends to depend on the security and usability of the API provided by the programming language of the output.

View More Papers

“I used to live in Florida”: Exploring the Impact...

Imani N. S. Munyaka (University of California, San Diego), Daniel A Delgado, Juan Gilbert, Jaime Ruiz, Patrick Traynor (University of Florida)

Read More

Parrot-Trained Adversarial Examples: Pushing the Practicality of Black-Box Audio...

Rui Duan (University of South Florida), Zhe Qu (Central South University), Leah Ding (American University), Yao Liu (University of South Florida), Zhuo Lu (University of South Florida)

Read More

Beyond the Bytes: Understanding the Limitations of Intrinsic Binary...

Peter Lafosse (Owner and Co-Founder of Vector 35 Inc.)

Read More

SENSE: Enhancing Microarchitectural Awareness for TEEs via Subscription-Based Notification

Fan Sang (Georgia Institute of Technology), Jaehyuk Lee (Georgia Institute of Technology), Xiaokuan Zhang (George Mason University), Meng Xu (University of Waterloo), Scott Constable (Intel), Yuan Xiao (Intel), Michael Steiner (Intel), Mona Vij (Intel), Taesoo Kim (Georgia Institute of Technology)

Read More