Exploring the cybercrime potential of LLMs: a focus on phishing and malware generation

Official URL: https://dx.doi.org/10.1007/978-3-031-94855-8_7

Language Large Models (LLMs) are revolutionizing various sectors by automating complex tasks, enhancing productivity, and fostering innovation. From generating human-like text to facilitating advanced research, LLMs are increasingly becoming integral to societal advances. However, the same capabilities that make LLMs so valuable also pose significant cybersecurity threats. Malicious actors can exploit these models to create sophisticated phishing emails, deceptive websites, and malware, which could lead to substantial security breaches. In response to these challenges, our paper introduces a comprehensive framework to assess the robustness of six leading LLMs (Gemini API, Gemini Web, GPT-4o API, GPT-4o Web, Llama 3 70B, and Mixtral 8x7B) against both direct and elaborate malicious prompts to generate phishing and malware attacks. This framework not only measures the ability – or the lack thereof – of LLMs to resist being manipulated into performing harmful actions, but also provides insights into enhancing their security features to safeguard against such prompt injection attempts. Our findings reveal that even direct prompt injections can successfully compel all tested LLMs to generate phishing emails, websites, and malware. This issue becomes particularly pronounced with elaborate malicious prompts, which achieve high rates of malicious compliance, especially in scenarios involving phishing. Specifically, models such as Llama 3 70B, Gemini API, and Gemini Web show high compliance in generating convincing phishing content under elaborate instructions, while GPT-4o models (both the API and Web versions) excel in creating phishing webpages even when presented with direct prompts. Finally, local models demonstrate nearly perfect compliance with malware generation prompts, underscoring the critical need for sophisticated detection methods and enhanced security protocols tailored to mitigate such elaborate threats. Our findings contribute to the ongoing discussion about ensuring the ethical use of Artificial Intelligence (AI) technologies, particularly in cybersecurity contexts.