The rapid adoption of AI models like DeepSeek and Qwen has introduced new challenges in cybersecurity, as threat actors quickly exploit these platforms to develop malicious content. Check Point Research (CPR) has observed a shift from ChatGPT to these newer models, which offer fewer restrictions and are easier to manipulate. Cybercriminals are sharing jailbreaking techniques to bypass anti-fraud protections, create infostealers, and optimize spam distribution, posing significant risks to organizations and individuals.
Jailbreaking, a method to remove AI model restrictions, has become a popular tactic among cybercriminals. For instance, DeepSeek and Qwen are being used to bypass banking protections, develop malware, and distribute spam. These activities are shared openly on the web, enabling even low-skilled attackers to leverage advanced AI tools for malicious purposes.
Real-world examples include the creation of infostealers using Qwen, jailbreaking prompts for DeepSeek, and the use of multiple AI models to enhance spam distribution efficiency. The lack of robust anti-abuse mechanisms in these newer models has made them attractive to threat actors, escalating the risks of financial theft, data breaches, and other cyber threats.
Darren Guccione, CEO of Keeper Security, emphasizes the importance of proactive measures to mitigate these risks. Organizations should implement strict data classification policies, leverage Privileged Access Management (PAM) solutions, and ensure vendor compliance with security certifications like SOC 2 Type 2 and ISO 27001. Educating employees about the risks of using untrusted AI platforms is also critical to maintaining cybersecurity.
As AI technologies continue to evolve, organizations must prioritize security to defend against the growing misuse of these tools. Check Point Research will continue to monitor how threat actors exploit AI models and provide insights to help organizations stay ahead of emerging threats.
