Researchers at security vendor CyberArk have utilized their open-source Fuzzy AI tool to jailbreak the latest model of Anthropic’s Claude, just weeks after successfully jailbreaking OpenAI’s o3 model. Eran Shimony, CyberArk’s principal vulnerability researcher, shared key insights on what this means for partners and customers looking to securely integrate GenAI into their workloads.
FuzzyAI jailbreaks Claude 3.7 faster than it took on previous models
Shimony shared the news of the successful jailbreaking in a LinkedIn post on Feb 26. As he states, it took him less time to jailbreak Claude 3.7 than it did to perform similar testing on the previous 3.5 model. His testing also prompted the model to produce malicious code.
In his post, Shimony highlighted the following as his key takeaways from the jailbreak:
- “As LLMs improve development speed, they may accelerate malware development as a byproduct when their guardrails are bypassed. Of course, this statement holds for other tasks as well.
- It appears that while some vendors lean toward building safer LLMs, others prioritize usability, potentially at the expense of broader public security.”
Shimony’s team announced Fuzzy AI in December, promising the tool was designed to test all available LLMs before companies utilized them in their business workflows.
The key features of FuzzyAI include:
● Comprehensive Fuzzing: FuzzyAI probes AI models with various attack techniques to expose vulnerabilities, such as bypassing guardrails, information leakage, prompt injection, or harmful output generation.
● An Extensible Framework: Organizations and researchers can add their own attack methods to tailor tests for domain-specific vulnerabilities.
● Community Collaboration: A growing community-driven ecosystem ensures continuous adversarial techniques and defense mechanism advancements.
“Many of us started in the traditional world of operating system vulnerability research. It took businesses almost two decades to develop robust solutions against malware and significantly reduce the number of zero days,” Shimony told Channel Insider. “If it took so long to secure operating systems, browsers, and computer networks—despite a large group of people focusing on improving their security—imagine the resources needed to enhance the security of LLMs and Agentic AI frameworks.”
Open-source tool one of several ways partners and customers can protect themselves
CyberArk’s Fuzzy AI is one way organizations can prioritize security as they embrace LLMs across tasks and business functions. The team hopes this tool doesn’t scare people off from engaging with the technology but enables them to adopt emerging models with less risk.
“Our perspective is that improving the security posture of LLMs through an offensive mindset is the best way to challenge these systems in real-world scenarios,” Shimony told Channel Insider. “By challenging LLMs and generating jailbreaks, we, and most importantly, the community, can improve the sensitivity of LLM models to adversarial attacks, ultimately reducing the occurrence of jailbreaks.”
Shimony also highlights the tool can, to this point, jailbreak every model it finds, showing the importance of ongoing testing before application.
This testing capability will be crucial as organizations continue to adopt these tools at high rates.
While virtually everyone seems to be jumping on the AI train, security concerns still exist. This article covers some of the key security risks MSPs face in the new AI world.