For business leaders around the globe, AI promises an exciting edge in innovation and efficiency. The rapid evolution of AI models – and the agentic AI applications they facilitate – look like a game-changer for companies in all categories. AI systems can talk to customers, automate manual processes and analyze masses of data to address issues more quickly.
The vast majority of businesses implementing AI are opting to leverage the power of pre-trained models from the likes of OpenAI, Anthropic, Google and Microsoft as the engine for their own AI systems. For many, however, security is seen as an afterthought, or worse, as a bottleneck that may slow down high-speed innovation. That is a mistake.
Companies may be tempted to integrate agentic AI without fully understanding the risks of using agents that can carry out tasks autonomously. For example, a meeting booking agent may be given access to corporate calendars and email systems via an API. A security breach in that system may cause nothing more than inconvenience. But what if the agent has the instructions and tools to book flights or make money transfers? Suddenly the consequences of a security breach are more severe.
Every extra element that is added to an AI system adds a new attack surface for threat actors to target. As agentic systems become more complex and are increasingly integrated into workflows and customer-facing applications, the permutations and possible corner cases – issues arising outside normal parameters – become virtually impossible to predict.
USING AI TO SECURE AI
It would be tempting to assume that AI vendors have uncovered and protected against threats to their systems but the reality is that even the biggest AI companies use manual red-teaming to test their models. By its nature, this is slow and can never be comprehensive; as every interaction with AI is driven by custom intent, pre-configured measures fall short.
In the AI era, security solutions must be as adaptable and advanced as the threats they face. The most effective way to secure AI is at the inference layer, which requires a holistic approach that encompasses defense, automated red-teaming and governance, risk and compliance.
This is where AI can be used as an offensive weapon against itself. Agentic warfare, as we have termed it, involves using AI to create, adapt and configure tailored attacks based on the AI use cases and risk level. Customizable attack scenarios allow enterprises to test the resilience of their AI systems against complex real-world threat scenarios.
Applying our agentic warfare capability with automated attacks, we stress-tested all the top AI models under headings including harassment, illegal acts, misinformation, sexually-explicit content, toxicity/hate and violence/harm. Our Inference Red-Team solution successfully ‘broke’ all the top AI models, forming the basis for our CalypsoAI Security Leaderboard.
Unlike other rankings that focus on the oversimplified Attack Success Rate, our Leaderboard introduces the CalypsoAI Security Index, a ranking based on a Risk-to-Performance (RTP) ratio and Cost of Security. The Leaderboard helps enterprises that are weighing up AI investment to understand the trade-off between a model’s performance and how secure it is.
THE CALYPSOAI SECURITY LEADERBOARD
In the first iteration of our CalypsoAI Security Leaderboard, Anthropic’s Claude 3.5 Sonnet model had the highest CASI score, with models from Microsoft and DeepSeek also in the top five. The Leaderboard will be updated regularly to take account of new model releases and upgrades.
It is increasingly clear that adopting AI is a business imperative, not a luxury. Almost one-in-three business leaders intend to invest over $25 million in AI this year. By 2026, 74% of companies plan to increase their investments in GenAI.
As companies assess models on their cost and performance, security cannot be ignored. Using a high-performing but relatively insecure model – one that is vulnerable to data theft, for example – could jeopardize an enterprise’s entire AI investment and any expected returns, as well as bringing reputational damage. With so much at stake, AI security must be as sophisticated as the technology it protects; agentic warfare is the new frontier.
