Comparing the Effectiveness of LLM Guardrails Available on the Market

Large Language Models (LLMs) have become increasingly powerful in recent years, leading to a greater demand for systems that can ensure their safe and responsible use. Two key strategies that play a crucial role in this are model alignment and guardrails, which work in different ways to promote safety at varying stages of the model’s interaction with users.

Alignment primarily focuses on shaping the behavior of the model during the training phase. It involves employing methods that aid the model in producing responses that align with human values, ethical standards, and intended objectives. This process often includes techniques like supervised fine-tuning and reinforcement learning from human feedback to guide the model towards generating appropriate and constructive outputs as a norm.

Despite the efforts put into aligning models correctly, there are instances where even well-aligned models can generate problematic or unsafe content. This is where guardrails step in to play a crucial role. Guardrails function as control mechanisms that come into play during the deployment and utilization of the model. They do not alter the fundamental behavior of the model itself, but rather provide an additional layer of defense against misuse, forbidden content, and harmful actions, thus acting as a safeguard between the user and the AI model.

The comparison study conducted by our team focused on evaluating the effectiveness of the guardrails integrated into the three major cloud-based LLM platforms. We analyzed how each platform’s guardrails handled a wide range of queries from innocuous prompts to potentially malicious instructions. Our assessment involved examining both false positives, where innocuous content is mistakenly blocked, and false negatives, where harmful content manages to evade these protective guardrails.

Guardrails not only serve as an essential barrier against misuse but also help prevent harmful outputs that violate policy guidelines. They function as an additional safety net between the user and the AI model, sifting through inputs and outputs to detect and block content that infringes on established safety protocols. Guardrails differ from model alignment, which focuses on training the AI model to inherently understand and comply with safety guidelines.

While guardrails act as external filters that can be adjusted or modified without changing the model itself, alignment shapes the core behavior of the model through techniques such as reinforcement learning from human feedback and constitutional AI during the training process. The aim of alignment is to inherently guide the model to avoid generating harmful outputs. In contrast, guardrails offer an added layer of protection by enforcing specific rules and detecting edge cases that the model’s training process might overlook.

Our study revealed varying levels of effectiveness in the guardrails implemented by the different platforms. While these guardrails displayed the ability to block numerous harmful prompts or responses, their efficacy significantly varied. We identified common failure cases in the form of both false positives and false negatives across these systems. False positives were often caused by excessively sensitive guardrails misclassifying harmless requests as threats, with code review prompts being a common example. On the other hand, false negatives were observed when prompt injection techniques, especially those employing role-playing scenarios or indirect requests, successfully circumvented input guardrails.

The examination also underscored the significance of model alignment in mitigating risks associated with harmful content generation. Output guardrails showed a low rate of false positives, largely due to LLMs being aligned to reject harmful requests or refrain from generating prohibited content in response to benign prompts. Nonetheless, our findings indicated that when the internal model alignment was insufficient, output filters may not consistently capture harmful content that had managed to slip through.

Palo Alto Networks offers a range of solutions to help organizations safeguard their AI systems, including Prisma AIRS, AI Security Posture Management (AI-SPM), and Unit 42’s AI Security Assessment. Should you suspect a security breach or encounter an urgent issue, do not hesitate to reach out to the Unit 42 Incident Response team for assistance in resolving the matter.

In conclusion, guardrails and model alignment play complementary roles in ensuring the safe and ethical use of LLMs. Guardrails act as an external security layer that prevents misuse and enforces compliance with established guidelines, while alignment works to instill the model with the capacity to generate appropriate outputs by default. By understanding the strengths and limitations of these mechanisms, organizations can better protect their AI systems from potential threats and ensure responsible and secure use of LLM platforms.