A security-focused hub for risks and controls around large language models, retrieval systems, prompts, guardrails, and model outputs.
34
Terms
9
Comparisons
8
Questions
LLM security covers the ways language-model applications can be attacked, misused, or misconfigured. It combines ordinary application security with prompt controls, retrieval governance, access control, and output evaluation.
LLM failures can expose confidential files, produce unsafe advice, leak data through prompts, or undermine trust in automated workflows. Security teams need vocabulary that connects model behavior to practical controls.
Relevant for vendor due diligence, secure deployment reviews, AI audit evidence, red-team programs, and incident response for LLM applications.
Core model and adaptation concepts that shape security and governance choices.
Security risks that target instructions, data, retrieval context, and model behavior.
Controls used to reduce unsafe outputs, measure failures, and make systems auditable.
A side-by-side comparison of Retrieval-Augmented Generation and Fine-Tuning. Understand when to retrieve external context at runtime and when to adapt a pretrained model through additional training.
Distinguish between Prompt Injection (hijacking model behavior via malicious input) and Jailbreaking (bypassing safety guardrails to produce restricted content). Both are LLM security threats but differ in mechanism and goal.
A side-by-side comparison of Prompt Injection and Data Poisoning. Understand how runtime instruction manipulation differs from attacks on training data or the training process.
A side-by-side comparison of Guardrails and Content Moderation / Safety Filters. Understand the difference between broad AI system boundaries and specific controls for harmful or disallowed content.
A side-by-side comparison of System Prompt and User Prompt. It explains how application-level behavior instructions differ from the immediate task or question supplied by a user.
A side-by-side comparison of Red Teaming and AI Audit. Understand how the terms differ, when each applies, and what the distinction means for AI governance, system design, or assurance evidence.
A side-by-side comparison of Security By Design and Guardrails. Understand how lifecycle security planning differs from specific controls that keep AI systems within acceptable boundaries.
A side-by-side comparison of Hallucination and Confabulation. Understand how both describe plausible but unsupported or fabricated AI output, and why hallucination is often used as the broader operational term.
A side-by-side comparison of Grounding and Groundedness. Understand the difference between the process of connecting outputs to sources and the property of an output being supported by evidence.
Retrieval Augmented Generation (RAG) is a technique that combines generative AI with retrieval from external data sources. The model uses retrieved documents, passages, or records as grounding context to produce responses that are more current, specific, and verifiable.
Guardrails are technical, procedural, or policy controls designed to keep AI systems within acceptable boundaries. They may reduce risks such as harmful outputs, data leakage, unauthorized access, policy violations, or unsafe behavior.
Grounding is the process of connecting an AI model’s output to trusted, verifiable sources of information at inference time. In generative AI, it helps improve factual reliability by supplying relevant context without changing the model’s underlying weights.
A hallucination is an AI-generated output that appears plausible or confident but is false, unsupported, misleading, or fabricated. In generative AI, hallucinations may include incorrect facts, invented sources, false citations, or reasoning that is not grounded in reliable evidence.
Use system prompt for durable application instructions and constraints; use user prompt for the user’s immediate request, context, and desired output.
Red teaming is a structured adversarial testing process used to identify weaknesses, unsafe behavior, or harmful outputs in an AI system. It involves deliberately probing the system with challenging, malicious, or edge-case inputs so developers can evaluate and improve safeguards.
Data poisoning is an adversarial attack in which a malicious actor interferes with the data or training process used to build a model. By inserting, modifying, or corrupting training examples, the attacker seeks to manipulate model behavior, degrade performance, or create targeted failures.
Content moderation and safety filters are controls designed to detect, block, reduce, or escalate harmful, illegal, or policy-disallowed content. In AI systems, they are often combined with policy rules, model-based classifiers, logging, and human review for higher-risk cases.