Intermediate
Why can RAG create privacy and security risk?
Retrieval Augmented Generation (RAG) is a technique that combines generative AI with retrieval from external data sources. The model uses retrieved documents, passages, or records as grounding context to produce responses that are more current, specific, and verifiable.
What are guardrails in an LLM system?
Guardrails are technical, procedural, or policy controls designed to keep AI systems within acceptable boundaries. They may reduce risks such as harmful outputs, data leakage, unauthorized access, policy violations, or unsafe behavior.
What is grounding in generative AI?
Grounding is the process of connecting an AI model’s output to trusted, verifiable sources of information at inference time. In generative AI, it helps improve factual reliability by supplying relevant context without changing the model’s underlying weights.
What is hallucination in an AI system?
A hallucination is an AI-generated output that appears plausible or confident but is false, unsupported, misleading, or fabricated. In generative AI, hallucinations may include incorrect facts, invented sources, false citations, or reasoning that is not grounded in reliable evidence.
How is a system prompt different from a user prompt?
Use system prompt for durable application instructions and constraints; use user prompt for the user’s immediate request, context, and desired output.
What is red teaming for LLMs?
Red teaming is a structured adversarial testing process used to identify weaknesses, unsafe behavior, or harmful outputs in an AI system. It involves deliberately probing the system with challenging, malicious, or edge-case inputs so developers can evaluate and improve safeguards.
Advanced
What is data poisoning in AI systems?
Data poisoning is an adversarial attack in which a malicious actor interferes with the data or training process used to build a model. By inserting, modifying, or corrupting training examples, the attacker seeks to manipulate model behavior, degrade performance, or create targeted failures.
What is content moderation for AI outputs?
Content moderation and safety filters are controls designed to detect, block, reduce, or escalate harmful, illegal, or policy-disallowed content. In AI systems, they are often combined with policy rules, model-based classifiers, logging, and human review for higher-risk cases.