Beginner
What is the EU AI Act?
The EU AI Act is a European Union regulation establishing a risk-based framework for AI systems and, in some cases, general-purpose AI models. It defines categories of AI use and assigns obligations related to safety, transparency, governance, and compliance according to the level of risk.
Who is a provider under the EU AI Act?
A provider is a person, organization, public authority, agency, or other body that develops an AI system or general-purpose AI model, or has one developed, and places it on the market or puts it into service under its own name or trademark.
How is a provider different from a deployer?
Use Provider for the actor that develops, places on the market, or puts into service an AI system or GPAI model under its name; use Deployer for the actor that uses an AI system under its authority.
What is conformity assessment for an AI system?
Conformity assessment is the process of determining whether a system, product, or process satisfies specified legal or technical requirements.
What is LLM security?
LLM, or large language model, is a language model with a large number of parameters trained on broad text or multimodal data to understand and generate language.
What is prompt injection?
Prompt injection is an adversarial technique that manipulates a language model by placing malicious or unintended instructions in user input, documents, web pages, or other content the model processes.
What is a jailbreak in an LLM system?
Prompt injection or jailbreak refers to adversarial input intended to bypass a model's instructions, safety policies, or operational controls. Mitigations include input filtering, retrieval isolation, least-privilege tool access, policy enforcement, and layered guardrails.
How is RAG different from fine-tuning?
Use RAG when the model needs current, source-grounded information; use fine-tuning when the model needs specialized behavior, task performance, domain adaptation, or style learned from data.
What is AI governance?
AI Governance is the framework of policies, processes, roles, controls, and oversight used to steer AI development and use across its lifecycle. It aims to ensure responsible, lawful, secure, transparent, and accountable AI practice within an organization or jurisdiction.
What is an AI management system?
An AI Management System is an organizational platform or structured set of processes for managing AI projects and systems throughout their lifecycle. It coordinates governance, data flows, model development, deployment, monitoring, compliance, and accountability activities.
What is ISO/IEC 42001?
ISO/IEC 42001 is an international standard for establishing, implementing, maintaining, and improving an AI management system. It helps organizations govern AI development and use through policies, roles, risk management, oversight, and continual improvement.
What is an AI inventory system register?
An AI Inventory or System Register is an organized catalogue of AI systems and use cases within an organization. It typically records owners, purposes, data sources, deployment status, risks, controls, and other information needed for governance, audit, and accountability.
What is personal data in AI?
Personal data is information relating to an identified or identifiable natural person under data protection law. In EU contexts, its meaning is tied to Article 4(1) of Regulation (EU) 2016/679 and related legal interpretation.
What is special-category data in AI?
Special categories of personal data are highly sensitive types of personal data identified in EU data protection law.
Is biometric data personal data?
Biometric data is personal data produced by specific technical processing of a person's physical, physiological, or behavioral characteristics. Examples include facial images, fingerprints, voice patterns, gait, and other identifiers that can support recognition or identification.
What is anonymisation in AI data governance?
Anonymisation and de-identification are techniques used to reduce the risk that individuals can be identified from a dataset. In practice, irreversible anonymisation can be difficult, especially when datasets can be linked with other information.
What is AI safety?
AI Safety is the interdisciplinary field and set of practices focused on preventing harm from AI systems. It includes reliability, misuse prevention, bias and error mitigation, alignment with human values, and analysis of both near-term and long-term risks.
How is risk different from harm?
Use risk for potential adverse outcomes before or during assessment; use harm for the adverse effect that has occurred or may be concretely described.
What is a safety case for AI?
An AI safety case is a documented argument, supported by evidence, that an AI system is acceptably safe for a defined use and operating context. It is especially relevant for higher-risk sectors where assurance must be explicit, reviewable, and maintained over time.
What is model drift?
Model drift is the degradation or change in model performance over time as real-world data, user behavior, or operating conditions diverge from the data used during training or evaluation. Detecting drift is important for deciding when to retrain, recalibrate, or replace a model.
What is model evaluation?
Evaluation is the process of measuring the quality, behavior, or performance of a model, system, or change against defined criteria. In machine learning it may use validation and test data, while LLM evaluation may also include safety, factuality, robustness, and user-impact assessments.
What is a benchmark in AI evaluation?
A benchmark is a standardized test, dataset, task, or evaluation procedure used to measure and compare the performance of AI systems.
How are precision and recall different?
Use Precision when false positives are costly; use Recall when missed positives are costly.
Why can accuracy be misleading?
Accuracy is a classification metric equal to the proportion of predictions that are correct among all predictions made. It is useful in some evaluation settings but can be misleading when classes are imbalanced, where precision, recall, or other metrics may better reflect model performance.
What is agentic AI?
Agentic AI refers to AI systems that can pursue objectives by perceiving context, making decisions, planning steps, and taking actions with some degree of autonomy.
What is an AI agent?
An AI Agent is an entity or software system that perceives an environment, interprets inputs, and takes actions to achieve goals.
What is an agentic workflow?
An Agentic Workflow is a process in which an AI agent plans and carries out multiple steps toward a defined goal. It may include reasoning, tool invocation, external system interaction, monitoring, and self-correction as conditions change.
What is an agentic loop?
An Agentic Loop is the repeated cycle through which an AI agent observes a situation, reasons about what to do, acts, and uses feedback to continue or stop. The loop continues until a goal is achieved, a failure condition occurs, or another termination rule is met.
What is biometric AI?
Biometric technology refers to systems that use biometric data to recognize, verify, or identify a person's identity. Examples include facial recognition, fingerprint scanning, iris recognition, voice recognition, and related identity-assurance tools.
What is biometric data?
Biometric data is personal data produced by specific technical processing of a person's physical, physiological, or behavioral characteristics. Examples include facial images, fingerprints, voice patterns, gait, and other identifiers that can support recognition or identification.
What is biometric identification?
Biometric identification is the automated recognition of a natural person's identity by comparing that person's biometric data against biometric data stored in a database. It is typically a one-to-many matching process used to establish who the person is.
How is biometric verification different from biometric identification?
Use Biometric Identification when the focus is automated recognition of a natural person's identity by comparing that person's biometric data against biometric data stored in a database; use Biometric Verification when the focus is automated one-to-one confirmation of a person's identity by comparing their biometric.
What is machine learning?
Machine Learning (ML) is a subfield of artificial intelligence in which systems learn patterns from data or experience to make predictions, classifications, recommendations, or decisions without being explicitly programmed for every case.
What is supervised learning?
Supervised learning is a machine learning approach in which a model learns from examples that pair inputs with known target outputs or labels. The model uses those labeled examples to learn mappings that can support prediction, classification, or regression on new data.
How is unsupervised learning different from supervised learning?
Use Supervised Learning when labeled input-output examples exist; use Unsupervised Learning when the goal is to discover structure without labels.
What is reinforcement learning?
Reinforcement Learning (RL) is a machine learning paradigm in which an agent learns to act in an environment by receiving rewards or penalties for its actions. The objective is to learn a policy that maximizes expected cumulative reward over time.
What is generative AI?
Generative AI (GenAI) refers to AI systems that create new content, such as text, images, audio, video, code, or other media, in response to prompts or context. These systems learn patterns from training data and generate outputs that resemble or transform the kinds of content they were trained on.
How is generative AI different from artificial intelligence?
Use Artificial Intelligence for the broad field and system category; use Generative AI when the system creates new content from prompts or context.
What is a large language model?
LLM, or large language model, is a language model with a large number of parameters trained on broad text or multimodal data to understand and generate language.
What is a diffusion model?
A diffusion model is a generative model trained to reverse a process that gradually adds noise to data. By learning to denoise from random noise toward a target data distribution, it can generate realistic images, audio, text representations, or other outputs.
Intermediate
When is an AI system placed on the market?
Placing on the market means the first making available of an AI system or general-purpose AI model on the EU market. The term is important because certain regulatory obligations attach at or before this point.
What is the intended purpose of an AI system?
Intended purpose is the use for which an AI system is designed, marketed, or deployed by its provider. It defines the expected context, users, functions, outputs, and limitations, and is central to risk classification, compliance, and accountability.
What is reasonably foreseeable misuse?
Reasonably foreseeable misuse is the use of an AI system outside its intended purpose in a way that can still be anticipated from human behavior or interaction with other systems.
What is CE marking for AI systems?
CE marking is a conformity mark indicating that a product or AI system meets applicable requirements under relevant European Union law. For AI systems, it signals that the provider has assessed compliance with the legal obligations that apply to the system.
What is general-purpose AI?
General-Purpose AI (GPAI) refers to AI models or systems that can be adapted to a wide range of tasks and applications. Foundation models are a common example, in contrast to narrow AI systems designed for a specific task.
What is systemic risk in GPAI?
Systemic risk is a risk associated with high-impact capabilities of general-purpose AI models that can produce significant effects across markets, society, public health, safety, security, or fundamental rights.
Why can RAG create privacy and security risk?
Retrieval Augmented Generation (RAG) is a technique that combines generative AI with retrieval from external data sources. The model uses retrieved documents, passages, or records as grounding context to produce responses that are more current, specific, and verifiable.
What are guardrails in an LLM system?
Guardrails are technical, procedural, or policy controls designed to keep AI systems within acceptable boundaries. They may reduce risks such as harmful outputs, data leakage, unauthorized access, policy violations, or unsafe behavior.
What is grounding in generative AI?
Grounding is the process of connecting an AI model’s output to trusted, verifiable sources of information at inference time. In generative AI, it helps improve factual reliability by supplying relevant context without changing the model’s underlying weights.
What is hallucination in an AI system?
A hallucination is an AI-generated output that appears plausible or confident but is false, unsupported, misleading, or fabricated. In generative AI, hallucinations may include incorrect facts, invented sources, false citations, or reasoning that is not grounded in reliable evidence.
How is a system prompt different from a user prompt?
Use system prompt for durable application instructions and constraints; use user prompt for the user’s immediate request, context, and desired output.
What is red teaming for LLMs?
Red teaming is a structured adversarial testing process used to identify weaknesses, unsafe behavior, or harmful outputs in an AI system. It involves deliberately probing the system with challenging, malicious, or edge-case inputs so developers can evaluate and improve safeguards.
How is AI risk assessment different from AI audit?
Use AI Risk Assessment to identify and manage risks; use AI Audit to evaluate evidence, controls, or claims against defined criteria.
What is AI assurance?
AI Assurance is the set of practices that provide justified confidence that an AI system works as intended in its operational context.
What is shadow AI?
Shadow AI is the use of AI tools or systems without formal approval, oversight, or governance by an organization. It creates risks around data leakage, compliance, security, and inconsistent decision-making, and is typically managed through policy, approved alternatives, training, and monitoring.
What is an AI use policy?
An AI Use Policy is an internal governance document that defines how AI tools and systems may be used within an organization. It typically covers acceptable and prohibited uses, roles and responsibilities, data handling, approval requirements, monitoring, and incident escalation.
What is vendor due diligence for AI?
AI vendor due diligence is the assessment of third-party AI products or providers for security, privacy, reliability, governance, and compliance risks. It helps organizations decide whether and how a vendor can be safely approved, procured, monitored, or restricted.
What is algorithmic accountability?
Algorithmic Accountability is responsibility for the design, operation, decisions, and impacts of algorithmic systems. It involves transparency, traceability, oversight, and mechanisms for explaining, contesting, or remedying harmful outcomes.
How is pseudonymisation different from anonymisation?
Use Anonymisation / De-identification when discussing reduced or removed identifiability; use Pseudonymisation when personal data can still be attributed with protected additional information.
What is differential privacy?
Use Data Anonymisation when modifying data to reduce identifiability; use Differential Privacy when limiting individual inference from outputs through a privacy framework.
Can synthetic data still create privacy risk?
Synthetic data is artificially generated data designed to resemble selected patterns, structures, or statistical properties of real data.
What is data provenance?
Data provenance is the recorded history of data, including its origin, creation, transformations, movements, and changes over time. In AI governance, provenance helps verify dataset lineage, assess trustworthiness, support audits, and understand how digital content or model inputs were produced.
What is data residency in AI?
Data residency and sovereignty concern where data is stored, processed, and governed, and which legal regimes apply to it. For regulated AI use, these concepts affect cloud-region selection, cross-border transfers, contractual controls, and compliance with sectoral or national requirements.
How do GDPR and the EU AI Act overlap?
Use GDPR when the issue is processing personal data; use the EU AI Act when the issue is AI system risk category, safety, transparency, governance, or conformity obligations.
How is data drift different from concept drift?
Use Data Drift when input data distribution changes; use Concept Drift when the relationship between inputs and the target outcome changes.
What is alignment in AI safety?
Alignment is the practice of designing, training, and governing an AI system so that its goals, outputs, and behavior remain consistent with human intentions, values, and acceptable use constraints.
What is a serious AI incident?
An AI Incident is an event or series of events in which the development, deployment, use, or malfunction of an AI system causes or could cause harm. Such harm may involve safety, discrimination, privacy, legal compliance, critical infrastructure, property, communities, or the environment.
What is human-in-the-loop control?
HITL, or human-in-the-loop, is an approach in which people remain involved in shaping, reviewing, approving, or improving AI system outputs or decisions. It is used to combine machine efficiency with human judgment, especially in higher-risk or context-sensitive tasks.
What is AI toxicity?
AI Toxicity is harmful, offensive, false, biased, or otherwise inappropriate output generated or amplified by an AI system. It may arise from training data, system design, prompt context, misalignment, or inadequate safeguards.
Why does post-market monitoring matter?
A post-market monitoring system consists of activities by AI system providers to collect and review experience from systems placed on the market or put into service. Its purpose is to identify whether corrective or preventive action is needed after deployment.
What is an F1 score?
The F1 score is the harmonic mean of precision and recall, used to evaluate classification and information extraction performance. It ranges from 0 to 1, with higher values indicating a better balance between correctly identified positives and missed positives.
How are false positives and false negatives different?
Use False Positive for an incorrect positive flag; use False Negative for an incorrect failure to detect a real positive condition.
What is a classification threshold?
A classification threshold is a chosen cutoff value used to convert a model's score or probability into a class prediction. Changing the threshold can alter the balance between false positives and false negatives, making threshold selection an important governance and performance decision.
What is validation data?
Validation data is data used to evaluate a model during development and tune choices such as hyperparameters, thresholds, or learning processes. It should be distinct from training and test data to help detect underfitting, overfitting, and generalization problems.
What is a test set?
A test set is a reserved dataset used to provide an independent evaluation of a trained model or AI system. It should be separated from training and validation data so that performance estimates better reflect behavior on unseen data before deployment or release.
What is model monitoring?
Model monitoring is the continuous observation of a deployed model's performance, inputs, outputs, drift, errors, latency, and operational health. It helps detect degradation, misuse, bias, security issues, and conditions that require retraining or intervention.
What is AI orchestration?
AI Orchestration is the coordination and routing of AI tasks, workflows, model calls, tools, and agents within a system. It helps break complex processes into manageable steps and assign them to the appropriate models, sub-agents, or automation components.
What is function calling in AI agents?
Function calling is a capability that allows a language model to invoke external tools, APIs, or predefined functions. It extends model behavior beyond text generation by enabling structured actions, retrieval, computation, or system integration.
What is the Model Context Protocol?
Model Context Protocol (MCP) is a protocol for connecting AI models or assistants to external tools, data sources, and services through a standardized interface. It helps separate model behavior from tool integration and can improve portability, governance, and context management.
What is action space in agentic AI?
Action Space is the set of tools, operations, permissions, or moves available to an agent when performing a task. A well-scoped action space should be sufficient for the task while limiting unnecessary capabilities that may increase errors or governance risk.
Why do permissions matter for AI agents?
IAM, or Identity and Access Management permissions, are granular access controls that define which principals may perform which actions on which resources. They are commonly assigned through roles to users, groups, service accounts, or applications to enforce least-privilege access.
What is a router agent?
A router agent is an agent that classifies a user request or task and directs it to the most appropriate specialized agent, tool, or workflow. It is used in multi-agent systems to coordinate responsibilities and improve task handling.
What is a remote biometric identification system?
A remote biometric identification system is an AI system used to identify natural persons without their active involvement, typically at a distance. It compares a person's biometric data with biometric data stored in a reference database.
What is real-time remote biometric identification?
A real-time remote biometric identification system is a system in which biometric data capture, comparison, and identification occur without significant delay. The category includes instant identification as well as short delays intended to prevent circumvention.
What is an emotion recognition system?
An emotion recognition system is an AI system intended to identify or infer the emotions or intentions of natural persons, often from biometric data. Because such systems can affect privacy, autonomy, and fairness, they are subject to heightened scrutiny in governance and regulatory contexts.
What is biometric categorisation?
A biometric categorisation system is an AI system that assigns natural persons to specific categories on the basis of biometric data.
Why is facial recognition high risk?
Facial Recognition Technology (FRT) refers to automated systems that identify, verify, or categorize people by analyzing facial features from digital images or video. It is a biometric technology and may raise privacy, accuracy, and discrimination concerns depending on its use.
What are deepfakes in biometric AI?
Deepfakes are AI-generated or AI-manipulated images, audio, or video that make people, objects, places, entities, or events appear authentic when they are not.
What is self-supervised learning?
Self-supervised learning is a machine learning approach that creates training signals from unlabeled data itself. It converts aspects of an unsupervised problem into supervised learning by generating surrogate labels, making it useful when labeled data is scarce but unlabeled data is abundant.
What is semi-supervised learning?
Semi-supervised learning is a machine learning approach that uses both labeled and unlabeled data during training. It is useful when labeled examples are costly or limited but larger volumes of unlabeled data can help improve representation, classification, or prediction.
What is training data?
Training data is the dataset used to teach or fit a machine learning model by adjusting its learnable parameters. Its quality, quantity, diversity, labeling, and provenance strongly influence the performance, reliability, and compliance posture of the resulting model.
What is overfitting?
Overfitting occurs when a model learns the training data too closely, including noise or idiosyncratic details, and performs poorly on new data. It is commonly indicated by high training performance but weaker validation or test performance.
What is underfitting?
Underfitting occurs when a model is too simple or insufficiently trained to capture meaningful patterns in the data. An underfit model performs poorly on both training data and new data because it has not learned the underlying relationships needed for the task.
What is a decision tree?
A decision tree is an interpretable supervised learning model that represents decision rules in a tree-like structure. Inference follows a path from a root node through feature-based conditions to a leaf node that provides the predicted class or value.
What is a generative adversarial network?
Generative Adversarial Network (GAN) is a machine learning architecture in which a generator and discriminator are trained in opposition. The generator learns to create realistic samples, while the discriminator learns to distinguish generated samples from real data.
What is a deepfake?
Deepfakes are AI-generated or AI-manipulated images, audio, or video that make people, objects, places, entities, or events appear authentic when they are not.
What is synthetic media?
Synthetic media is media content that is wholly or partly generated or manipulated by AI. It can include text, images, audio, video, avatars, voices, and other digital artifacts.
What is prompt engineering?
Prompt engineering is the practice of designing, testing, and refining inputs to generative AI models so they produce desired outputs more reliably. It can involve specifying roles, constraints, examples, context, format, and evaluation criteria without changing the model's underlying weights.
What is temperature in generative AI?
Temperature is a model-generation parameter that controls the randomness of an AI system’s output. Lower values tend to produce more focused and deterministic responses, while higher values increase variety, creativity, and the risk of less coherent or less reliable output.
What is AI watermarking?
AI Watermarking is the embedding of visible or hidden identifiers into AI-generated or AI-modified content to indicate origin, provenance, or generation status. Watermarks may be added during generation or after generation and may require specialized tools to detect.
Advanced
Who supervises AI systems in the EU?
The AI Office is the European Commission function established to support implementation, monitoring, and supervision of AI systems, general-purpose AI models, and AI governance under the EU AI framework.
What is a serious incident under AI governance?
A serious incident is an AI-system incident or malfunction that directly or indirectly causes severe harm, such as death, serious health harm, critical infrastructure disruption, fundamental-rights infringement, or serious property or environmental damage.
What is data poisoning in AI systems?
Data poisoning is an adversarial attack in which a malicious actor interferes with the data or training process used to build a model. By inserting, modifying, or corrupting training examples, the attacker seeks to manipulate model behavior, degrade performance, or create targeted failures.
What is content moderation for AI outputs?
Content moderation and safety filters are controls designed to detect, block, reduce, or escalate harmful, illegal, or policy-disallowed content. In AI systems, they are often combined with policy rules, model-based classifiers, logging, and human review for higher-risk cases.
What is an AI model card?
An AI Model Card is a concise documentation artifact that accompanies a trained model or AI service. It typically describes intended use, limitations, evaluation results, relevant risks, data or training context, and other information that supports transparency and responsible deployment.
What is transparency in AI governance?
Transparency is the degree to which an AI system’s purpose, data use, operation, limitations, and outputs can be understood or examined by relevant stakeholders.
What is training data privacy risk?
Training data is the dataset used to teach or fit a machine learning model by adjusting its learnable parameters. Its quality, quantity, diversity, labeling, and provenance strongly influence the performance, reliability, and compliance posture of the resulting model.
What is encryption in AI data governance?
Data encryption is the transformation of readable data into an encoded form that is unintelligible without an authorized key or decryption process. It is used to protect sensitive information from unauthorized access during storage, transmission, or processing.
What is anomaly detection in AI monitoring?
Anomaly detection is the process of identifying observations, events, or patterns that differ significantly from expected behavior. In AI and data analysis, it is used to flag outliers that may indicate errors, fraud, security incidents, system faults, or other unusual conditions.
What is security by design for AI systems?
Security by design is an approach in which security requirements and safeguards are integrated from the start of system development. For AI systems, it includes anticipating misuse, protecting data and models, and embedding access controls, monitoring, and resilience throughout the lifecycle.
What is groundedness in LLM evaluation?
Groundedness is the property of a model output being supported by specific source material or provided context. In generative AI, groundedness helps assess whether an answer is traceable to evidence rather than unsupported model generation.
What is unsupported-claim rate?
UCR (unsupported-claim rate) is the percentage of claims in a model response that are not grounded in supporting evidence. A high UCR indicates that the system may be producing too many unsupported or hallucinated assertions.
What is a multi-agent system?
Multi-agent systems are systems composed of multiple interacting agents that can perceive, decide, communicate, and act within an environment. They are used to model distributed problem solving, coordination, negotiation, competition, and collective behavior.
How should agentic AI be governed?
AI Governance is the framework of policies, processes, roles, controls, and oversight used to steer AI development and use across its lifecycle. It aims to ensure responsible, lawful, secure, transparent, and accountable AI practice within an organization or jurisdiction.
What is synthetic media?
Synthetic media is media content that is wholly or partly generated or manipulated by AI. It can include text, images, audio, video, avatars, voices, and other digital artifacts.
How can biometric AI harm fairness?
Fairness in AI refers to the principle that AI systems should treat individuals and groups equitably and avoid unjustified discrimination or harmful bias. It involves identifying and mitigating bias in data, models, deployment contexts, and decision processes.
What is deep learning?
Deep learning is a subfield of machine learning that uses neural networks with multiple layers to learn representations and patterns from data.
How are classification and regression different?
Use Classification when the target is a category; use Regression when the target is a continuous or numerical value.
What is content provenance?
Provenance and watermarking refer to signals that help identify how content was created, where it came from, or whether it was AI-generated or modified. These mechanisms support authenticity, trust, moderation, auditability, and responsible content governance.
Why does generated content need human review?
Human evaluation is the process of having people judge the quality, safety, usefulness, or correctness of model outputs. It is especially important for tasks where there is no single correct answer, such as translation quality, summarization, dialogue, or creative generation.