El aprendizaje por refuerzo con Transformer aplica arquitecturas basadas en Transformer a problemas de aprendizaje por refuerzo. El enfoque usa mecanismos de atención para modelar secuencias de estados, acciones, recompensas y contexto para aprendizaje de políticas, planificación o toma de decisiones.
Transformer Reinforcement Learning (TRL) refers to the application of reinforcement learning (RL) techniques to train transformer-based models for generative tasks. This approach addresses limitations of traditional generative models, which are often trained on next-token inference without explicit optimization for desirable qualities like coherence, safety, and sensibility. TRL directly optimizes the language model on complex objectives by using RL, often incorporating human feedback (RLHF) to guide the learning process. Examples include fine-tuning models to generate less toxic content using reward models and using TRL to fine-tune Gemma, a generative model. For more information, see Hugging Face DLCs: Fine-tuning Gemma with Transformer Reinforcement Learning (TRL) on Vertex AI.