Преобразование данных — это conversion данных в format, structure или representation, suitable for analysis, indexing, modeling или processing. В retrieval-augmented generation оно может включать cleaning, validation, chunking, formatting и preparation for embedding или search.
In the context of Retrieval Augmented Generation (RAG), data transformation refers to the conversion of data into a format suitable for indexing and processing by an LLM. This often involves splitting data into smaller chunks to make it manageable for embedding and indexing. Other transformations might include cleaning and validation steps to ensure data quality. For more information, see RAG Engine overview.