Офлайн-инференс — процесс generating predictions in batches and storing them for later use. Он useful when predictions can be precomputed and served from a cache rather than calculated on demand.
The process of a model generating a batch of predictions and then caching (saving) those predictions. Apps can then access the inferred prediction from the cache rather than rerunning the model. For example, consider a model that generates local weather forecasts (predictions) once every four hours. After each model run, the system caches all the local weather forecasts. Weather apps retrieve the forecasts from the cache. Offline inference is also called static inference. Contrast with online inference. See Production ML systems: Static versus dynamic inference in Machine Learning Crash Course for more information.