Inference is the process of using a trained model to generate predictions, classifications, recommendations, or other outputs from input data. It differs from training because the model applies learned parameters rather than learning them from examples.
In traditional machine learning, the process of making predictions by applying a trained model to unlabeled examples. See Supervised Learning in the Intro to ML course to learn more. In large language models, inference is the process of using a trained model to generate a response to an input prompt. Inference has a somewhat different meaning in statistics. See the Wikipedia article on statistical inference for details.
In the context of the Vertex AI platform, inference refers to the process of running data points through a machine learning model to calculate an output, such as a single numerical score. This process is also known as "operationalizing a machine learning model" or "putting a machine learning model into production." Inference is an important step in the machine learning workflow, since it enables models to be used to make inferences on new data. In Vertex AI, inference can be performed in various ways, including batch inference and online inference. Batch inference involves running a group of inference requests and outputting the results in one file, while online inference allows for real-time inferences on individual data points.
In machine learning, the process of using a trained model to make predictions.