A replica is a copy of a model, training set, or model component stored or executed on another machine or device. Replicas are used in distributed training and inference, such as data parallelism, to improve scalability and throughput.
A copy (or part of) of a training set or model, typically stored on another machine. For example, a system could use the following strategy for implementing data parallelism: 1. Place replicas of an existing model on multiple machines. 2. Send different subsets of the training set to each replica. 3. Aggregate the parameter updates. A replica can also refer to another copy of an inference server. Increasing the number of replicas increases the number of requests that the system can serve simultaneously but also increases serving costs.