A host is the part of a machine learning training or inference system that manages overall program flow and input processing when accelerators such as GPUs or TPUs are used. The host typically runs on a CPU, while accelerator devices perform tensor computation.
When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that controls both of the following: - The overall flow of the code. - The extraction and transformation of the input pipeline. The host typically runs on a CPU, not on an accelerator chip; the device manipulates tensors on the accelerator chips.