When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that controls both of the following: - The overall flow of the code. - The extraction and transformation of the input pipeline. The host typically runs on a CPU, not on an accelerator chip; the device manipulates tensors on the accelerator chips.