A mesh is a distributed computing configuration in which devices or accelerators are organized for coordinated computation. In machine learning, mesh layouts can define how tensors, workloads, or model components are distributed across hardware resources.
In ML parallel programming, a term associated with assigning the data and model to TPU chips, and defining how these values will be sharded or replicated. Mesh is an overloaded term that can mean either of the following: - A physical layout of TPU chips. - An abstract logical construct for mapping the data and model to the TPU chips. In either case, a mesh is specified as a shape.