Sun 27 March 2022
The Case for Tiny Tasks in Compute Clusters
If we break down tasks in massively data-parallel jobs so they complete in < 1s, we could more evenly spread work across compute clusters, improving job runtime and resource utilization.