Tangram: Hiding GPU Heterogeneity for Efficient LLM Parallelization
#GPUcluster #LLM #Performance
https://hgpu.org/?p=30879
Tangram: Hiding GPU Heterogeneity for Efficient LLM Parallelization
The scale of LLM training jobs requires parallelization planning over large GPU clusters. Due to different GPU types and interconnects added over time, these GPU clusters are increasingly heterogen…
HGPU group