Efficient sparse collective communication and its application to accelerate distributed deep learning
2021
Efficient collective communication is crucial to parallel-computing applications such as distributed training of large-scale recommendation systems and natural language processing models. Existing collective communication libraries focus on optimizing operations for dense inputs, resulting in transmissions of many zeros when inputs are sparse. This counters current trends that see increasing data sparsity in large models.
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
56
References
0
Citations
NaN
KQI