Multi-instance semantic similarity transferring for knowledge distillation

Knowledge distillation is a popular paradigm for learning portable neural networks by transferring the knowledge from a large model into a smaller one…