Face recognition based on deep feature distillation

GE Shiming; ZHAO Shengwei; LIU Wenyu; LI Chenyu

doi:10.11860/j.issn.1673-0291.2017.06.005

Abstract：Deep learning has been widely used in face recognition system due to its powerful ability in feature representation.However,the high inferring complexity and feature representation re-duce the efficiencies in feature extraction and retrieval respectively,which hinders the practical deployments of face recognition system.To address these issues,this paper proposes deep feature distillation in order to uniformly compress the deep network parameters and feature dimensions by distilling the knowledge from large teacher network and domain related data via multi-task deep learning.Combined feature regression and face classification,the method uses a pre-trained large depth network as a teacher network to guide the training of small network,which the knowledge transferred to the lightweight student network to achieve efficient feature extraction. The experimental results on LFW benchmark show that in the condition of the student model rec-ognition accuracy is reduced by 3.7％ compared with the teacher model,the network has been compressed to about 2×107 in model size and 128 dimensional feature,which achieves the reduc-tions of 7.1 times in model parameters,32 times in feature dimension and 95.1％ in inferring complexity.The results demonstrate the validity and efficiency of the proposed method.