Deep Learning Face Representation by Joint Identification Verification(联合独立验证深度学习人脸)
自然场景图片中的文本检测和定位
[10] He Kaiming, Zhang Xiangyu, Ren Shaoqing, etal. Deep residua 1 learning for image recognition[C] // 2016 IEEE Conf erence on Computer Vision and Pattern Recognit ion (CVPR), 2016: 770-778.
近年来，深度学习领域出现新的卷积神经网络模型，推动了文 本检测领域的发展。场景图片中的文本检测方法大致上可以划分基 于候选区域的方法和基于滑动窗口的方法。基于候选区域的方法主 要利用图像的边缘信息和角点信息，或者根据文本区域通常在灰度、 颜色等特征上的相似性，对文本区域的连通域进行合并作为候选 区域。此类方法通过检测场景图片中的连通域来作为文本的候选区 域。常见的基于候选区域的文本检测方法有Matas等人提出的最大 稳定极值区域（MSER）、极值区域的方法和笔画宽度变换（Stroke
[8JOROUGHI H, SHAKERI M, RAY N, et al. Face recognition us ing mult i-moda1 low-rank dictionary learning[C] // Proceedings of the 2017 IEEE Internationa1 Conference on Image Processing. Piscataway: IEEE, 2017: 1081-1086.
关键词：卷积神经网络（CNN ）;文本定位；最大稳定极值区域（MSER）
深度迁移学习深度学习
深度迁移学习一、深度学习1)ImageNet Classification with Deep Convolutional Neural Networks主要思想：该神经网络有6000万个参数和650,000个神经元，由五个卷积层，以及某些卷积层后跟着的max-pooling层，和三个全连接层，还有排在最后的1000-way的softmax层组成。
使用了非饱和的神经元和一个非常高效的GPU关于卷积运算的工具。
1、采用了最新开发的正则化方法，称为“dropout”。
2、采用ReLU来代替传统的tanh引入非线性，；3、采用2块显卡来进行并行计算，减少了更多显卡需要主机传递数据的时间消耗，在结构上，部分分布在不同显卡上面的前后层节点之间无连接，从而提高了训练速度；4、同层相邻节点的响应进行局部归一化提高了识别率（top5错误率降低1.2%）；5、有交叠的pooling（top5错误率降低0.3%）；体系架构：（1）ReLU训练带ReLUs的深度卷积神经网络比带tanh单元的同等网络要快好几倍。
如下图，带ReLU的四层卷积神经网络（实线）在CIFAR-10数据集上达到25%训练误差率要比带tanh神经元的同等网络（虚线）快六倍。
（2）在多个GPU上训练（3）局部响应归一化具体见Very Deep Convolutional Networks for Large-Scale Image Recognition（4）重叠Pooling每个网格间隔距离为s，而每一次进行降采样将从网格中心为中心，采样z*z个像素。
如果s=z，则与传统方法相同，而如果s<z，则会进行重复采样。
本文章将s=2，z=3，成功的将Top-1和Top-5的错误率分别降低了0.4%和0.3%（与s=2，z=2相比）。
而且，在实验中发现，采用重叠采样将会略微更难产生过拟合。
（5）总体结构该网络包括八个带权层；前五层是卷积层，剩下三层是全连接层。
深度学习转变图像识别
深度学习转变图像识别近年来，随着深度学习的快速发展，图像识别技术得到了前所未有的提升。
传统的图像处理技术已经无法满足现代社会对图像识别的高精度、高效率的需求。
深度学习通过模拟人脑神经网络的工作方式，为图像识别提供了全新的视角和方法，极大地推动了这一领域的进步。
深度学习的基本原理深度学习是一种基于人工神经网络的机器学习方法。
它通过构建多层次的神经网络来进行特征提取与模式识别。
其核心思想是通过大量的数据进行自我训练，从而自动提取特征，无需手动设计特征提取算法。
深度学习可以分为以下几个主要组成部分：神经元与神经网络：深度学习中的基本单位是神经元，每个神经元接收输入，进行加权计算，并通过激活函数产生输出。
多个神经元可以通过不同层次组合形成神经网络。
前馈与反向传播：在神经网络中，信息从输入层传递至输出层被称为前馈。
为了优化网络参数，使用反向传播算法根据输出结果调整权重，从而提高模型的准确性。
激活函数：激活函数用于引入非线性因素，使得神经网络能够更好地解决复杂问题。
常见的激活函数包括Sigmoid函数、ReLU （Rectified Linear Unit）和Tanh（双曲正切）等。
损失函数与优化器：损失函数用以衡量模型预测值与真实值之间的差距，而优化器则通过调整权重来最小化损失函数，从而提高模型性能。
图像识别的发展历程在深度学习兴起之前，传统的图像识别方法主要依靠手工设计特征。
这些方法包括但不限于边缘检测、纹理分析和形状匹配等。
然而，这些技术面对复杂场景时力不从心，尤其是处理多样化、复杂性高、变化迅速的自然图像时常常会出现错误。
2012年，尤尔根·许曼（Geoffrey Hinton）团队在ImageNet挑战赛中提出了卷积神经网络（CNN），标志着图像识别领域进入了深度学习时代。
此后，不同类型的深度学习模型相继被提出，包括但不限于以下几种：卷积神经网络（CNN）：最为经典和广泛应用的一种网络架构，特别适合处理二维图像，其优点在于能够自动提取局部特征并保持空间关系。
Deep Learning Face Representation from Predicting 10,000 Classes
Deep Learning Face Representation from Predicting10,000Classes Yi Sun1Xiaogang Wang2Xiaoou Tang1,31Department of Information Engineering,The Chinese University of Hong Kong2Department of Electronic Engineering,The Chinese University of Hong Kong3Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences sy011@.hk xgwang@.hk xtang@.hkAbstractThis paper proposes to learn a set of high-level feature representations through deep learning,referred to as Deep hidden IDentity features(DeepID),for face verification. We argue that DeepID can be effectively learned through challenging multi-class face identification tasks,whilst they can be generalized to other tasks(such as verification)and new identities unseen in the training set.Moreover,the generalization capability of DeepID increases as more face classes are to be predicted at training.DeepID features are taken from the last hidden layer neuron activations of deep convolutional networks(ConvNets).When learned as classifiers to recognize about10,000face identities in the training set and configured to keep reducing the neuron numbers along the feature extraction hierarchy,these deep ConvNets gradually form compact identity-related features in the top layers with only a small number of hidden neurons.The proposed features are extracted from various face regions to form complementary and over-complete representations.Any state-of-the-art classifiers can be learned based on these high-level representations for face verification.97.45%verification accuracy on LFW is achieved with only weakly aligned faces.1.IntroductionFace verification in unconstrained conditions has been studied extensively in recent years[21,15,7,34,17,26, 18,8,2,9,3,29,6]due to its practical applications and the publishing of LFW[19],an extensively reported dataset for face verification algorithms.The current best-performing face verification algorithms typically represent faces with over-complete low-level features,followed by shallow models[9,29,6].Recently,deep models such as ConvNets[24]have been proved effective for extracting high-level visual features[11,20,14]and are used for face verification[18,5,31,32,36].Huang et al.[18] learned a generative deep model without supervision.Cai Figure1.An illustration of the feature extraction process.Arrows indicate forward propagation directions.The number of neurons in each layer of the multiple deep ConvNets are labeled beside each layer.The DeepID features are taken from the last hidden layer of each ConvNet,and predict a large number of identity classes. Feature numbers continue to reduce along the feature extraction cascade till the DeepID layer.et al.[5]learned deep nonlinear metrics.In[31],the deep models are supervised by the binary face verification target.Differently,in this paper we propose to learn high-level face identity features with deep models through face identification,i.e.classifying a training image into one of n identities(n≈10,000in this work).This high-dimensional prediction task is much more challenging than face verification,however,it leads to good generalization of the learned feature representations.Although learned through identification,these features are shown to be effective for face verification and new faces unseen in the training set.We propose an effective way to learn high-level over-complete features with deep ConvNets.A high-level illustration of our feature extraction process is shown in Figure1.The ConvNets are learned to classify all the faces available for training by their identities,with the last hidden layer neuron activations as features(referred to as2014 IEEE Conference on Computer Vision and Pattern RecognitionDeep hidden IDentity features or DeepID).Each ConvNet takes a face patch as input and extracts local low-level features in the bottom layers.Feature numbers continue to reduce along the feature extraction cascade while gradually more global and high-level features are formed in the top layers.Highly compact160-dimensional DeepID is acquired at the end of the cascade that contain rich identity information and directly predict a much larger number(e.g., 10,000)of identity classes.Classifying all the identities simultaneously instead of training binary classifiers as in [21,2,3]is based on two considerations.First,it is much more difficult to predict a training sample into one of many classes than to perform binary classification.This challenging task can make full use of the super learning capacity of neural networks to extract effective features for face recognition.Second,it implicitly adds a strong regularization to ConvNets,which helps to form shared hidden representations that can classify all the identities well.Therefore,the learned high-level features have good generalization ability and do not over-fit to a small subset of training faces.We constrain DeepID to be significantly fewer than the classes of identities they predict,which is key to learning highly compact and discriminative features. We further concatenate the DeepID extracted from various face regions to form complementary and over-complete rep-resentations.The learned features can be well generalized to new identities in test,which are not seen in training, and can be readily integrated with any state-of-the-art face classifiers(e.g.,Joint Bayesian[8])for face verification.Our method achieves97.45%face verification accuracy on LFW using only weakly aligned faces,which is almost as good as human performance of97.53%.We also observe that as the number of training identities increases,the verification performance steadily gets improved.Although the prediction task at the training stage becomes more challenging,the discrimination and generalization ability of the learned features increases.It leaves the door wide open for future improvement of accuracy with more training data.2.Related workMany face verification methods represent faces by high-dimensional over-complete face descriptors,followed by shallow models.Cao et al.[7]encoded each face image into 26K learning-based(LE)descriptors,and then calculated the L2distance between the LE descriptors after PCA.Chen et al.[9]extracted100K LBP descriptors at dense facial landmarks with multiple scales and used Joint Bayesian[8] for verification after PCA.Simonyan et al.[29]computed 1.7M SIFT descriptors densely in scale and space,encoded the dense SIFT features into Fisher vectors,and learned lin-ear projection for discriminative dimensionality reduction. Huang et al.[17]combined1.2M CMD[33]and SLBP [1]descriptors,and learned sparse Mahalanobis metrics for face verification.Some previous studies have further learned identity-related features based on low-level features.Kumar et al.[21]trained attribute and simile classifiers to detect facial attributes and measure face similarities to a set of reference people.Berg and Belhumeur[2,3]trained classifiers to distinguish the faces from two different people.Features are outputs of the learned classifiers.They used SVM classifiers,which are shallow structures,and their learned features are still relatively low-level.In contrast,we classify all the identities from the training set simultaneously.More-over,we use the last hidden layer activations as features instead of the classifier outputs.In our ConvNets,the neuron number of the last hidden layer is much smaller than that of the output,which forces the last hidden layer to learn shared hidden representations for faces of different people in order to well classify all of them,resulting in highly discriminative and compact features with good generalization ability.A few deep models have been used for face verification or identification.Chopra et al.[10]used a Siamese network [4]for deep metric learning.The Siamese network extracts features separately from two compared inputs with two identical sub-networks,taking the distance between the outputs of the two sub-networks as dissimilarity.[10] used deep ConvNets as the sub-networks.In contrast to the Siamese network in which feature extraction and recognition are jointly learned with the face verification target,we conduct feature extraction and recognition in two steps,with thefirst feature extraction step learned with the target of face identification,which is a much stronger supervision signal than verification.Huang et al.[18] generatively learned features with CDBNs[25],then used ITML[13]and linear SVM for face verification.Cai et al.[5]also learned deep metrics under the Siamese network framework as[10],but used a two-level ISA network[23] as the sub-networks instead.Zhu et al.[35,36]learned deep neural networks to transform faces in arbitrary poses and illumination to frontal faces with normal illumination,and then used the last hidden layer features or the transformed faces for face recognition.Sun et al.[31]used multiple deep ConvNets to learn high-level face similarity features and trained classification RBM[22]for face verification.Their features are jointly extracted from a pair of faces instead of from a single face.3.Learning DeepID for face verification3.1.Deep ConvNetsOur deep ConvNets contain four convolutional layers (with max-pooling)to extract features hierarchically,fol-lowed by the fully-connected DeepID layer and the softmax output layer indicating identity classes.The input is39×Figure 2.ConvNet structure.The length,width,and height of each cuboid denotes the map number and the dimension of each map for all input,convolutional,and max-pooling layers.The inside small cuboids and squares denote the 3D convolution kernel sizes and the 2D pooling region sizes of convolutional and max-pooling layers,respectively.Neuron numbers of the last two fully-connected layers are marked beside each layer.31×k for rectangle patches,and 31×31×k for square patches,where k =3for color patches and k =1for gray patches.Figure 2shows the detailed structure of the ConvNet which takes 39×31×1input and predicts n (e.g .,n =10,000)identity classes.When the input sizes change,the height and width of maps in the following layers will change accordingly.The dimension of the DeepID layer is fixed to 160,while the dimension of the output layer varies according to the number of classes it predicts.Feature numbers continue to reduce along the feature extraction hierarchy until the last hidden layer (the DeepID layer),where highly compact and predictive features are formed,which predict a much larger number of identity classes with only a few features.The convolution operation is expressed asy j (r )=max 0,b j (r )+ik ij (r )∗x i (r ),(1)where x i and y j are the i -th input map and the j -th outputmap,respectively.k ij is the convolution kernel between the i -th input map and the j -th output map.∗denotes convolution.b j is the bias of the j -th output map.We use ReLU nonlinearity (y =max (0,x ))for hidden neurons,which is shown to have better fitting abilities than the sigmoid function [20].Weights in higher convolutional layers of our ConvNets are locally shared to learn different mid-or high-level features in different regions [18].r in Equation 1indicates a local region where weights are shared.In the third convolutional layer,weights are locally shared in every 2×2regions,while weights in the fourth convolutional layer are totally unshared.Max-pooling is formulated asy ij,k =max 0≤m,n<sx i j ·s +m,k ·s +n ,(2)where each neuron in the i -th output map y i pools over an s ×s non-overlapping local region in the i -th input map x i.Figure 3.Top:ten face regions of medium scales.The five regionsin the top left are global regions taken from the weakly aligned faces,the other five in the top right are local regions centered around the five facial landmarks (two eye centers,nose tip,and two mouse corners).Bottom:three scales of two particular patches.The last hidden layer of DeepID is fully connected to both the third and fourth convolutional layers (after max-pooling)such that it sees multi-scale features [28](features in the fourth convolutional layer are more global than those in the third one).This is critical to feature learning because after successive down-sampling along the cascade,the fourth convolutional layer contains too few neurons and becomes the bottleneck for information propagation.Adding the bypassing connections between the third con-volutional layer (referred to as the skipping layer)and the last hidden layer reduces the possible information loss in the fourth convolutional layer.The last hidden layer takes the functiony j =max0,ix 1i ·w 1i,j +ix 2i ·w 2i,j +b j,(3)where x 1,w 1,x 2,w 2denote neurons and weights in thethird and fourth convolutional layers,respectively.It lin-early combines features in the previous two convolutional layers,followed by ReLU non-linearity.The ConvNet output is an n -way softmax predicting the probability distribution over n different identities.y i =exp(y i) n j =1exp(y j ),(4)where yj= 160i =1x i ·w i,j +b j linearly combines the 160DeepID features x i as the input of neuron j ,and y j is its output.The ConvNet is learned by minimizing −log y t ,with the t -th target class.Stochastic gradient descent is used with gradients calculated by back-propagation.3.2.Feature extractionWe detectfive facial landmarks,including the two eye centers,the nose tip,and the two mouth corners,with the facial point detection method proposed by Sun et al.[30]. Faces are globally aligned by similarity transformation according to the two eye centers and the mid-point of the two mouth corners.Features are extracted from60face patches with ten regions,three scales,and RGB or gray channels.Figure3shows the ten face regions and the three scales of two particular face regions.We trained 60ConvNets,each of which extracts two160-dimensional DeepID vectors from a particular patch and its horizontally flipped counterpart.A special case is patches around the two eye centers and the two mouth corners,which are not flipped themselves,but the patches symmetric with them (for example,theflipped counterpart of the patch centered on the left eye is derived byflipping the patch centered on the right eye).The total length of DeepID is19,200 (160×2×60),which is ready for thefinal face verification.3.3.Face verificationWe use the Joint Bayesian[8]technique for face ver-ification based on the DeepID.Joint Bayesian has been highly successful for face verification[9,6].It represents the extracted facial features x(after subtracting the mean) by the sum of two independent Gaussian variablesx=μ+ ,(5) whereμ∼N(0,Sμ)represents the face identity and ∼N(0,S )the intra-personal variations.Joint Bayesian models the joint probability of two faces given the intra-or extra-personal variation hypothesis,P(x1,x2|H I)and P(x1,x2|H E).It is readily shown from Equation5that these two probabilities are also Gaussian with variationsΣI=Sμ+S SμSμSμ+S(6)andΣE=Sμ+S 00Sμ+S,(7)respectively.Sμand S can be learned from data with EM algorithm.In test,it calculates the likelihood ratior(x1,x2)=log P(x1,x2|H I)P(x1,x2|H E),(8)which has closed-form solutions and is efficient.We also train a neural network for verification and com-pare it to Joint Bayesian to see if other models can also learn from the extracted features and how much the features and a good face verification model contribute to the performance, respectively.The neural network contains one inputlayer Figure 4.The structure of the neural network used for face verification.The layer type and dimension are labeled beside each layer.The solid neurons form a subnetwork.taking the DeepID,one locally-connected layer,one fully-connected layer,and a single output neuron indicating face similarities.The input features are divided into60 groups,each of which contains640features extracted from a particular patch pair with a particular ConvNet.Features in the same group are highly correlated.Neurons in the locally-connected layer only connect to a single group of features to learn their local relations and reduce the feature dimension at the same time.The second hidden layer is fully-connected to thefirst hidden layer to learn global relations.The single output neuron is fully connected to the second hidden layer.The hidden neurons are ReLUs and the output neuron is sigmoid.An illustration of the neural network structure is shown in Figure4.It has38,400input neurons with19,200DeepID features from each patch,and 4,800neurons in the following two hidden layers,with every80neurons in thefirst hidden layer locally connected to one of the60groups of input neurons.Dropout learning[16]is used for all the hidden neu-rons.The input neurons cannot be dropped because the learned features are compact and distributed representa-tions(representing a large number of identities with very few neurons)and have to collaborate with each other to represent the identities well.On the other hand,learning high-dimensional features without dropout is difficult due to gradient diffusion.To solve this problem,wefirst train60 subnetworks,each with features of a single group as input.A particular subnetwork is illustrated in Figure4.We then use thefirst-layer weights of the subnetworks to initialize those of the original network,and tune the second and third layers of the original network with thefirst layer weights clipped.4.ExperimentsWe evaluate our algorithm on LFW,which reveals the state-of-the-art of face verification in the wild.Though LFW contains5749people,only85have more than15 images,and4069people have only one image.It is inadequate to train identity classifiers with so few images per person.Instead,we trained our model on CelebFaces[31]and tested on LFW(Section4.1-4.3).CelebFaces contains87,628face images of5436celebrities from the Internet,with approximately16images per person on average.People in LFW and CelebFaces are mutually exclusive.We randomly choose80%(4349)people from Celeb-Faces to learn the DeepID,and use the remaining20% people to learn the face verification model(Joint Bayesian or neural networks).For feature learning,ConvNets are supervised to classify the4349people simultaneouslyfrom a particular kind of face patches and theirflipped counterparts.We randomly select10%images of each training person to generate the validation data.After each training epoch,we observe the top-1validation set error rates and select the model that provides the lowest one.In face verification,our feature dimension is reduced to150by PCA before learning the Joint Bayesian model. Performance almost retains in a wide range of dimensions. In test,each face pair is classified by comparing the Joint Bayesian likelihood ratio to a threshold optimized in the training data.To evaluate the performance of our approach at an even larger training scale in Section4.4,we extend CelebFaces to the CelebFaces+dataset,which contains202,599face images of10,177celebrities.Again,people in LFW and CelebFaces+are mutually exclusive.The ConvNet structure and feature extraction process described in the previous section remains unchanged.4.1.Multi-scale ConvNetsWe verify the effectiveness of directly connecting neu-rons in the third convolutional layer(after max-pooling) to the last hidden layer(the DeepID layer),such that it sees both the third and fourth convolutional layer features, forming the so-called multi-scale ConvNets.It also results in reducing feature numbers from the convolutional layers to the DeepID layer(shown in Figure1),which helps the latter to learn higher-level features in order to well represent the face identities with fewer neurons.Figure5compares the top-1validation set error rates of the60ConvNets learned to classify the4349classes of identities,either with or without the skipping layer.The lower error rates indicate the better hidden features learned.Allowing the DeepID to pool over multi-scale features reduces validation errors by an average of4.72%.It actually also improves thefinal face verification accuracy from95.35%to96.05%when concatenating the DeepID from the60ConvNets and using Joint Bayesian for face verification.4.2.Learning effective featuresClassifying a large number of identities simultaneously is key to learning discriminative and compact hidden features.To verify this,we increase the identity classes Figure5.Top-1validation set error rates of the60ConvNets trained on the60different patches.The blue and red markers show error rates of the conventional ConvNets(without the skipping layer)and the multi-scale ConvNets,respectively.for training exponentially(and output neuron numbers correspondingly)from136to4349whilefixing the neuron numbers in all previous layers(the DeepID is kept to be 160dimensional).We observe the classification ability of ConvNets(measured by the top-1validation set error rates) and the effectiveness of the learned hidden representations for face verification(measured by the test set verification accuracy)with the increasing identity classes.The input is a single patch covering the whole face in this experiment.As shown in Figure6,both Joint Bayesian and neural network improve linearly in verification accuracy when the identity classes double.The improvement is significant.When identity classes increase32times from136to4349,the accuracy increases by10.13%and8.42%for Joint Bayesian and neural networks,respectively,or2.03%and1.68%on average,respectively,whenever the identity classes double. At the same time,the validation set error rates drop,even when the predicted classes are tens of times more than the last hidden layer neurons,as shown in Figure7.This phenomenon indicates that ConvNets can learn from classi-fying each identity and form shared hidden representations that can classify all the identities well.More identity classes help to learn better hidden representations that can distinguish more people(discriminative)without increasing the feature length(compact).The linear increasing of test accuracy with respect to the exponentially increasing training data indicates that our features would be further improved if even more identities are available.Examples of the160-dimensional DeepID learned from the4349training identities and extracted from LFW test pairs are shown in Figure8.Wefind that faces of the same identity tend to have more commonly activated neurons(positive features being in the same position)than those of different identities. So the learned features extract identity information.We also test the4349-dimensional classifier outputs as features for face verification.Joint Bayesian only achieves approximately66%accuracy on these features,while the neural network fails,where it accounts all the face pairs asFigure6.Face verification accuracy of Joint Bayesian(red line) and neural network(blue line)learned from the DeepID,where the ConvNets are trained with136,272,544,1087,2175,and4349 classes,respectively.Figure7.Top-1validation set error rates of ConvNets learned to classify136,272,544,1087,2175,and4349classes,respectively. positive or negative pairs.With so many classes and few samples for each class,the classifier outputs are diverse and unreliable,therefore cannot be used as features.4.3.Over-complete representationWe evaluate how much combining features extracted from various face patches would contribute to the perfor-mance.We train the face verification model with features from k patches(k=1,5,15,30,60).It is impossible to numerate all the possible combinations of patches,so we select the most representative ones.We report the best-performing single patch(k=1),the global color patches in a single scale(k=5),all the global color patches (k=15),all the color patches(k=30),and all the patches (k=60).As shown in Figure9,adding more features from various regions,scales,and color channels consistently improves the bing60patches increases the accuracy by4.53%and5.27%over best single patch for Joint Bayesian and neural networks,respectively.We achieve96.05%and94.32%accuracy using Joint Bayesian and neural networks,respectively.The curves show that the performance may be further improved if more features areextracted.Figure8.Examples of the learned160-dimensional DeepID.The left column shows three test pairs in LFW.Thefirst two pairs are of the same identity,the third one is of different identities.The corresponding features extracted from each patch are shown in the right.The features are in one dimension.We rearrange them as 5×32for the convenience of illustration.The feature values are non-negative since they are taken from the ReLUs.Approximately 40%features have positive values.The brighter squares indicate higher values.Figure9.Test accuracy of Joint Bayesian(red line)and neural networks(blue line)using features extracted from1,5,15,30, and60patches.Performance consistently improves with more features.Joint Bayesian is approximately1.8%better on average than neural networks.4.4.Method comparisonTo show how our algorithm would benefit from more training data,we enlarge the CelebFaces dataset to Celeb-Faces+,which contains202,599face images of10,177 celebrities.People in CelebFaces+and LFW are mutually exclusive.We randomly choose8700people from Celeb-Faces+to learn the DeepID,and use the remaining1477 people to learn Joint Bayesian for face verification.Since extracting DeepID from many different face patches also helps,we increase the patch number to100by usingfive different scales of patches instead of three.This results ina32,000-dimensional DeepID feature vector,which is then reduced to150dimensions by PCA.Joint Bayesian learned on this150-dimensional feature vector achieves97.20% test accuracy on LFW.Due to the difference in data distributions,models well fitted to CelebFaces+may not have equal generalization ability on LFW.To solve this problem,Cao et al.[6] proposed a practical transfer learning algorithm to adapt the Joint Bayesian model from the source domain to the target domain.We implemented their algorithm by using the1477 people from CelebFaces+as the source domain data and nine out of ten folders from LFW as the target domain data for transfer learning Joint Bayesian,and conduct ten-fold cross validation on LFW.The transfer learning Joint Bayesian based on our DeepID features achieves97.45% test accuracy on LFW,which is on par with the human-level performance of97.53%.We compare with the state-of-the-art face verification methods on LFW.In the comparison,we report three results.Thefirst two are trained on CelebFaces and CelebFaces+,respectively,without transfer learning,and tested on LFW.The third one is trained on CelebFaces+ with transfer learning on LFW.Table1comprehensively compares the accuracies,the number of facial points used for alignment,the number of outside training images(if applicable),and thefinal feature dimensions for each face (if applicable).Low feature dimensions indicate efficient face recognition systems.Figure10compares the ROC curves.Our DeepID learning method achieves the best performance on LFW.Thefirst four best methods compared used dense facial landmarks,while our faces are weakly aligned with onlyfive points.The deep learning work (DeepFace)[32]independently developed by Facebook at the same time of this paper achieved the second best performance of97.25%accuracy on LFW.It utilized3D alignment and pose transform as preprocessing,and more than seven million outside training images plus training images from LFW.5.Conclusion and DiscussionThis paper proposed to learn effective high-level features revealing identities for face verification.The features are built on top of the feature extraction hierarchy of deep ConvNets and are summarized from multi-scale mid-level features.By representing a large amount of different identities with a small number of hidden variables,highly compact and discriminative features are acquired.The features extracted from different face regions are comple-mentary and further boost the performance.It achieved 97.45%face verification accuracy on LFW,while only requiring weakly aligned faces.Even more compact and discriminative DeepID can be learned if more identities are available to increasethe Figure10.ROC comparison with the state-of-the-art face verifica-tion methods on LFW.TL in our method means transfer learning Joint Bayesian.dimensionality of prediction at the training stage.We look forward to larger training sets to further boost our performance.A recent work[27]reported98.52%accuracy on LFW with Gaussian Processes and multi-source training sets,achieving even higher than human performance.This could be due to the fact that the nonparametric Bayesian kernel method can adapt model complexity to data distri-bution.Gaussian processes can also be modeled with deep learning[12].This could be another interesting direction to be explored in the future.AcknowledgementWe thank Xiaoxiao Li and Cheng Li for their help and discussion.This work is partially supported by”CUHK Computer Vision Cooperation”grant from Huawei,the General Research Fund sponsored by the Research Grants Council of Hong Kong(Project No.CUHK416510and 416312),National Natural Science Foundation of China (91320101),and Guangdong Innovative Research Team Program(No.201001D010*******).References[1]T.Ahonen and M.Pietikainen.Soft histograms for local binarypatterns.2007.2[2]T.Berg and P.Belhumeur.Tom-vs-Pete classifiers and identity-preserving alignment for face verification.In Proc.BMVC,2012.1,2,8[3]T.Berg and P.Belhumeur.POOF:Part-based one-vs-one featuresforfine-grained categorization,face verification,and attribute esti-mation.In Proc.CVPR,2013.1,2[4]J.Bromley,I.Guyon,Y.Lecun,E.S¨a ckinger,and R.Shah.Signatureverification using a“Siamese”time delay neural network.In Proc.NIPS,1994.2[5]X.Cai,C.Wang,B.Xiao,X.Chen,and J.Zhou.Deep nonlinear met-ric learning with independent subspace analysis for face verification.In ACM Multimedia,2012.1,2。
人脸识别技术与生物特征识别培训ppt
算法优化
硬件升级
持续优化人脸识别算法，提高识别速 度和准确率。
升级硬件设备，提高人脸识别系统的 处理能力和响应速度。
数据训练
使用大规模、多样化的数据集进行训 练，提高人脸识别模型的泛化能力。
05
培训内容与实践
人脸识别技术基础培训
人脸识别技术原理
详细介绍人脸识别技术的原理、算法和实现过程，包括特征提取 、比对和识别等关键技术。
分类
按照所利用的特征类型，生物特征识 别技术可分为基于生理特征和基于行 为特征的识别技术。常见生物特征识来自技术介绍0102
03
04
指纹识别
利用指纹的唯一性和稳定性进 行身份鉴别。
虹膜识别
通过分析眼睛的虹膜纹理进行 身份鉴别。
视网膜识别
通过分析眼睛的视网膜结构进 行身份鉴别。
面部识别
通过分析人的面部特征进行身 份鉴别。
人脸识别技术与生物 特征识别培训
汇报人：可编辑
xx年xx月xx日
• 人脸识别技术概述 • 人脸识别的关键技术 • 生物特征识别技术介绍 • 人脸识别技术的挑战与解决方案 • 培训内容与实践 • 总结与展望
目录
01
人脸识别技术概述
人脸识别技术的定义与原理
总结词 人脸识别技术是一种基于生物特征识 别技术，通过计算机图像处理和人工 智能算法，自动识别和验证个人身份 的技术。
个人隐私。
THANKS
感谢观看
比对
将提取出的特征与数据库中的特征进行比对，以实现人脸的识别或验证。
深度学习在人脸识别中的应用
深度学习模型
深度学习模型如卷积神经网络（CNN）被广泛应用于人脸识 别，能够自动提取高层次的特征表示。
深度学习--人脸识别
卷积波尔兹曼机（Convolutional RBM）
卷积过程：用一个可训练的滤波器fx去卷积一个输入的 图像（第一阶段是输入的图像，后面的阶段就是Feature Map了），然后加一个偏置bx，得到卷积层Cx。 子采样过程：每邻域n个像素通过池化（pooling）步骤 变为一个像素，然后通过标量Wx+1加权，再增加偏置bx+1， 然后通过一个sigmoid激活函数，产生一个大概缩小n倍的 特征映射图Sx+1。
深度信念网络（ Deep Belief Networks ）
深度信念网络是一个包含多层隐层（隐层数大于2） 的概率模型，每一层从前一层的隐含单元捕获高度相关 的关联。
DBNs是一个概率生成模型，与传统的判别模型 的神经网络相对，生成模型是建立一个观察数据和 标签之间的联合分布，对P(Observation|Label)和 P(Label|Observation)都做了评估。 典型的DNBs,可视数据v和隐含向量h的关系可 以用概率表示成如下所示形式：
深度模型(Deep models)
●受限波尔兹曼机RBM ●深度信念网络DBN ●卷积受限波尔兹曼机CRBM ●混合神经网络-受限波尔兹曼机CNN-RBM
…….
“深度模型”是手段， “特征学习”是目的！
深度学习
1.什么是深度学习？ 2.深度学习的基本思想
3.深度学习的常用方法
1）自动编码机(AutoEncoder) 2）稀疏编码（Sparse Coding） 3）受限波尔兹曼机（Restrict Boltzmann Machine , RBM）
(a)LBP：Local Binary Pattern(局部二值模式) (b)LE:an unsupervised feature learning method,PCA (c)CRBM:卷积受限波尔兹曼机 (d)FIP:Face Identity-Preserving
深度学习文献阅读笔记（1）
深度学习⽂献阅读笔记（1） 转眼间已经研⼆了。
突然想把曾经看过的⽂献总结总结与⼤家分享，留作纪念，⽅便以后參考。
1、深度追踪：通过卷积⽹络进⾏差异特征学习的视觉追踪(DeepTrack:Learning Discriminative Feature Representations by Convolutional Neural Networks for visual Tracking）（英⽂，会议论⽂，2014年。
EI检索） 将卷积神经⽹络⽤于⽬标跟踪的⼀篇⽂章，可将CNN不只能够⽤做模式识别。
做⽬标跟踪也是能够。
毕竟本质上是⼀种特征提取的⼿段。
2、基于深度学习的车标识别⽅法研究（中⽂。
期刊，2015年。
知⽹） 将传统CNN⽤于车标识别，先进⾏车标定位提取。
在送⼊CNN中进⾏训练，最后採⽤⽀持向量机进⾏分类，属于⽼⽅法新问题。
实验硬件配置：主频2.80GHZCPU，2G内存，未⽤到GPU加速。
3、基于深度学习⽹络的射线图像缺陷识别⽅法（中⽂，期刊，2014年，知⽹） 将CNN直接⽤于射线图像缺陷检測，⽼⽅法新问题。
对CNN结构描写叙述得⾮常清楚，适合CNN⼊门。
4、深度学习及其在⽬标和⾏为识别中的新进展（中⽂，期刊，2014年，知⽹） 主要综述了深度学习中⾃编码器和限制玻尔兹曼机的结构以及应⽤进展。
综述⽐較全⾯，也够权威。
对两者的原理和改进进展都描写叙述得⾮常清楚，指出“深度学习得到的是⼀个多层深度结构，信号在这个多层结构中进⾏传播，最后得到信号的表达。
学习到多层的⾮线性的函数关系，更好的对视觉信息进⾏建模”，值得參考。
5、基于超像素卷积神经⽹络的显著性⽬标检測（Super CNN:A Superpixel wise Convolutional Neural Network for salient object detection）（英⽂，期刊，2015年，IEEE检索） CNN在⽬标检測领域的应⽤，先对图像进⾏超像素切割，得到三个序列（超像素序列，⼀个空间核矩阵。
基于深度学习的人脸鉴伪与识别技术研究与原型实现
2.2.1 VGG.................................................................................................... 13
和相关技术 ..................................................................................... 8
第二章 基础理
2.1 卷积神经网络 ................................................................................................ 8
a
. The
ec f c
h he
de : f
he
e
ac
e
ae
a
e a
he
f aa ee
e
e
e
cha ac e
ade
he ba
a e
f he
f he
f he e
ed a
he
g a
c
f he
, he
f he VGG
a
e,
de . Sec d , he e
g a e ae
. Th d, c
f
c
d ced
b
目录
................................................................................................................. 1
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
2Department of Electronic Engineering, The Chinese University of Hong Kong
3Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
sy011@.hk
xgwang@.hk
xtang@.hk
Abstract
The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset [11], 99.15% face verification accuracy is achieved. Compared with the best deep learning result [21] on LFW, the error rate has been significantly reduced by 67%.
1
signal, its DeepID2 features are extracted in the top hidden layer of the learned hierarchical nonlinear feature representation, and then mapped to one of a large number of identities through another function g(DeepID2). In the testing stage, the learned DeepID2 features can be generalized to other tasks (such as face verification) and new identities unseen in the training data. The identification supervisory signal tend to pull apart DeepID2 of different identities since they have to be classified into different classes. Therefore, the learned features would have rich identity-related or interpersonal variations. However, the identification signal has a relatively weak constraint on DeepID2 extracted from the same identity, since dissimilar DeepID2 could be mapped to the same identity through function g(·). This leads to problems when DeepID2 features are generalized to new tasks and new identities in test where g is not applicable anymore. We solve this by using an additional face verification signal, which requires that every two DeepID2 vectors extracted from the same identity are close to each other while those extracted from different identities are kept away. The strong per-element constraint on DeepID2 can effectively reduce the intra-personal variations. On the other hand, using the verification signal alone (i.e. only distinguishing a pair of DeepID2 at a time) is not as effective in extracting identity-related features as using the identification signal (i.e. distinguishing thousands of identities at a time). Therefore, the two supervisory signals emphasize different aspects in feature learning and should be employed together.
Deep Learning Face Representation by Joint Identification-Verification
arXiv:1406.4773v1 [cs.CV] 18 Jun 2014
Yi Sun1
Xiaogang Wang2
Xiaoou Tang1,3
1Department of Information Engineering, The Chinese University of Hong Kongame identity could look much different when presented in different poses, illuminations, expressions, ages, and occlusions. Such variations within the same identity could overwhelm the variations due to identity differences and make face recognition challenging, especially in unconstrained conditions. Therefore, reducing the intra-personal variations while enlarging the inter-personal differences is an eternal topic in face recognition. It can be traced back to early subspace face recognition methods such as LDA [1], Bayesian face [17], and unified subspace [23, 24]. For example, LDA approximates inter- and intra-personal face variations by using two linear subspaces and finds the projection directions to maximize the ratio between them. More recent studies have also targeted the same goal, either explicitly or implicitly. For example, metric learning [6, 9, 15] maps faces to some feature representation such that faces of the same identity are close to each other while those of different identities stay apart. However, these models are much limited by their linear nature or shallow structures, while inter- and intra-personal variations are complex, highly nonlinear, and observed in high-dimensional image space.