[2014 Data Science and Advanced Analytics (DSAA)] SAR target recognition based on deep learning

SAR Target Recognition Based on Deep LearningSizhe Chen, Haipeng WangKey Laboratory for Information Sciences of Electromagnetic Waves (MoE)Fudan University, Shanghai, ChinaEmail: hpwang@Abstract— Deep learning algorithms such as convolutional neural networks (CNN) have been successfully applied in computer vision. This paper attempts to adapt the optical camera-oriented CNN to its microwave counterpart, i.e. synthetic aperture radar (SAR). As a preliminary study, a single layer of convolutional neural network is used to automatically learn features from SAR images. Instead of using the classical backpropagation algorithm, the convolution kernel is trained on randomly sampled image patches using unsupervised sparse auto-encoder. After convolution and pooling, an input SAR image is then transformed into a series of feature maps. These feature maps are then used to train a final softmax classifier. Initial experiments on MSTAR public data set show that an accuracy of 90.1% can be achieved on three types of targets classification task, and an accuracy of 84.7% is achievable on ten types of targets classification task.Keywords—Convolutional Neural Network; sparse auto-encoder; Synthetic Aperture Radar; Automatic Target RecognitionI.I NTRODUCTIONSynthetic aperture radar (SAR) is a kind of airborne or spaceborne two-dimensional high resolution imaging radar. Unlike optical remote sensing, which cannot work in poor weather conditions and night, SAR can operate regardless of weather conditions, day and night, so it is of high value in military and civil applications. But the interpretation of SAR images needs specialists, since unlike natural images, SAR images reflect the backscattering intensity of electromagnetic wave. Moreover, searching targets of interest in the massive SAR images by human is time-consuming and extremely difficult, which justifies the need for efficient SAR automatic target recognition (ATR) algorithms.MIT Lincoln Laboratory has pioneered research on this topic, as evidenced in their two SAR ATR system: Template-based SAIP (semi-automated image intelligence processing system) program [1, 2] and Model-based MSTAR (moving and stationary target acquisition and recognition) program [3]. The main component of SAIP system is standard pattern recognition architecture. The researchers collected several hundreds of thousands airborne SAR images, designed five different features extraction algorithms, and finally use these feature vectors as inputs to train a classifier. The performance is nearly perfect when testing conditions are similar to the training conditions, however, when testing conditions changed the recognition rate will decrease drastically [2]. The reason is that SAR images are very sensitive to all kinds of variations, such as articulation, obscuration, and camouflage on military targets, and changes of background environment. Uncertainties in the real world are so large that it is unrealistic to collect sample images covering all possible variations. To solve these problems, a model-based module was later added on the back-end of MSTAR system. The front-end of MSTAR system is still a typical pattern recognition system, while targets unable to be recognized by front-end will be sent into back-end, which is an iterative searching classifier. In each iteration, features extracted from test images are compared with features predicted from target CAD model using electromagnetic signature prediction code. The needs for another iteration are determined according to the degree of matching. In both of these SAR ATR systems, designing and selecting useful features plays an important role in recognition accuracy.Just like machine learning used in speech recognition and computer vision, the most important thing is feature engineering: designing better features to represent targets and then use these features as input to train a classifier. Previously, researchers in computer vision community developed many powerful features for object recognition, such as SIFT (Scale-invariant feature transform) and HOG (Histogram of Oriented Gradients). Over the last few years, considerable efforts have been made to design multi-stage architectures to automatically learn representations or features from data instead of hand-crafted features in speech recognition and computer vision domain. In many speech recognition or natural images object recognition benchmark dataset, these feature learning algorithms have achieved superior performance.Nowadays, deep architecture with convolution and pooling are found to be highly effective and commonly used incomputer vision and object recognition [4-14]. The most impressive result was achieved in 2012 ImageNet contest, where 1.2 million images in the training set with 1000 different object classes. On the test data set including 150,000 images, the deep Convolutional Neural Network (CNN) approach described in [4] achieved the error rates considerably lower than the previous state-of-the-art results. Furthermore, CNN has achieved superior classification accuracy on different tasks, such as handwritten digits or Latin and Chinese character recognition [5, 6], traffic sign recognition [7], face detection and recognition [8]. Deep networks are shown to be powerful for computer vision and image recognition tasks because they extract appropriate features while jointly performing discrimination [9].A few researchers in SAR recognition area have tried to use multi-layer network to learn features for classification [15, 16]. However, in their work, complex image pre-processing and hand-crafted feature extraction on the first layer are needed. Fully unsupervised feature learning algorithm begins from extracted features instead of raw input.This paper attempts to adapt the optical camera-oriented CNN to its microwave counterpart, i.e. synthetic aperture radar (SAR). As a preliminary study, a single layer of convolutional neural network is used to automatically learn features from SAR images. It will be shown that although SAR images are extremely noisy and identifying targets is not an easy task even for humans, features learned from raw input pixels can still be useful for classification.II.A LGORITHMSMost successful and commonly used feature learning models typically consist of multi-stage trainable feature extractor, greedily layer-wise training one layer at a time, treating the features learned from previous layer as the input to the next layer, using unsupervised feature learning algorithm. For each of these layers, a number of parameters are chosen from cross-validation or empirically: the size of convolution receptive field, the number of feature maps in each layer, dimension of pooling region. Results in [12] demonstrate that with large number of hidden nodes and dense feature extraction, using simple unsupervised learning algorithm and only a single layer of features, state-of-art results on both CIFAR-10 and NORB datasets can be achieved. Because our goal is firstly to find whether features learned from raw input SAR images would be useful for target recognition or not, for simplicity, we choose a single layer convolutional neural network (CNN).Our approach consists of the following steps to learn a feature representations: (1) Extract random patches from unlabeled images, the dimension of these patches is the same as convolution kernel. (2) Subtract mean value and then apply ZCA whitening pre-processing to the sampled patches. (3) Instead of training purely supervised from labeled image data using classical backpropagation and stochastic gradient decent, our convolutional layer is trained using sparse auto-encoder on randomly sampled image patches.Given the learned convolution kernel and a set of labeled training images, we can perform feature extraction and classification: (1) Convolve the input images with learned convolution kernel. (2) Pool features over local neighborhoods using max operation, reducing the dimension of feature values and achieving invariance to small distortion. (3) Train a softmax regression classifier to predict the labels given the pooled convolved feature vectors, the weights on this final layer is trained supervised using backpropagation. Fig. 1 shows the flowchart of the algorithm.Fig. 1 Flowchart of the algorithmA.Generate image patches for unsupervised learningOur approach begins by resizing all of the SAR images to 128 128pixels. And then extract a large number of image patches from unlabeled training input images. The size of each image patches is w w, where w referred to the size of convolution receptive field. Next convert each image patch into an N –dimensional vector, where N w w. Suppose the total number of sampled image patches is m, and concatenate all of them into N m matrix: X , , , , where .SAR imagedatasetB.Pre-processingIn this step, each patch is normalized by subtracting the mean value of each image patch and then apply Zero Component Analysis (ZCA) Whitening [17]. This pre-processing approach is commonly used in deep learning. The main idea here is that since adjacent pixel values are highly correlated, the raw input is highly redundant. The goal of ZCA whitening is to make the input less correlated with each other, and to make all input elements have the same variance. We can compute the ZCA whitened data as: , whereT UP / U T and U and P are the eigenvectors and eigenvalues of the covariance matrix ofX: Σ∑ T.C.Sparse auto-encoderThe sparse auto-encoder is a kind of one hidden layer neural network that sets its output target value to be the same as the input data. In other word, it tries to learn an identity function , . By minimizing the reconstruction error with an extra sparse penalty term that encourage the hidden units to keep low average activation, convolution kernel W and biases b will be learned using backpropagation from unlabeled data. Imposing a sparsity constraint on the hidden layer, then structure of input data will be discovered by the auto-encoder.For each training sample , the activation value on the hidden layer is given by:(1) where , denotes to connection weightsand bias respectively, and 1/ 1 is the logistic sigmoid function. However, sigmoid function cannot be used on the output layer, since the output of logistic sigmoid function is in the range of [0, 1], while input data are not rescaled in the range [0, 1]. So a linear activation function are chosen for the output layer:(2) where , . With a training set of m examples, the overall cost function is defined as:W,b 112,(3)The first term is an average sum of squared error term, tryingto minimize reconstruction error. The second term isregularization term, which is commonly used to preventoverfitting.Sparsity constraint means that for each inputs , most ofthe corresponding hidden units are close to 0, only afew of them will be activated. Given a training set with msamples, the average activation value of hidden unit j is:1(4)The sparsity penalty term is based on the Kullback-Leibler(KL) divergence between a Bernoulli random variable withmean and a Bernoulli random variable with mean . KL-divergence is used to measure the extent of discrepancybetween two different distributions. This penalty term can bewritten as:KL log1 log11 (5)where is sparsity parameter, typically a small value aroundzero (say 0.02 .KL 0, if , and increasesmonotonically as deviates from . This penalty term seeksto enforce the average activation of hidden units to be equalto .Suppose the number of hidden units is K, the overall costfunction is:, W,b KL(6)D.Convolutional feature extraction and poolingA single stage of convolution network consists of three layers:a convolution filter layer, a non-linearity layer, a featurepooling layer. At the output, each input image is represented bysets of arrays called feature maps. Each feature maps representsa particular feature extracted at all locations on the wholeimages. Fig. 2 shows the Architecture of the 1-layer CNN.Convolution filter layer: given an input image ofpixels, the output y composed of K feature maps of size nw 1 n w 1 . K corresponds to the number of hiddenunits in the sparse auto-encoder trained on small patchespreviously. All trainable convolution kernel have sizeand convolve them with large input images. The modelcomputes ∑where is the 2D discreteconvolution operator, is a trainable bias, andmeans flipping the matrix horizontally and vertically. Both convolution kernel and biases are unsupervised trained using sparse auto-encoder.Fig. 2 Architecture of the 1-layer CNNNon-linearity Layer: Simply apply logistic sigmoidfunction 1/ 1 point-wise. Then a methodcalled Local Contrast Normalization was employed to performlocal subtractive and divisive normalizations, which enforceslocal competition between adjacent features in a feature map, and between features at the same spatial location in different feature maps [10]. The subtractive normalization operationremoves the weighted average of neighboring neurons from thecurrent neuron: ∑ , , , whereis a Gaussian weighting window (typically of size 9 9)normalized so that ∑ 1. The divisive normalizationcomputes /max , where∑, ,/ .Feature pooling layer: Max pooling operation is chosen,which computes the maximum value over a neighborhood region in each feature map: , , , . This process not only reduce the dimension of output feature mapsbut also introduce small translation invariance, which helps to build robust features for classification. E. Softmax regression classifierIn this step, pooled features and their corresponding labels are used to train the final softmax classifier. Softmax regression model focuses on multi-class classification. Given a test input , output of softmax classifier will be a k dimensional vector, each element of which refers to the estimated probability of the class label equal to each of the k different values conditioned on this input feature: | ; , for each value of 1, , . The output k dimensional hypothesis takes the form:1 ;2 ; ;1∑(7)Here, , , ,are the parameters to be learned supervised. The cost function would be:1 1log ∑(8)where 1 · is the indicator function, so that 1{a true statement}= 1, and 1{a false statement} = 0. For example, 1{2 + 2 =4} =1, whereas 1{1 + 1 = 3} = 0. The second term is weightdecay term which prevents overfitting. The minimum ofneeds to be solved by gradient decent. The gradient of costfunction is:11; (9) Then plug it into gradient decent algorithm, for eachiteration the parameter will perform the update:.III. E XPERIMENTAL RESULTSA. DatasetThe experiment data set was collected by the Sandia National Laboratory (SNL) SAR sensor platform. The collection was jointly sponsored by Defense AdvancedInput imageContrastNormalizationPoolingResearch Projects Agency (DARPA) and Air Force Research Laboratory as part of the Moving and Stationary Target Acquisition and Recognition (MSTAR) program. They collected hundreds of thousands of SAR images containing ground military targets, and only a small subset of which are public available on the website [18]. The public released datasets include 10 different types of ground military targets (BMP2, BRDM2, BTR60, BTR70, D7, T62, T72, ZIL131, ZSU234, 2S1), and was collected by using an X-band SAR sensor in one foot resolution spotlight mode, full aspect coverage (in the range of 0 to 360 degree). The data collected at 17 degree depression angles are used for training, and the data collected at 15 degree depression angles are used for testing.Fig. 3 Ten types of military targets in SAR images (left) andtheir corresponding optical images (right).Fig. 4 Visualization of learned convolutional kernels from SAR images (top) vs. that learned from optical images asgiven in [17] (bottom).The ten types of targets in SAR images are shown in Fig. 3 with the corresponding optic images given for comparison. As we can see, the same target appears to be very different than optical image. The most prominent difference lies to the ‘speckle’ appearance of SAR image. Fig. 4 compares the learned convolutional kernels of SAR images vs. conventional optical images. Interestingly, the SAR features looks very different from optical features which are basically edge detector templates. The SAR features looks more like ‘sinc’ shaped responses probabaly due to SAR speckle.B. Classification ResultsThe algorithms are tested on two classification experiments. The first one is classifying three types of military targets. The details of training and testing data set are shown in TABLE Ι. In the experiment, deformation targets are included in the test set. For example, T72_132, T72_812, and T72_s7 are all belong to T72 tank, but they have different militaryconfigurations such as whether contain machine guns, antenna,(2) BMP2(1) BRDM2(4) BTR60(3) BTR70(6) D7(5) 2S1(8) T62(7) T72(10) ZIL131(9) ZSU234protection armor or not. The classification accuracy of three types of target is 90.1% (error rate 9.9%).TABLE I. SAMPLE NUMBER OF TRAINING AND T ESTING SET FOR THREETYPESTraining set Sample number Testing set Sample numberBMP2_c21 BTR70_c71 T72_132 233233232BMP2_c21BMP2_9563BMP2_9566BTR70_c71T72_132T72_812T72_s7196195196196196195191The second one is classifying ten types of military targets. In this experiment, all of the targets are in standard configuration, deformation targets are not included. The details of training and testing data set are shown in TABLE Ⅱ. In this experiment, after adjust hyper parameters by cross-validation, we achieve a classification accuracy of 84.7% (error rate 15.3%). The detailed ambiguity matrix is shown in TABLE III.TABLE II. SAMPLE NUMBER OF TRAINING AND T ESTING SET FOR TENTYPESTraining set Sample number Testing set Sample numberBMP2BRDM2BTR60BTR70D72S1T62T72ZIL131ZSU234233298256233299299299232299299BMP2BRDM2BTR60BTR70D72S1T62T72ZIL131ZSU234195274195196274274273196274274For comparison, similar studies using handcrafted features [19] to classify the same set of MSTAR datasets have achieved accuracy of 96.79% for three types and 90.5% for ten types, respectively. It can be seen that the fully automatic single-layer CNN is able to achieve reasonable performance (i.e. ~5% lower than manually designed classifier) which makes us confident that a multi-layer ‘deep’ CNN might achieve comparable or better performance than hand-crafted algorithms.TABLE III. CLASSIFICATION A MBIGUITY M ATRIXTest setClassification ResultsBMP2 BRDM2 BTR60 BTR70 D7 2S1 T62 T72 ZIL131 ZSU234Classificationaccuracy (%)BMP2 157 9 2 9 0 4 0 4 6 4 80.5 BRDM2 9 220 6 18 0 3 1 2 15 0 80.2 BTR60 0 11 168 4 4 4 1 2 1 0 86.1 BTR70 3 4 3 181 0 4 0 0 1 0 92.3 D7 0 0 0 0 252 0 8 2 5 7 91.9 2S1 14 9 5 5 0 190 7 22 21 1 69.3 T62 2 1 5 0 4 7 2423 7 2 88.6T72 3 3 1 1 0 8 2168 9 1 85.7 ZIL131 5 6 5 7 1 12 3 9 226 0 82.4 ZSU234 1 1 3 0 4 1 2 7 6 249 90.8average classification rate: 84.7%IV.C ONCLUSION AND FUTURE WORKAs a preliminary study, a single stage of convolutional network is adapted to automatically learn features useful for SAR target recognition. Using learned conolutional features, an accuracy of 90.1% and 84.7% are achieved on three and ten types of targets MSTAR dataset classifcation, respectively. The next step is to learn hierarchical features by using multi-stage convolutional network, considering the scarcity of SAR image samples, data augmentation technique and overfitting reduction methods must be utilized.R EFERENCES[1]Novak L M, Owirka G J, Brower W S, et al. “The automatic target-recognition system in SAIP,” Lincoln Laboratory Journal, 1997, 10(2). [2]Novak L M. “State-of-the-art of SAR automatic target recognition,”Radar Conference, 2000. The Record of the IEEE 2000 International.IEEE, 2000: 836-843.[3]Wissinger J, Ristroph R, Diemunsch J R, et al. “MSTAR's extensiblesearch engine and model-based inferencing toolkit,” AeroSense'99.International Society for Optics and Photonics, 1999: 554-570.[4]Krizhevsky A, Sutskever I, Hinton G E. “ImageNet Classification withDeep Convolutional Neural Networks,” NIPS. 2012, 1(2): 4.[5]Ciresan D C, Meier U, Gambardella L M, et al. “Deep, big, simpleneural nets for handwritten digit recognition,” Neural computation, 2010, 22(12): 3207-3220.[6]Ciresan D C, Meier U, Schmidhuber J. “Transfer learning for Latin andChinese characters with deep neural networks,” Neural Networks (IJCNN), The 2012 International Joint Conference on. IEEE, 2012: 1-6. [7]Cireşan D, Meier U, Masci J, et al. “Multi-column deep neural networkfor traffic sign classification,” Neural Networks, 2012, 32: 333-338. [8]Taigman Y, Yang M, Ranzato M, et al. “Deep-Face: Closing the Gap toHuman-Level Performance in Face Verification,” IEEE CVPR. 2014. [9]Yann LeCun. “Learning invariant feature hierarchies,” Computervision–ECCV 2012. Workshops and demonstrations. Springer Berlin Heidelberg, 2012: 496-505.[10]Jarrett K, Kavukcuoglu K, Ranzato M, et al. “What is the best multi-stage architecture for object recognition?” Computer Vision, 2009 IEEE 12th International Conference on. IEEE, 2009: 2146-2153.[11]Kavukcuoglu K, Sermanet P, Boureau Y L, et al. “LearningConvolutional Feature Hierarchies for Visual Recognition,” NIPS. 2010, 1(2): 5. [12]Coates A, Ng A Y, Lee H. “An analysis of single-layer networks inunsupervised feature learning,” International Conference on Artificial Intelligence and Statistics. 2011: 215-223.[13]Zeiler M D, Krishnan D, Taylor G W, et al. “Deconvolutionalnetworks,” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010: 2528-2535.[14]Zeiler M D, Taylor G W, Fergus R. “Adaptive deconvolutional networksfor mid and high level feature learning,” Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 2011: 2018-2025.[15]Diemunsch J R, Wissinger J. “Moving and stationary target acquisitionand recognition (MSTAR) model-based automatic target recognition: search technology for a robust ATR.” Aerospace/Defense Sensing and Controls. International Society for Optics and Photonics, 1998: 481-492.[16]Sun Z, Xue L, Xu Y, et al. “Marginal Fisher Analysis Feature ExtractionAlgorithm Based on Multilayer Auto-encoder,” Journal of Information and Computational Science, 2012, 9(18): 5897-5906.[17]Deep learning tutorial: /wiki/index.php/UFLDL_Tutorial[18]MSTAR public targets dataset: https:///index.php?collection=mstar[19]ZHANG Xinzheng, HUANG Peikang. “SAR ATR based on Bayesiancompressive sensing,” Systems Engineering and Electronics, 2013, 35(1): 40-44. [in Chinese]。