Evolving Hierarchical RBF Neural Networks for Breast Cancer Detection

Evolving Hierarchical RBF Neural Networks forBreast Cancer DetectionYuehui Chen,Yan Wang,and Bo YangSchool of Information Science and EngineeringJinan University,Jinan250022,P.R.Chinayhchen@Abstract.Hierarchical RBF networks consist of multiple RBF networksassembled in different level or cascade architecture.In this paper,anevolved hierarchical RBF network was employed to detect the breastcancel.For evolving a hierarchical RBF network model,Extended Com-pact Genetic Programming(ECGP),a tree-structure based evolutionaryalgorithm and the Differential Evolution(DE)are used tofind an op-timal detection model.The performance of proposed method was thencompared with Flexible Neural Tree(FNT),Neural Network(NN),andRBF Neural Network(RBF-NN)by using the same breast cancer dataset.Simulation results show that the obtained hierarchical RBF networkmodel has a fewer number of variables with reduced number of inputfeatures and with the high detection accuracy.1IntroductionBreast cancer is the most common cancer in women in many countries.Most breast cancers are detected as a lump/mass on the breast,or through self-examination or mammography[1].Screening mammography is the best tool available for detecting cancerous lesions before clinical symptoms appear[7]. Surgery through a biopsy or lumpectomy have been also been the most com-mon methods of removal.Fine needle aspiration(FNA)of breast masses is a cost-effective,non-traumatic,and mostly invasive diagnostic test that obtains information needed to evaluate malignancy.Recently,a new less invasive tech-nique,which uses super-cooled nitrogen to freeze and shrink a non-cancerous tumor and destroy the blood vessels feeding the growth of the tumour,has been developed[2]in the USA.Various artificial intelligence techniques have been used to improve the di-agnoses procedures and to aid the physician’s efforts[3][4][5][6].In our previ-ous studies,the performance of Flexible Neural Tree(FNT)[11],Neural Net-work(NN),Wavelet Neural Network(WNN)and an ensemble method to detect breast-cancer have been evaluated[12].Hierarchical RBF networks(HRBF)consist of multiple RBF networks assem-bled in different level or cascade architecture in which a problem was divided and solved in more than one step.Mat Isa et ed Hierarchical Radial Basis Func-tion(HiRBF)to increase RBF performance in diagnosing cervical cancer[14].I.King et al.(Eds.):ICONIP2006,Part III,LNCS4234,pp.137–144,2006.c Springer-Verlag Berlin Heidelberg2006138Y.Chen,Y.Wang,and B.YangThe HiRBF cascaded together two RBF networks,where both networks have different structure but using the same learning algorithms.Thefirst network classifies all data and performs afiltering process to ensure that only certain attributes to be fed to the second network.The study shows that the HiRBF performs better compared to single RBF.Hierarchical RBF network has been proved effective in the reconstruction of smooth surfaces from sparse noisy data points[15].In order to improve the model generalization performance,a selec-tive combination of multiple neural networks by using Bayesian method was proposed in[16].In this paper,an automatic method for constructing HRBF networks is pro-posed.Based on a pre-defined instruction/operator set,the HRBF network can be created and evolved.The HRBF network allows input variables selection.In our previous studies,in order to optimize the Flexible Neural Tree(FNT)and the hierarchical TS fuzzy model(H-TS-FS),the hierarchical structure of FNT and H-TS-FS was evolved using Probabilistic Incremental Program Evolution al-gorithm(PIPE)[10][11]and Ant Programming[13]with specific instructions.In this research,the hierarchical structure is evolved using the Extended Compact Genetic Programming(ECGP),a tree-structure based evolutionary algorithm. Thefine tuning of the parameters encoded in the structure is accomplished us-ing the DE algorithm.The proposed method interleaves both optimizations. The novelty of this paper is in the usage of hierarchical RBF network model for selecting the important input variables and for breast cancel detection.The paper is organized as follows.The RBF network is introduced in Section 2.An optimal design method for constructing the HRBF networks is described in Section3.Section4gives the simulation results.Finally in Section5we present some concluding remarks.2The RBF NetworkAn RBF network is a feed-forward neural network with one hidden layer of RBF units and a linear output layer.By an RBF unit we mean a neuron with multiple real inputs x=(x1,...,x n)and one output y computed as:y=ϕ(ξ);ξ= x−c Cb(1)whereϕ:R→R is a suitable activation function,let us consider Gaussian radial basis functionϕ(z)=e−z2.The center c∈R n,the width b∈R and an n×n real matrix C are a unit’s parameters,||·||C denotes a weighted normdefined as x 2C =(Cx)T(Cx)=x T C T Cx.Thus,the network represents the following real function f:R n→R m:f s(x)=hj=1w js e−( x−c Cb)2,s=1,...,m,(2)where w js∈R are weights of s-th output unit and f s is the s-th network output.Evolving Hierarchical RBF Neural Networks for Breast Cancer Detection 139x x x x 1234x x x Fig.1.A RBF neural network (left),an example of hierarchical RBF network (middle),and a tree-structural representation of the HRBF network (right)The goal of an RBF network learning is to find suitable values of RBF units’parameters and the output layer’s weights,so that the RBF network function approximates a function given by a set of examples of inputs and desired outputs T ={x (t ),d (t );t =1,...,k },called a training set .The quality of the learned RBF network is measured by the error function :E =12k t =1m j =1e 2j (t ),e j (t )=d j (t )−f j (t ).(3)3The Hierarchical RBF Network 3.1Encode and CalculationA function set F and terminal instruction set T used for generating a HRBF network model are described as S =F T ={+2,+3,...,+N } {x 1,...,x n },where +i (i =2,3,...,N )denote non-leaf nodes’instructions and taking i argu-ments.x 1,x 2,...,x n are leaf nodes’instructions and taking no arguments.The output of a non-leaf node is calculated as a HRBF network model (see Fig.1).From this point of view,the instruction +i is also called a basis function operator with i inputs.In this research,Gaussian radial basis function is used and the number of radial basis functions used in hidden layer of the network is same with the number of inputs,that is,m =n .In the creation process of HRBF network tree,if a nonterminal instruction,i.e.,+i (i =2,3,4,...,N )is selected,i real values are randomly generated and used for representing the connection strength between the node +i and its chil-dren.In addition,2×n 2adjustable parameters a i and b i are randomly created as radial basis function parameters.The output of the node +i can be calcu-lated by using Eqn.(1)and Eqn.(2).The overall output of HRBF network tree can be computed from left to right by depth-first method,recursively.Finding an optimal or near-optimal HRBF network structure is formulated as a product140Y.Chen,Y.Wang,and B.Yangof evolution.In our previously studies,the Genetic Programming(GP),Proba-bilistic Incremental Program Evolution(PIPE)have been explored for structure optimization of the FNT.In this paper,the ECGP is employed tofind an optimal or near-optimal structure of HRBF networks.3.2Tree Structure Optimization by ECGPFinding an optimal or near-optimal HRBF is formulated as a product of evo-lution.In our previously studies,the Genetic Programming(GP),Probabilistic Incremental Program Evolution(PIPE)have been explored for structure op-timization of the FNT[10][11].In this paper,the Extended Compact Genetic Programming(ECGP)[17]is employed tofind an optimal or near-optimal HRBF structure.ECGP is a direct extension of ECGA to the tree representation which is based on the PIPE prototype tree.In ECGA,Marginal Product Models(MPMs)are used to model the interaction among genes,represented as random variables, given a population of Genetic Algorithm individuals.MPMs are represented as measures of marginal distributions on partitions of random variables.ECGP is based on the PIPE prototype tree,and thus each node in the prototype tree is a random variable.ECGP decomposes or partitions the prototype tree into sub-trees,and the MPM factorises the joint probability of all nodes of the proto-type tree,to a product of marginal distributions on a partition of its sub-trees.A greedy search heuristic is used tofind an optimal MPM mode under the framework of minimum encoding inference.ECGP can represent the probability distribution for more than one node at a time.Thus,it extends PIPE in that the interactions among multiple nodes are considered.3.3Parameter Optimization with DE AlgorithmThe DE algorithm wasfirst introduced by Storn and Price in1995[8].It resem-bles the structure of an evolutionary algorithm(EA),but differs from traditional EAs in its generation of new candidate solutions and by its use of a’greedy’selec-tion scheme.DE works as follows:First,all individuals are randomly initialized and evaluated using thefitness function provided.Afterwards,the following pro-cess will be executed as long as the termination condition is not fulfilled:For each individual in the population,an offspring is created using the weighted difference of parent solutions.The offspring replaces the parent if it isfitter.Otherwise, the parent survives and is passed on to the next iteration of the algorithm.Ingeneration k,we denote the population members by x k1,x k2,...,x kN .The DEalgorithm is given as follows[9]:S1Set k=0,and randomly generate N points x01,x02,...,x0N from searchspace to form an initial population;S2For each point x k i(1≤i≤N),execute the DE offspring generation schemeto generate an offspring x(i k+1);S3If the given stop criteria is not met,set k=k+1,goto step S2.Evolving Hierarchical RBF Neural Networks for Breast Cancer Detection141The DE Offspring Generation approach used is given as follows,S1Choose one point x d randomly such that f(x d)f(x k i),another two points x b,x c randomly from the current population and a subset S={j1,...,j m} of the index set{1,...,n},while m<n and all j i mutually different;S2Generate a trial point u=(u1,u2,...,u n)as follows:DE Mutation.Generate a temporary point z as follows,z=(F+0.5)x d+(F−0.5)x i+F(x b−x c);(4) Where F is a give control parameter;DE Crossover.for j∈S,u j is chosen to be z j;otherwise u j is chosen a to be(x k i)j;S3If f(u)≤f(x k i),set x k+1i =u;otherwise,set x k+1i=x k i.3.4Procedure of the General Learning AlgorithmThe general learning procedure for constructing the HRBF network can be de-scribed as follows.S1Create an initial population randomly(HRBF network trees and its corre-sponding parameters);S2Structure optimization is achieved by using ECGP algorithm;S3If a better structure is found,then go to step S4,otherwise go to step S2; S4Parameter optimization is achieved by DE algorithm.In this stage,the ar-chitecture of HRBF network model isfixed,and it is the best tree developed during the end of run of the structure search;S5If the maximum number of local search is reached,or no better parameter vector is found for a significantly long time then go to step S6;otherwise go to step S4;S6If satisfactory solution is found,then the algorithm is stopped;otherwise go to step S2.3.5Variable Selection Using HRBF Network ParadigmsIt is often a difficult task to select important variables for a classification or re-gression problem,especially when the feature space is large.Conventional RBF neural network usually cannot do this.In the perspective of HRBF network framework,the nature of model construction procedure allows the HRBF net-work to identify important input features in building a HRBF network model that is computationally efficient and effective.The mechanisms of input selec-tion in the HRBF network constructing procedure are as follows.(1)Initially the input variables are selected to formulate the HRBF network model with same probabilities;(2)The variables which have more contribution to the objective function will be enhanced and have high opportunity to survive in the next generation by an evolutionary procedure;(3)The evolutionary operators i.e., crossover and mutation,provide a input selection method by which the HRBF network should select appropriate variables automatically.142Y.Chen,Y.Wang,and B.YangTable parative results of the FNT,NN,RBF[12]and the proposed HRBF network classification methods for the detection of breast cancerCancer typeFNT(%)NN(%)RBF-NN(%)HRBF(%)Benign93.3194.0194.1296.83Malignant 93.4595.4293.2196.83Table 2.The important features selected by the HRBF networkx 0,x 1,x 2,x 3,x 6,x 7,x 9,x 18,x 20,x 25,x 27,x 29Fig.2.The optimized HRBF network for breast cancel detection4SimulationsAs a preliminary study,we made use of the Wisconsin breast cancer data set from the UCI machine-learning database repository [18].This data set has 30attributes (30real valued input features)and 569instances of which 357are of benign and 212are of malignant type.The data set is randomly divided into a training data set and a test data set.The first 285data is used for training and the remaining 284data is used for testing the performance of the different models.All the models were trained and tested with the same set of data.The instruc-tion sets used to create an optimal HRBF network classifier is S =F T ={+2,...,+5} {x 0,x 1,...,x 29}.Where x i (i =0,1,...,29)denotes the 30input fea-tures.The optimal hierarchical HRBF network for breast cancel detection prob-lem is shown in Figure 2.The classification results for testing data set are shown in Table 1.For comparison purpose,the detection performances of the FNT,NN and RBF-NN are also shown in Table 1(for details,see [12]).The important features for constructing the HRBF network models are shown in Table 2.ItEvolving Hierarchical RBF Neural Networks for Breast Cancer Detection143 parison of false positive rate(fp)and true positive rate(tp)for FNT, NN,RBF-NN[12]and hierarchical HRBF networkCancer FNT NN RBF-NN HRBFType fp(%)tp(%)fp(%)tp(%)fp(%)tp(%)fp(%)tp(%) Benign 3.8891.71 4.8593.37 6.697.14 2.9196.69 Malignant 2.7686.41 4.9796.129.296.87 3.3197.09should be noted that the obtained HRBF network classifier has smaller size and reduced features and with high accuracy in breast cancel detection.Receiver Operating Characteristics(ROC)analysis of the FNT,NN,RBF-NN and the HRBF network model is shown in Table3.5ConclusionIn this paper,we presented an optimized HRBF network for the detection of breast cancel and compared the results with some advanced artificial intelligence techniques,i.e.,FNT,NN and RBF-NN.As depicted in Table1,the prelimi-nary results are very encouraging.The best accuracy was offered by the HRBF network method followed by the RBF neural network for detecting benign types and PSO trained neural network for detecting the malignant type of cancer. An important advantage of the HRBF network model is the ability to reduce the number of input variables as presented in Table2.ROC analysis(Table3) illustrates that RBF neural network has the highest false positive rate and the HRBF network model has the lowest false positive rates for detecting benign and malignant cancer.The time required to construct these models are not very much and hope these tools would assist the physician’s effort to improve the currently available automated ways to diagnose breast cancer. AcknowledgmentThis research was partially supported the Natural Science Foundation of China under contract number60573065,and The Provincial Science and Technology Development Program of Shandong under contract number SDSP2004-0720-03. References1.DeSilva,C.J.S.et al.,Artificial Neural networks and Breast Cancer Prognosis,TheAustralian Computer Journal,26,pp.78-81,1994.2.The Weekend Australia,Health Section,pp.7.July,13-14,2002.3.David B.Fogel,Eugene C.Wasson,Edward M.Boughton and Vincent W.Porto,A step toward computer-assisted mammography using evolutionary programmingand neural networks,Cancer Letters,Volume119,Issue1,pp.93-97,1997.144Y.Chen,Y.Wang,and B.Yang4.Charles E.Kahn,Jr,Linda M.Roberts,Katherine A.Shaffer and Peter Haddawy,Construction of a Bayesian network for mammographic diagnosis of breast cancer, Computers in Biology and Medicine,Volume27,Issue1,pp.19-29,1997.5.Shinsuke Morio,Satoru Kawahara,Naoyuki Okamoto,Tadao Suzuki,TakashiOkamoto,Masatoshi Haradas and Akio Shimizu,An expert system for early de-tection of cancer of the breast,Computers in Biology and Medicine,Volume19, Issue5,pp.295-305,1989.6.Barbara S.Hulka and Patricia G.Moorman,Breast Cancer:Hormones and OtherRisk Factors,Maturitas,Volume38,Issue1,pp.103-113,2001.7.Jain,R.and Abraham,A.,A Comparative Study of Fuzzy Classifiers on BreastCancer Data,Australiasian Physical And Engineering Sciences in Medicine,Aus-tralia,Volume27,No.4,pp.147-152,2004.8.R.Storn,and K.Price,”Differential evolution-a simple and efficient adaptivescheme for global optimization over continuous spaces,”Technical report,Interna-tional Computer Science Institute,Berkley,1995.9.K.Price,”Differential Evolution vs.the Functions of the2nd ICEO,”In pro-ceedings of1997IEEE International Conference on Evolutionary Computation (ICEC’97),Indianapolis,USA,pp.153-157,1997.10.Chen,Y.,Yang,B.,Dong,J.,Nonlinear systems modelling via optimal design ofneural trees,International Journal of Neural ystems,14,pp.125-138,2004.11.Chen,Y.,Yang,B.,Dong,J.,Abraham A.,Time-series forcasting usingflexibleneural tree model,Information Science,Vol.174,Issues3/4,pp.219-235,2005. 12.Chen,Y.,Abraham,A.,Yang,B.,Hybrid Neurocomputing for Breast Cancer De-tection,The Fourth International Workshop on Soft Computing as Transdisci-plinary Science and Technology(WSTST’05),pp.884-892,Springer,2005.13.Chen,Y.,Yang,B.and Dong,J.,Evolving Flexible Neural Networks using AntProgramming and PSO algorithm,International Symposium on Neural Networks (ISNN’04),LNCS3173,pp.211-216,2004.14.N.A.Mat Isa,Mashor,M.Y.,and Othman,N.H.,”Diagnosis of Cervical Cancerusing Hierarchical Radial Basis Function(HiRBF)Network,”In Sazali Yaacob,R.Nagarajan,Ali Chekima(Eds.),Proceedings of the International Conference on Artificial Intelligence in Engineering and Technology,pp.458-463,2002.15.S.Ferrari,I.Frosio,V.Piuri,and N.Alberto Borghese,”Automatic MultiscaleMeshing Through HRBF Networks,”IEEE Trans.on Instrumentation and Mea-surment,vol.54,no.4,pp.1463-1470,2005.16.Z.Ahmad,J.Zhang,”Bayesian selective combination of multiple neural networksfor improving long-range predictions in nonlinear process modelling,”Neural Com-put&Applic.Vol.14.pp.78C87,2005.17.K.Sastry and D.E.Goldberg.”Probabilistic model building and competent ge-netic programming”,In R.L.Riolo and B.Worzel,editors,Genetic Programming Theory and Practise,chapter13,pp.205-220.Kluwer,2003.18.Merz J.,and Murphy,P.M.,UCI repository of machine learning databases,/-learn/MLRepository.html,1996.。