Rapid and briefcommunication

Pattern Recognition36(2003)3015–3018/locate/patcogRapid and briefcommunicationAn adaptive rough fuzzy single pass algorithm for clusteringlarge data setsS.Asharaf a,M.Narasimha Murty b;∗a Systems Science and Automation Indian Institute of Science,Bangalore560012,Indiab Department of Computer Science and Automation,Indian Institute of Science,Bangalore560012,IndiaReceived2December2002;accepted26December20021.IntroductionCluster analysis has been widely applied in many areas such as data mining,geographical data processing,medicine, classiÿcation ofstatisticalÿndings in social studies and so on.Most ofthese domains deal with massive collections of data.Hence the methods to handle them must be e cient both in terms ofthe number ofdata set scans and memory usage.Several algorithms have been proposed in the literature for clustering large data sets viz;CLARANS[1],DB-SCAN [1],CURE[1],K-Means[2],etc.Most ofthese require more than one pass through the data set toÿnd the required abstraction.Hence they are computationally expensive for the clustering oflarge data sets.Even though we have a single pass clustering algorithm called BIRCH[1],it uses a memory expensive data structure called CF tree.In this scenario the Leader algorithm[3],which requires only a single data set scan and less memory,turns out to be a potential candidate.This paper introduces an e cient variant ofleader algo-rithm called Adaptive Rough Fuzzy Leader(ARFL)algo-rithm which out-performs the conventional leader algorithm. It employs a combination ofRough Set theory[4]and Fuzzy set theory[5]to capture the intrinsic uncertainty involved in cluster analysis.The paper is organized as follows.Section 2discuss the conventional Leader algorithm,Section3in-troduces the proposed algorithm,in Section4a comparative study ofthe algorithm with conventional leader algorithm and single pass K-means algorithm is given and Section5 deals with conclusions.∗Corresponding author.Tel.:+91-80-394-2779;fax: +91-80-360-2911.E-mail addresses:asharaf@csa.iisc.ernet.in(S.Asharaf), mnm@csa.iisc.crnet.in(M.N.Murty).2.Conventional leader algorithm for clustering Leader clustering algorithm makes only a single pass through the data set andÿnds a set ofleaders as the cluster representatives.It uses a user speciÿed threshold and one ofthe patterns as the starting leader.At any step,the algo-rithm assigns the current pattern to the most similar cluster (leader)or the pattern itselfmay get added as a leader ifits similarity with the current set ofleaders does not qualif y it to get added to any ofthe clusters based on a user speciÿed threshold.The found set of leaders acts as the prototype set representing the clusters and is used for classifying test data.3.Adapt ive rough fuzzy leader(ARFL)clust eringThe ARFL clustering scheme that we propose in this paper divides the data set into a set ofoverlapping clusters.To deÿne the clusters it employs the Rough set theory and here each cluster is represented by a leader,a Lower Bound and an Upper Bound.The Lower Bound ofa cluster contains all the patterns that deÿnitely belong to the cluster.There can be overlap in the Upper Bounds oftwo or more clusters and the algorithm is adaptive in the sense that based on the nature ofthe overlap,the Upper Bound may get adapted. This is a two phase algorithm employing a single pass through the data set.In theÿrst phase,the algorithm per-forms a pass through the data set andÿnds an abstraction of the clusters as some Leaders and Supporting Leaders.The Supporting Leaders are patterns with an intrinsic ambiguity in their assignment to some leaders and they themselves may provide a better level ofabstraction in deÿning the clusters, ifthey get added as leaders.Theÿrst phase starts with any ofthe patterns as the start-ing leader.At any step in this phase,the algorithm uses two user speciÿed parameters called Lower Threshold(L T)0031-3203/03/$30.00?2003Pattern Recognition Society.Published by Elsevier Ltd.All rights reserved. doi:10.1016/S0031-3203(03)00081-53016S.Asharaf,M.N.Murty/Pattern Recognition36(2003)3015–3018 and Upper Threshold(U T)along with the fuzzy member-ship values ofthe pattern among the various leaders avail-able to determine whether a pattern should get added to theLower Bound ofsome leader or Upper Bound ofone/moreleaders or the pattern itselfshould get added as a leader.Thedegree and nature ofoverlap in the Upper Bound ofdi erentleaders on a candidate pattern and a user speciÿed parame-ter called Overlap Threshold(O T)is used to determine(a)whether the addition ofthe current pattern(ifit happens)isas a leader or supporting leader and(b)whether adaptationis needed in the Upper Bound region ofone/more clusters.The fuzzy membership of a candidate pattern CP i in acluster represented by Leader L k is found asU ik=(D(CP i;L k)=D(CP i;L j))2=(m−1)−1;(1)where D()is some measure ofdissimilarity and m is a user speciÿed fuzzy weighting factor.At any step r ofthe algorithm,let N l be the number of currently available leaders.Depending on the value of U ik and the user speciÿed parameters,one ofthe three cases can arise for the assignment of the current pattern CP i.(1)It gets added to the Lower Bound ofany cluster.The current pattern CP i gets added to the Lower Bound ofthe cluster represented by L cifMAX{U ik=k=1:::N l}=U ic and D(CP i;L c)¡L T.(2)It gets added to the the upper bound ofone/more clus-ter/clustersCP i falls in to the Upper Bound ofall the clusters L r for which D(CP i;L r)¡U T.Ifthe number ofclusters that are overlapping in CP i is more than O T,the Up-per Bound ofeach overlapping cluster L o is adapted by modifying its U T value asmul=1−(D(CP i;L o)D(CP i;L r))U T(L o)=MAX{mul∗U T(L o);L T}(2) where r takes value from1to N o,which is the number ofoverlapping clusters.In this case CP i gets added as a leader ifit does not f all in the class ofthe majority ofthe overlap-ping clusters.It gets added as a supporting leader if all the available classes have equal membership among the overlapping leaders.Another case that can arise is when overlap on CP i does not cross O T.In this case it will get added as a supporting leader and no adapta-tion takes place in the Upper Bound ofthe overlapping clusters.(3)Gets added as Leader since it is outside the regiondeÿned by any ofthe existing clusters.The second phase ofthe algorithm adds some ofthe Supporting Leaders as leaders to the existing set of leaders ifit improves the quality ofthe prototype set.In this phase the algorithm tries to classify the sup-porting leaders using the available leaders.Ifthey are incorrectly classiÿed they get added as leaders since they themselves contribute to the quality ofthe proto-type set due to this addition.The AlgorithmData Structure usedUses two sets{Leader}:-keeps the set ofall leaders. {Supporting Leader}:-to maintain the supportingleaders.Algorithm1.Initialize{Leader}with any one pattern from the Data Set to be Clustered2.For all the patterns in the Data Set Do{Find the fuzzy membership value of the current pattern CP i for the set of available Leaders as per Eq.(1).Find the Leader L j with the maximum Fuzzy membership value among{Leader}If D(CP i;L j)¡L Tadd CP i to the lower bound ofthe cluster representedby L jElse{For all the Leaders L k from{Leader}such thatD(CP i;L k)¡U T Do{//adding current pattern CP i to the Upper Bound ofone or more LeadersOverlap=Overlap+1;Record the Class label of L k}If(Overlap¿O T){For all all the over lapping Leaders L o doAdjust the U T(L o)as per Eq.(2)Find the class Max Class in which the maximum number ofOverlapping Leaders L o arefalling inIfMax Class Class label(CP i)add current patternCP i to{Leader}ElseIfall Classes have equal membership values among theset ofoverlapping Leaders{L o}add CP i to{Supporting Leader}}Else add CP i to{Supporting Leader}}Ifthe current pattern CP i does not belong to eitherthe Lower Bound or the Upper Bound ofany ofthe Leaders f rom{Leader}add pattern CP i to{Leader}}S.Asharaf,M.N.Murty /Pattern Recognition 36(2003)3015–30183017Table 1Shows the performance of SPKM,L,FL,RL and ARFL algorithms SPKM LEADER FL RL ARFL NP CA NP CA NP CA NP CA NP CA 112327.90111079.99113881.28111282.78108385.81130728.56131981.97134883.68131783.56130786.11147031.50143783.68141484.39148484.46147087.13158831.08154484.25158885.30155585.18158888.66220138.70198387.13187787.25196587.94207789.35100011001200130014001500160017001800190020002100COMPARATIVE STUDY− No of Patterns − >− C l a s s i f i c a t i o n A c c u r a c y − >Fig.1.Shows a comparison ofSD,S Dbw and hybrid validity indices when used for the evaluation of ARFL algorithm.3.For all the Supporting Leader SL j from {support Leader }doClassify SL j using {Leader }If SL j is incorrectly classiÿed add SL j to {Leader }4.Return the set {Leader }as the set ofprototypesrepresenting the clusters4.Experimental resultsSingle Pass K-Means(SPKM),Conventional Leader (L)algorithm,Fuzzy Leader (FL)algorithm,Rough Leader (RL)algorithm and ARFL clustering algorithm are imple-mented and used for generating the prototypes.4.1.Prototype selectionLet X ={X i =i =1:::n }be the data set.Let the cluster ab-straction generated by a clustering algorithm be C ={C i =i =1:::m }.Let the corresponding cluster descriptions be given by the prototypes R ={R i =i =1:::m }.In the case ofthe conventional Leader algorithm and the variants ofLeader algorithm,the obtained leaders form the prototypes.For the Single Pass K-Means algorithm the centroid ofeach cluster C i forms the prototype representing that cluster.4.2.Data set usedThe data set used is handwritten digit data.The training data consist of 667patterns for each digit from 0to 9,to-3018S.Asharaf,M.N.Murty/Pattern Recognition36(2003)3015–3018tally6670patterns.The test data consists of3333patterns [3].Each ofthis patterns is of193dimensions and the last dimension shows the class label.parative studiesTable1and Fig.1show the Number ofPrototypes(NP) generated and the corresponding Classiÿcation Accuracy (CA)obtained using Nearest Neighbour Classiÿer,for the di erent algorithms whose performance is compared. Among the di erent algorithms we have considered,it can be seen from the table andÿgure that,the ARFL algorithm gives the best performance in terms of both memory usage and classiÿcation accuracy.The proposed ARFL algorithm is pragmatic in the sense that it selects and maintains only the most appropriate patterns as the representatives ofthe clusters.5.ConclusionA novel variant ofthe conventional Leader algorithm f or clustering oflarge data set is proposed.The advantages of the proposed scheme are:(a)It generates cluster abstrac-tion in a single data set scan,(b)the clusters generated by this new algorithm can be ofarbitrary shape,(c)the quality ofthe cluster abstraction generated by ARFL out-weighs the other single pass clustering schemes,(d)this is a scalable algorithm and(e)The memory requirement ofthe algorithm is limited to the space required for two simple set implementations.References[1]A.K.Pujari,Data Mining Techniques,Universities Press,Nancy,2001.[2]F.Farnstrom,J.Lewis,C.Elkan,Scalability for clusteringalgorithms revisited,SIGKDD Explor.2(1)(2000)51–57.[3]T.R.Babu,M.N.Murty,Comparison ofG enetic algorithmbased prototype selection scheme,Pattern Recognition34(2) (2001)523–525.[4]Z.Pawlak,Rough sets,put.Inf.Sci.11(1982)341–356.[5]L.Zadeh,Fuzzy sets,Inf.Control8(1965)338–353.。