Using a Genetic Algorithm for Optimizing the Functional Decomposition of Multiple-Valued Fu

合集下载

多目标遗传算法

A more appropriate approach to deal with multiple objectives is to use techniques that were originally designed for that purpose in the eld of Operations Research. Work in that area started a century ago, and many approaches have been re ned and commonly applied in economics and control theory.
Abstract: In this paper we propose the use of the genetic algorithm (GA) as a tool to solve multiobjective
optimization problems in structures. Using the concept of min-max optimum, a new GA-based multiobjective optimization technique is proposed and two truss design problems are solved using it. The results produced by this new approach are compared to those produced by other mathematical programminቤተ መጻሕፍቲ ባይዱ techniques and GA-based approaches, proving that this technique generates better trade-o s and that the genetic algorithm can be used as a reliable numerical optimization tool.

Genetic Algorithms for Multiobjective Optimization Formulation ...

Carlos M.Fonseca†and Peter J.Fleming‡Dept.Automatic Control and Systems Eng.University of SheﬃeldSheﬃeld S14DU,U.K.AbstractThe paper describes a rank-basedﬁtness as-signment method for Multiple Objective Ge-netic Algorithms(MOGAs).Conventionalniche formation methods are extended to thisclass of multimodal problems and theory forsetting the niche size is presented.Theﬁt-ness assignment method is then modiﬁed toallow direct intervention of an external deci-sion maker(DM).Finally,the MOGA is gen-eralised further:the genetic algorithm is seenas the optimizing element of a multiobjectiveoptimization loop,which also comprises theDM.It is the interaction between the twothat leads to the determination of a satis-factory solution to the problem.Illustrativeresults of how the DM can interact with thegenetic algorithm are presented.They alsoshow the ability of the MOGA to uniformlysample regions of the trade-oﬀsurface.1INTRODUCTIONWhilst most real world problems require the simulta-neous optimization of multiple,often competing,cri-teria(or objectives),the solution to such problems isusually computed by combining them into a single cri-terion to be optimized,according to some utility func-tion.In many cases,however,the utility function isnot well known prior to the optimization process.Thewhole problem should then be treated as a multiobjec-tive problem with non-commensurable objectives.Inthis way,a number of solutions can be found whichprovide the decision maker(DM)with insight into thecharacteristics of the problem before aﬁnal solution ischosen.2VECTOR EV ALUATED GENETICALGORITHMSBeing aware of the potential GAs have in multiob-jective optimization,Schaﬀer(1985)proposed an ex-tension of the simple GA(SGA)to accommodate vector-valuedﬁtness measures,which he called the Vector Evaluated Genetic Algorithm(VEGA).The se-lection step was modiﬁed so that,at each generation, a number of sub-populations was generated by per-forming proportional selection according to each ob-jective function in turn.Thus,for a problem with q objectives,q sub-populations of size N/q each would be generated,assuming a population size of N.These would then be shuﬄed together to obtain a new popu-lation of size N,in order for the algorithm to proceed with the application of crossover and mutation in the usual way.However,as noted by Richardson et al.(1989),shuf-ﬂing all the individuals in the sub-populations together to obtain the new population is equivalent to linearly combining theﬁtness vector components to obtain a single-valuedﬁtness function.The weighting coeﬃ-cients,however,depend on the current population. This means that,in the general case,not only will two non-dominated individuals be sampled at diﬀer-ent rates,but also,in the case of a concave trade-oﬀsurface,the population will tend to split into diﬀer-ent species,each of them particularly strong in one of the objectives.Schaﬀer anticipated this property of VEGA and called it speciation.Speciation is unde-sirable in that it is opposed to the aim ofﬁnding a compromise solution.To avoid combining objectives in any way requires a diﬀerent approach to selection.The next section de-scribes how the concept of inferiority alone can be used to perform selection.3A RANK-BASED FITNESSASSIGNMENT METHOD FORMOGAsConsider an individual x i at generation t which is dom-inated by p(t)i individuals in the current population.Its current position in the individuals’rank can be given byrank(x i,t)=1+p(t)i.All non-dominated individuals are assigned rank1,see Figure1.This is not unlike a class of selection meth-ods proposed by Fourman(1985)for constrained opti-mization,and correctly establishes that the individual labelled3in theﬁgure is worse than individual labelled 2,as the latter lies in a region of the trade-oﬀwhich is less well described by the remaining individuals.The13511211f1f2Figure1:Multiobjective Rankingmethod proposed by Goldberg(1989,p.201)would treat these two individuals indiﬀerently. Concerningﬁtness assignment,one should note that not all ranks will necessarily be represented in the pop-ulation at a particular generation.This is also shown in the example in Figure1,where rank4is absent. The traditional assignment ofﬁtness according to rank may be extended as follows:1.Sort population according to rank.2.Assignﬁtnesses to individuals by interpolatingfrom the best(rank1)to the worst(rank n∗≤N) in the usual way,according to some function,usu-ally linear but not necessarily.3.Average theﬁtnesses of individuals with the samerank,so that all of them will be sampled at the same rate.Note that this procedure keeps the global populationﬁtness constant while maintain-ing appropriate selective pressure,as deﬁned by the function used.Theﬁtness assignment method just described appears as an extension of the standard assignment ofﬁtness according to rank,to which it maps back in the case of a single objective,or that of non-competing objectives.4NICHE-FORMATION METHODS FOR MOGAsConventionalﬁtness sharing techniques(Goldberg and Richardson,1987;Deb and Goldberg,1989)have been shown to be to eﬀective in preventing genetic drift,in multimodal function optimization.However,they in-troduce another GA parameter,the niche sizeσshare, which needs to be set carefully.The existing theory for setting the value ofσshare assumes that the solu-tion set is composed by an a priori knownﬁnite num-ber of peaks and uniform niche placement.Upon con-vergence,local optima are occupied by a number of individuals proportional to theirﬁtness values.On the contrary,the global solution of an MO prob-lem isﬂat in terms of individualﬁtness,and there is no way of knowing the size of the solution set before-hand,in terms of a phenotypic metric.Also,local optima are generally not interesting to the designer, who will be more concerned with obtaining a set of globally non-dominated solutions,possibly uniformly spaced and illustrative of the global trade-oﬀsurface. The use of ranking already forces the search to concen-trate only on global optima.By implementingﬁtness sharing in the objective value domain rather than the decision variable domain,and only between pairwise non-dominated individuals,one can expect to be able to evolve a uniformly distributed representation of the global trade-oﬀsurface.Niche counts can be consistently incorporated into the extendedﬁtness assignment method described in the previous section by using them to scale individualﬁt-nesses within each rank.The proportion ofﬁtness allo-cated to the set of currently non-dominated individuals as a whole will then be independent of their sharing coeﬃcients.4.1CHOOSING THE PARAMETERσshare The sharing parameterσshare establishes how far apart two individuals must be in order for them to decrease each other’sﬁtness.The exact value which would allow a number of points to sample a trade-oﬀsurface only tangentially interfering with one another obviously de-pends on the area of such a surface.As noted above in this section,the size of the set of so-lutions to a MO problem expressed in the decision vari-able domain is not known,since it depends on the ob-jective function mappings.However,when expressed in the objective value domain,and due to the deﬁni-tion of non-dominance,an upper limit for the size of the solution set can be calculated from the minimum and maximum values each objective assumes within that set.Let S be the solution set in the decision variable domain,f(S)the solution set in the objective domain and y=(y1,...,y q)any objective vector in f(S).Also,letm=(miny y1,...,minyy q)=(m1,...,m q)M=(maxy y1,...,maxyy q)=(M1,...,M q)as illustrated in Figure2.The deﬁnition of trade-oﬀsurface implies that any line parallel to any of the axes will have not more than one of its points in f(S),which eliminates the possibility of it being rugged,i.e.,each objective is a single-valued function of the remaining objectives.Therefore,the true area of f(S)will be less than the sum of the areas of its projections according to each of the axes.Since the maximum area of each projection will be at most the area of the correspond-ing face of the hyperparallelogram deﬁned by mand Figure2:An Example of a Trade-oﬀSurface in3-Dimensional SpaceM,the hyperarea of f(S)will be less thanA=qi=1qj=1j=i(M j−m j)which is the sum of the areas of each diﬀerent face of a hyperparallelogram of edges(M j−m j)(Figure3). In accordance with the objectives being non-commensurable,the use of the∞-norm for measuring the distance between individuals seems to be the most natural one,while also being the simplest to compute. In this case,the user is still required to specify an indi-vidualσshare for each of the objectives.However,the metric itself does not combine objective values in any way.Assuming that objectives are normalized so that all sharing parameters are the same,the maximum num-ber of points that can sample area A without in-terfering with each other can be computed as the number of hypercubes of volumeσqsharethat can be placed over the hyperparallelogram deﬁned by A(Fig-ure4).This can be computed as the diﬀerence in vol-ume between two hyperparallelograms,one with edges (M i−m i+σshare)and the other with edges(M i−m i), divided by the volume of a hypercube of edgeσshare, i.e.N=qi=1(M i−m i+σshare)−qi=1(M i−m i)Figure3:Upper Bound for the Area of a Trade-oﬀSurface limited by the Parallelogram deﬁned by (m1,m2,m3)and(M1,M2,M3)(q−1)-order polynomial equationNσq−1share −qi=1(M i−m i+σshare)−qi=1(M i−m i)Pareto set of interest to the DM by providing external information to the selection algorithm.Theﬁtness assignment method described earlier was modiﬁed in order to accept such information in the form of goals to be attained,in a similar way to that used by the conventional goal attainment method(Gembicki,1974),which will now be brieﬂy introduced.5.1THE GOAL ATTAINMENT METHOD The goal attainment method solves the multiobjective optimization problem deﬁned asminx∈Ωf(x)where x is the design parameter vector,Ωthe feasible parameter space and f the vector objective function, by converting it into the following nonlinear program-ming problem:minλ,x∈Ωλsuch thatf i−w iλ≤g iHere,g i are goals for the design objectives f i,and w i≥0are weights,all of them speciﬁed by the de-signer beforehand.The minimization of the scalarλleads to theﬁnding of a non-dominated solution which under-or over-attains the speciﬁed goals to a degree represented by the quantities w iλ.5.2A MODIFIED MO RANKINGSCHEME TO INCLUDE GOALINFORMATIONThe MO ranking procedure previously described was extended to accommodate goal information by altering the way in which individuals are compared with one another.In fact,degradation in vector components which meet their goals is now acceptable provided it results in the improvement of other components which do not satisfy their goals and it does not go beyond the goal boundaries.This makes it possible for one to prefer one individual to another even though they are both non-dominated.The algorithm will then identify and evolve the relevant region of the trade-oﬀsurface. Still assuming a minimization problem,consider two q-dimensional objective vectors,y a=(y a,1,...,y a,q) and y b=(y b,1,...,y b,q),and the goal vector g= (g1,...,g q).Also consider that y a is such that it meets a number,q−k,of the speciﬁed goals.Without loss of generality,one can write∃k=1,...,q−1:∀i=1,...,k,∀j=k+1,...,q,(y a,i>g i)∧(y a,j≤g j)(A) which assumes a convenient permutation of the objec-tives.Eventually,y a will meet none of the goals,i.e.,∀i=1,...,q,(y a,i>g i)(B)or even all of them,and one can write∀j=1,...,q,(y a,j≤g j)(C) In theﬁrst case(A),y a meets goals k+1,...,q and, therefore,will be preferable to y b simply if it domi-nates y b with respect to itsﬁrst k components.For the case where all of theﬁrst k components of y a are equal to those of y b,y a will still be preferable to y b if it dominates y b with respect to the remaining com-ponents,or if the remaining components of y b do not meet all their goals.Formally,y a will be preferable to y b,if and only ify a,(1,...,k)p<y b,(1,...,k) ∨y a,(1,...,k)=y b,(1,...,k) ∧y a,(k+1,...,q)p<y b,(k+1,...,q) ∨∼ y b,(k+1,...,q)≤g(k+1,...,q)In the second case(B),y a satisﬁes none of the goals. Then,y a is preferable to y b if and only if it dominates y b,i.e.,y a p<y bFinally,in the third case(C)y a meets all of the goals, which means that it is a satisfactory,though not nec-essarily optimal,solution.In this case,y a is preferable to y b,if and only if it dominates y b or y b is not satis-factory,i.e.,(y a p<y b)∨∼(y b≤g)The use of the relation preferable to as just described, instead of the simpler relation partially less than,im-plies that the solution set be delimited by those non-dominated points which tangentially achieve one or more goals.Setting all the goals to±∞will make the algorithm try to evolve a discretized description of the whole Pareto set.Such a description,inaccurate though it may be,can guide the DM in reﬁning its requirements.When goals can be supplied interactively at each GA generation, the decision maker can reduce the size of the solution set gradually while learning about the trade-oﬀbe-tween objectives.The variability of the goals acts as a changing environment to the GA,and does not im-pose any constraints on the search space.Note that appropriate sharing coeﬃcients can still be calculated as before,since the size of the solution set changes in a way which is known to the DM.This strategy of progressively articulating the DM preferences,while the algorithm runs,to guide the search,is not new in operations research.The main disadvantage of the method is that it demands a higher eﬀort from the DM.On the other hand,it potentially reduces the number of function evaluations required when compared to a method for a posteriori articula-tion of preferences,as well as providing less alternativeddddDM a priori knowledgeGAobjective function values ﬁtnesses(acquired knowledge)resultsFigure 5:A General Multiobjective Genetic Optimizerpoints at each iteration,which are certainly easier for the DM to discriminate between than the whole Pareto set at once.6THE MOGA AS A METHOD FOR PROGRESSIVE ARTICULATION OF PREFERENCESThe MOGA can be generalized one step further.The DM action can be described as the consecutive evalu-ation of some not necessarily well deﬁned utility func-tion .The utility function expresses the way in which the DM combines objectives in order to prefer one point to another and,ultimately,is the function which establishes the basis for the GA population to evolve.Linearly combining objectives to obtain a scalar ﬁt-ness,on the one hand,and simply ranking individuals according to non-dominance,on the other,both corre-spond to two diﬀerent attitudes of the DM.In the ﬁrst case,it is assumed that the DM knows exactly what to optimize,for example,ﬁnancial cost.In the second case,the DM is making no decision at all apart from letting the optimizer use the broadest deﬁnition of MO optimality.Providing goal information,or using shar-ing techniques,simply means a more elaborated atti-tude of the DM,that is,a less straightforward utility function,which may even vary during the GA process,but still just another utility function.A multiobjective genetic optimizer would,in general,consist of a standard genetic algorithm presenting the DM at each generation with a set of points to be as-sessed.The DM makes use of the concept of Pareto optimality and of any a priori information available to express its preferences,and communicates them to the GA,which in turn replies with the next generation.At the same time,the DM learns from the data it is presented with and eventually reﬁnes its requirements until a suitable solution has been found (Figure 5).In the case of a human DM,such a set up may require reasonable interaction times for it to become attrac-tive.The natural solution would consist of speedingup the process by running the GA on a parallel ar-chitecture.The most appealing of all,however,would be the use of an automated DM,such as an expert system.7INITIAL RESULTSThe MOGA is currently being applied to the step response optimization of a Pegasus gas turbine en-gine.A full non-linear model of the engine (Han-cock,1992),implemented in Simulink (MathWorks,1992b),is used to simulate the system,given a num-ber of initial conditions and the controller parameter settings.The GA is implemented in Matlab (Math-Works,1992a;Fleming et al.,1993),which means that all the code actually runs in the same computation en-vironment.The logarithm of each controller parameter was Gray encoded as a 14-bit string,leading to 70-bit long chro-mosomes.A random initial population of size 80and standard two-point reduced surrogate crossover and binary mutation were used.The initial goal values were set according to a number of performance require-ments for the engine.Four objectives were used:t r The time taken to reach 70%of the ﬁnal output change.Goal:t r ≤0.59s.t s The time taken to settle within ±10%of the ﬁnal output change.Goal:t s ≤1.08s.os Overshoot,measured relatively to the ﬁnal output change.Goal:os ≤10%.err A measure of the output error 4seconds after thestep,relative to the ﬁnal output change.Goal:err ≤10%.During the GA run,the DM stores all non-dominated points evaluated up to the current generation.This constitutes acquired knowledge about the trade-oﬀs available in the problem.From these,the relevant points are identiﬁed,the size of the trade-oﬀsurface estimated and σshare set.At any time in the optimiza-trts ov err 00.20.40.60.81N o r m a l i z e d o b j e c t i v e v a l u e s Objective functions0.59s 1.08s 10% 10%Figure 6:Trade-oﬀGraph for the Pegasus Gas Turbine Engine after 40Generations (Initial Goals)tion process,the goal values can be changed,in order to zoom in on the region of interest.A typical trade-oﬀgraph,obtained after 40genera-tions with the initial goals,is presented in Figure 6and represents the accumulated set of satisfactory non-dominated points.At this stage,the setting of a much tighter goal for the output error (err ≤0.1%)reveals the graph in Figure 7,which contains a subset of the points in Figure 6.Continuing to run the GA,more deﬁnition can be obtained in this area (Figure 8).Fig-ure 9presents an alternative view of these solutions,illustrating the arising step responses.8CONCLUDING REMARKSGenetic algorithms,searching from a population of points,seem particularly suited to multiobjective opti-mization.Their ability to ﬁnd global optima while be-ing able to cope with discontinuous and noisy functions has motivatedan increasing number of applications in engineering and related ﬁelds.The development of the MOGA is one expression of our wish to bring decision making into engineering design,in general,and control system design,in particular.An important problem arising from the simple Pareto-based ﬁtness assignment method is that of the global size of the solution plex problems can be expected to exhibit a large and complex trade-oﬀsur-face which,to be sampled accurately,would ultimately overload the DM with virtually useless information.Small regions of the trade-oﬀsurface,however,can still be sampled in a Pareto-based fashion,while the deci-sion maker learns and reﬁnes its requirements.Niche formation methods are transferred to the objective value domain in order to take advantage of the prop-erties of the Paretoset.Figure 7:Trade-oﬀGraph for the Pegasus Gas Turbine Engine after 40Generations (New Goals)Figure 8:Trade-oﬀGraph for the Pegasus Gas Turbine Engine after 60Generations (New Goals)Figure 9:Satisfactory Step Responses after 60Gener-ations (New Goals)Initial results,obtained from a real world engineering problem,show the ability of the MOGA to evolve uni-formly sampled versions of trade-oﬀsurface regions. They also illustrate how the goals can be changed dur-ing the GA run.Chromosome coding,and the genetic operators them-selves,constitute areas for further study.Redundant codings would eventually allow the selection of the ap-propriate representation while evolving the trade-oﬀsurface,as suggested in(Chipperﬁeld et al.,1992). The direct use of real variables to represent an indi-vidual together with correlated mutations(B¨a ck et al., 1991)and some clever recombination operator(s)may also be interesting.In fact,correlated mutations should be able to identify how decision variables re-late to each other within the Pareto set.AcknowledgementsTheﬁrst author gratefully acknowledges support by Programa CIENCIA,Junta Nacional de Investiga¸c˜a o Cient´ıﬁca e Tecnol´o gica,Portugal.ReferencesB¨a ck,T.,Hoﬀmeister,F.,and Schwefel,H.-P.(1991).A survey of evolution strategies.In Belew,R.,editor,Proc.Fourth Int.Conf.on Genetic Algo-rithms,pp.2–9.Morgan Kaufmann.Chipperﬁeld, A.J.,Fonseca, C.M.,and Fleming, P.J.(1992).Development of genetic optimiza-tion tools for multi-objective optimization prob-lems in CACSD.In IEE Colloq.on Genetic Algo-rithms for Control Systems Engineering,pp.3/1–3/6.The Institution of Electrical Engineers.Di-gest No.1992/106.Deb,K.and Goldberg,D.E.(1989).An investigation of niche and species formation in genetic func-tion optimization.In Schaﬀer,J.D.,editor,Proc.Third Int.Conf.on Genetic Algorithms,pp.42–50.Morgan Kaufmann.Farshadnia,R.(1991).CACSD using Multi-Objective Optimization.PhD thesis,University of Wales, Bangor,UK.Fleming,P.J.(1985).Computer aided design of regulators using multiobjective optimization.In Proc.5th IFAC Workshop on Control Applica-tions of Nonlinear Programming and Optimiza-tion,pp.47–52,Capri.Pergamon Press. Fleming,P.J.,Crummey,T.P.,and Chipperﬁeld,A.J.(1992).Computer assisted control systemdesign and multiobjective optimization.In Proc.ISA Conf.on Industrial Automation,pp.7.23–7.26,Montreal,Canada.Fleming,P.J.,Fonseca,C.M.,and Crummey,T.P.(1993).Matlab:Its toolboxes and open struc-ture.In Linkens,D.A.,editor,CAD for Control Systems,chapter11,pp.271–286.Marcel-Dekker. Fourman,M.P.(1985).Compaction of symbolic lay-out using genetic algorithms.In Grefenstette, J.J.,editor,Proc.First Int.Conf.on Genetic Algorithms,pp.141–wrence Erlbaum. Gembicki,F.W.(1974).Vector Optimization for Con-trol with Performance and Parameter Sensitivity Indices.PhD thesis,Case Western Reserve Uni-versity,Cleveland,Ohio,USA.Goldberg,D.E.(1989).Genetic Algorithms in Search, Optimization and Machine Learning.Addison-Wesley,Reading,Massachusetts.Goldberg,D.E.and Richardson,J.(1987).Genetic algorithms with sharing for multimodal function optimization.In Grefenstette,J.J.,editor,Proc.Second Int.Conf.on Genetic Algorithms,pp.41–wrence Erlbaum.Hancock,S.D.(1992).Gas Turbine Engine Controller Design Using Multi-Objective Optimization Tech-niques.PhD thesis,University of Wales,Bangor, UK.MathWorks(1992a).Matlab Reference Guide.The MathWorks,Inc.MathWorks(1992b).Simulink User’s Guide.The MathWorks,Inc.Richardson,J.T.,Palmer,M.R.,Liepins,G.,and Hilliard,M.(1989).Some guidelines for genetic algorithms with penalty functions.In Schaﬀer, J.D.,editor,Proc.Third Int.Conf.on Genetic Algorithms,pp.191–197.Morgan Kaufmann. Schaﬀer,J.D.(1985).Multiple objective optimiza-tion with vector evaluated genetic algorithms.In Grefenstette,J.J.,editor,Proc.First Int.Conf.on Genetic Algorithms,pp.93–wrence Erl-baum.Wienke,D.,Lucasius,C.,and Kateman,G.(1992).Multicriteria target vector optimization of analyt-ical procedures using a genetic algorithm.Part I.Theory,numerical simulations and application to atomic emission spectroscopy.Analytica Chimica Acta,265(2):211–225.。

人工智能遗传算法

人工智能遗传算法英文回答：Genetic Algorithms for Artificial Intelligence.Genetic algorithms (GAs) are a class of evolutionary algorithms that are inspired by the process of natural selection. They are used to solve optimization problems by iteratively improving a population of candidate solutions.How GAs Work.GAs work by simulating the process of natural selection. In each iteration, the fittest individuals in thepopulation are selected to reproduce. Their offspring are then combined and mutated to create a new population. This process is repeated until a satisfactory solution is found.Components of a GA.A GA consists of the following components:Population: A set of candidate solutions.Fitness function: A function that evaluates thequality of each candidate solution.Selection: The process of choosing the fittest individuals to reproduce.Reproduction: The process of creating new individuals from the selected parents.Mutation: The process of introducing random changes into the new individuals.Applications of GAs.GAs have been used to solve a wide variety of problems, including:Optimization problems.Machine learning.Scheduling.Design.Robotics.Advantages of GAs.GAs offer several advantages over traditional optimization methods, including:They can find near-optimal solutions to complex problems.They are not easily trapped in local optima.They can be used to solve problems with multiple objectives.Disadvantages of GAs.GAs also have some disadvantages, including:They can be computationally expensive.They can be sensitive to the choice of parameters.They can be difficult to terminate.中文回答：人工智能中的遗传算法。

求解矩形Packing问题的基于遗传算法的启发式递归策略

Vol.33,No.9ACTA AUTOMATICA SINICA September,2007 An Improved Heuristic Recursive Strategy Based on Genetic Algorithm for the Strip RectangularPacking ProblemZHANG De-Fu1CHEN Sheng-Da1LIU Yan-Juan1Abstract An improved heuristic recursive strategy combining with genetic algorithm is presented in this paper.Firstly,this method searches some rectangles,which have the same length or width,to form some layers without waste space,then it uses the heuristic recursive strategies to calculate the height of the remaining packing order and uses the evolutionary capability of genetic algorithm to reduce the height.The computational results on several classes of benchmark problems have shown that the presented algorithm can compete with known evolutionary heuristics.It performs better especially for large test problems.Key words Strip packing problems,heuristic,recursive,genetic algorithm1IntroductionMany industrial applications,which belong to cutting and packing problems,have been found.Each application incorporates diﬀerent constraints and objectives.For ex-ample,in wood or glass industries,rectangular components have to be cut from large sheets of material.In warehousing contexts,goods have to be placed on shelves.In newspapers paging,articles and advertisements have to be arranged in pages.In the shipping industry,a batch of objects of var-ious sizes has to be shipped to the maximum extent in a larger container,and a bunch of opticalﬁbers has to be accommodated in a pipe with perimeter as small as possi-ble.In very-large scale integration(VLSI)ﬂoor planning, VLSI has to be laid out.These applications have a similar logical structure,which can be modeled by a set of pieces that must be arranged on a predeﬁned stock sheet so that the pieces do not overlap with one another,so they can be formalized as the packing problem[1].For more extensive and detailed descriptions of packing problems,the reader can refer to[1∼3].A two-dimensional strip rectangular packing problem is considered in this paper.It has the following characteris-tics:a set of rectangular pieces and a larger rectangle with aﬁxed width and inﬁnite length,designated as the con-tainer.The objective is toﬁnd a feasible layout of all the pieces in the container that minimizes the required con-tainer length and,where necessary,takes additional con-straints into account.This problem belongs to a subset of classical cutting and packing problems and has been shown to be non-deterministic polynomial(NP)hard[4,5].Opti-mal algorithms for orthogonal two-dimension cutting were proposed in[6,7].Gilmore and Gomory[8]solved prob-lem instances to optimality by linear programming tech-niques in1961.Christoﬁdes and Whitlock[9]solved the two-dimensional guillotine stock cutting problem to opti-mality by a tree-search method in1977.Cung et al.[10] developed a new version of the algorithm proposed in Hiﬁand Zissimopolous that used a best-ﬁrst branch-and-bound approach to solve exactly some variants of two-dimensional stock-cutting problems in2000.However,these algorithms might not be practical for large problems.In order to solve large problems,some heuristic algorithms were developed. Received June20,2006;in revised form October24,2006 Supported by Academician Start-up Fund(X01109),985Informa-tion Technology Fund(0000-X07204)in Xiamen University1.Department of Computer Science,Xiamen University,Xiamen 361005,P.R.ChinaDOI:10.1360/aas-007-0911The most documented heuristics are the bottom-left(BL), bottom-left-ﬁll(BLF)methods,and other heuristics[11∼13]. Although their computational speed is very fast,the so-lution quality is not desirable.Recently,genetic algo-rithms and its improved algorithms for the orthogonal pack-ing problem were proposed because of their powerful op-timization capability[14∼17].Kroger[14]used genetic algo-rithm for the guillotine variant of bin packing in1995. Jakobs[15]used a genetic algorithm for the packing of poly-gons using rectangular enclosures and a Bottom-left heuris-tic in1996.Liu et al.[16]further improved it.Hop-per and Turton[18]evaluated the use of the BLF heuris-tic with genetic algorithms on the nonguillotine rectangle-nesting problem in1999.In addition,an empirical in-vestigation of meta-heuristic and heuristic algorithms of the strip rectangular packing problems was given by[19]. Recently,some new models and algorithms were devel-oped by[20∼26].For example,quasi-human heuristic[20], constructive approach[21,22],a new placement heuristic[23], heuristic recursion(HR)algorithm[24],and hybrid heuristic algorithms[25,26]were developed.These heuristics are fast and eﬀective,especially,the bestﬁt in[23]not only is very fast,but alsoﬁnds better solutions than some well-known metaheuristics.In this paper,an improved heuristic recursive algorithm that combines with genetic algorithm is presented to solve the orthogonal strip rectangular packing problem.The computational results on a class of benchmark problems show that this algorithm can compete with known evolu-tionary heuristics,especially in large test problems.The rest of this paper is organized as follows.In Section 2,a clear mathematical formulation for the strip rectangu-lar packing problem is given.In Section3,the heuristic recursive algorithm is presented,and an improved heuris-tic recursive algorithm(IHR)is developed in detail.In Section4,the GA+IHR algorithm is puta-tional results are described in Section5.Conclusions are summarized in Section6.2Mathematical formulation of the problemGiven a rectangular board of given width and a set of rectangles with arbitrary sizes,the strip packing problem of rectangles is to pack each rectangle on the board so that no two rectangles overlap and the used board height is min-imized.This problem can also be stated as follows.912ACTA AUTOMATICA SINICA Vol.33Given a rectangular board with given width W,and n rectangles with length l i and width w i,1≤i≤n,let (x li,y li)denote the top-left corner coordinates of rectangle i,and(x ri,y ri)the bottom-right corner coordinates of rect-angle i.Other symbols are similar to[24].For all1≤i≤n, the coordinates of rectangles must satisfy the following con-ditions:1)(x ri−x li=l i and y li−y ri=w i)or(x ri−x li=w i and y li−y ri=l i);2)For all1≤j≤n,j=i,rectangle i and j can not overlap,namely,x ri≤x lj or x li≥x rj or y ri≥y lj or y li≤y rj;3)x L≤x li≤x R,x L≤x ri≤x R and y R≤y li≤h,y R≤y ri≤h.The problem is to pack all the rectangles on the board such that the used board height h is minimized.It is noted that for orthogonal rectangular packing prob-lems the packing process has to ensure the edges of each rectangle are parallel to the x-and y-axes,respectively, namely,all rectangles can not be packed aslant.In addi-tion,all rectangles except the rectangular board are allowed to rotate90degrees.3IHR algorithmThe HR algorithm for the strip rectangular packing prob-lem was presented in[24].It follows a divide-and-conquer approach:break the problem into several subproblems that are similar to the original problem but smaller in size,solve the subproblems recursively,and then combine these solu-tions to create a solution to the original problem.The HR algorithm is very simple and eﬃcient,and can be stated as follows:1)Pack a rectangle into the space to be packed,and divide the unpacked space into two subspaces.2)Pack each subspace by packing it recursively.If the subspace size is small enough to only pack a rectangle,then just pack this rectangle into the subspace in a straightfor-ward manner.3)Combine the solutions to the subproblems for the so-lution of the rectangular packing problem.In order to enhance the performance of the HR algo-rithm,the author in[18]presented some heuristic strate-gies to select a rectangle to be packed,namely,the rect-angle with the maximum area is given priority to pack.In detail,unpacked rectangles should be sorted by nonincreas-ing ordering of area size.The rectangle with maximum area should be selected to packﬁrst if it can be packed into the unpacked subspace.In addition,the longer side of the rect-angle to be packed should be packed along the bottom side of the subspace.It is the disadvantage of the HR algorithm that may have waste space in each layer(See Fig.1).In order to overcome this disadvantage,some layers without waste space areﬁrst considered.Some deﬁnitions are given to clearly describe the idea of the improved algorithm.Deﬁnition1.The reference rectangle is the rectangle that is packedﬁrstly and can form one layer with other rectangles,and its short edge is the height of that layer.Deﬁnition2.The combination layer is the layer that has no waste space,and the rectangles packed into it have the same height as the height of the layer and are spliced together one by one along the direction of W.The sum of the edge length of the rectangles along the W direction is the combination width.From Fig.1to Fig.4,each rectangle at the top of the container is the referee rectangle.Theﬁrst layer in Fig.1 is not a combination layer because it has waste space.The ﬁrst layer in Fig.2is a combination layer because it has no waste space and the spliced rectangles have the same width or length as the height of the layer.Although the second layer in Fig.2has no waste space,it is not a com-bination layer because the two middle rectangles are not spliced along the direction of W.By the deﬁnition of the combination layer,the combination width is W.If some combination layers are found before further computation, then they may decrease the cost of computation because the rectangles already packed into these combination lay-ers will not be considered in thefuture.From the above discussion,it is very important toﬁnd the combination layer.Given a packing ordering,the pro-cedure ofﬁnding the combination layer is given as follows.Find theﬁrst unpacked and unreferenced rectangle as the reference rectangle,and put this rectangle into a two-dimensional array.Then seek downwards from the refer-ence rectangle orderly.If one canﬁnd a rectangle whose length or width is equal to the width of the reference rect-angle,then put this rectangle into the two-dimensional ar-ray,repeat this until a combination layer or no combination layer is found.Repeat the above process until all rectan-gles are packed otherwise no rectangle can be the reference rectangle.In this process,the number of rectangles,which have been packed in the current layers,must be recorded. Finally,the number of all the combination layers must be recorded.The steps of the combination operator can be stated as follows:Combination()RepeatFind theﬁrst unpacked and unreferencedrectangle as the reference rectangle,and put thisrectangle into a two-dimensional array;For i=current position to nIf(the width or the length of rectangle i isequal to the width of the reference rectangle)If(combination width<W)Put the rectangle into the two-dimensionalarray;Else if(combination width=W)Pack all the rectangles of theNo.9ZHANG De-Fu et al.:An Improved Heuristic Recursive Strategy Based on Genetic Algorithm for (913)two-dimensional array on the container;Break;Record the number of all packed rectangles;Until all rectangles are packed or no rectangle is thereference rectangle;Record the number(Num)of all the combination layers;So,the IHR algorithm can be stated as follows:Step1.The combination layers are searched.Step2.Pack the remaining rectangles to the container by HR algorithm.By intuition,the more the number of the combination layers is,the faster the computational speed is.However, it is not always true that more combination layers can ob-tain a better solution.From Fig.1to Fig.4,we know that the best combination layer number is1.Fig.1has no com-bination layer,but layer1,layer3,and layer4waste a little space.Fig.2has one combination layer and is the optimal solution.Fig.3has two combination layers but layer3and layer4waste a little space.Fig.4has three combination layers,but layer4and layer5waste a little space.4GA+IHR4.1Genetic algorithm(GA)GA is a heuristic method used toﬁnd approximate solu-tions to hard optimization problems through application of the principles of evolutionary biology to computer science. It is modeled loosely on the principles of the evolution via natural selection,which use a population of individuals that undergo selection in the presence of variation inducing op-erators such as recombination(crossover)and mutation.In order to run GA,we must be able to create an initial pop-ulation of feasible solutions.The initial population is very important for achieving a good solution.There are many ways to do this based on the form of the problems.The evolution starts from a population of completely random in-dividuals and happens in generations.In each generation, theﬁtness of the whole population is evaluated.Multiple individuals are stochastically selected from the current pop-ulation and are modiﬁed to form a new population,which becomes current population in the next iteration of the algorithm.Further detailed theoretical and practical de-scriptions of genetic algorithm,the interested reader can refer to[27].Combining GA with IHR,the GA+IHR algorithm to solve the strip rectangular packing problem can be stated as follows:GA+IHR()Sort all rectangles by non-increasing ordering of area size;Combination();For i=0to NumInitialization();For j=1to NumberFor k=1to N/2Select two individuals in the parentsrandomly,then crossover with probabilityP c or copy with probability(1−P c)tocreate two middle oﬀspring;Mutate the middle oﬀspring withprobability P m;Compare the parent and the middle oﬀspring,if theﬁtness of the middle oﬀspring is less thanthe parent s,we accept it as new oﬀspring,otherwise we accept it with probability P b oraccept the parent with probability(1−P b);Select the best solution from the parents;Select the worse solution from the newgeneration;Replace the worse solution with the bestsolution;Save the best solution acquired from combinationlayer i to array A;Select the best solution from array A;where Num is the number of all the combination layers; Number is the iteration number of genetic algorithm;N is the number of population;P c=0.8,P m=0.2,but P m will increase as the parent chromosomes become more alike,P b=0.33.Theﬁtness value of genetic algorithm is calculated by HR algorithm.The required container length is the sum of combination height and currentﬁtness value.4.2InitializationThe GA is used to optimize the solution for unpacked rectangles.Theﬁtness value of GA is calculated by HR algorithm.In this paper,a string of integers,which forms an index into the set of rectangles,is used,and then the HR strategy is used to create the sequence of theﬁrst indi-viduals,and the sequence is divided into two equal parts, the two parts of theﬁrst individuals are then permuted to obtain N−1individuals(N is the size of population).The method of the permutation is to produce a point in each range randomly and exchange the position of the point for its neighbor.This initialization method can keep the diver-siﬁcation of each individual in the population.At the same time,it can keep the individual,which has betterﬁtness. From the experiment results,we know that the method has better eﬀect.4.3CrossoverThe role of the crossover operator is to allow the advanta-geous traits to spread throughout the population such that the population as a whole may beneﬁt from this chance dis-covery.The steps of the crossover operator are as follows:1)Choose two individuals Parent1and Parent2from the parents randomly.2)Get the items of individual Child1from Parent1and ly,if the sequence number of the Child1is odd,ﬁnd an item orderly in Parent1until the item is diﬀerent from all the items in Child1,otherwise ﬁnd an item orderly in Parent2until the item is diﬀerent from all the items in Child1.When the number of Child1 is n,a new individual is created successfully.3)Similarly,get the items of individual Child2from Parent2and ly,if the sequence number of the Child2is odd,ﬁnd an item orderly in Par-ent2until the item is diﬀerent from all the items in Child2, otherwiseﬁnd an item orderly in Parent1until the item is diﬀerent from all the items in Child2.When the number of Child2is n,a new individual is created successfully.As an example of crossover,suppose two individuals are already selected:Parent1:53267841Parent2:86517324 According to the steps of the crossover operator,the sequences of the children can be obtained:Child1:58362174Child2:856312744.4MutationIn each individual A,two diﬀerent points are chosen ran-domly,and the sequence within two points is inversed,then914ACTA AUTOMATICA SINICA Vol.33in the appointed iteration step,judge theﬁtness of the new individual B,if theﬁtness of B is less than that of A,B is accepted.As an example of mutation,suppose two mutation points (3and6)are already selected:A:24587136After mutation,we can get the new individualB:24178536Mutation is adaptive,that is,the mutation rate increases as the parent chromosomes become more alike.4.5ReplacementAfter the operations of crossover and mutation,a set of solutions are produced.For keeping the bestﬁtness and quickening the speed of convergence,a best solution is se-lected from the set of solutions,and it is saved to the next generation in each iteration step.5Computational resultsIn order to compare the relative performance of the pre-sented GA+IHR with other published heuristic and meta-heuristic algorithms,several test problems taken from the literature are used.Perhaps the most extensive instances given for this problem are found in[19],where21prob-lem instances are presented in seven diﬀerent sized cate-gories ranging from16to197items.The optimal solu-tions of these21instances are all known.Table1(see next page)presents an overview of the test problem Class1from [19].As we wanted to extensively test our algorithm,other test problems were generated at random.Table2shows an overview of the test problem Class2generated randomly with known optimal solution.The problem Class2can be accessed in[23].In order to verify the performance of GA+IHR,two best meta-heuristic GA+BLF and SA+BLF[23],Bestﬁt[13],and HR[19]are selected.The computational results are listed in Tables3and4.20iterative times are chosen for GA and 80iterative times are chosen in the mutation operation.On this test problem Class1,as listed in Table3,Gap of GA+IHR ranges from0.83to4.44with the average Gap 2.06.The average Gaps of GA+BLF,SA+BLF,Bestﬁt, and HR are4.57,4,5.69,and3.97,respectively.The av-erage Gap of GA+IHR is lower than those of GA+BLF, SA+BLF,Bestﬁt,and HR.And as listed in Table4,the average running time of GA+IHR is also lower than those of GA+BLF and SA+BLF,but is larger than that of Best ﬁt and HR.The packing results can be seen in Fig.5and Fig.6,where L denotes the optimal height.The heights of C11,C12,C13,and C72using GA+IHR are20,21,21,and 241,respectively.GA+IHR canﬁnd the optimal heights for C11,C23,and C32.In order to extensively test the performance of our al-gorithm for randomly generated instances,especially for larger instances,12problem instances ranging from10to 500were generated at random.The computational results are listed in Table5.For such problem,100iterative times are chosen for the GA and10iterative times are chosen in the mutation operation.Although our algorithm can ob-tain a better solution,it needs much time,especially for large problems.So for N12,40iterative times are chosen for the GA and10iterative times are chosen in the muta-tion operation.From Table5,we observe that the running time is acceptable.What is more,the Gap is better thanothers.Fig.5Packed results of C1forGA+IHRFig.6Packed results of C72for GA+IHR6ConclusionsIn this paper,the GA+IHR algorithm for the orthogo-nal stock-cutting problem has been presented.IHR is very simple and intuitive,and can solve the orthogonal stock-cutting problem eﬃciently.GA is an adaptive heuristic search algorithm.It has the capability of global search within the solution space.The idea of combination layers to reduce the number of unpacked rectangles has been used. During the process of iteration search,HR is called repeat-edly to calculate the height of an individual.As we know,ﬁnding the optimal solution is more diﬃcult for the packing problem as increasing the size of problem.But it can be overcome by using the characteristic of GA.The computa-tional results have shown that we can obtain the desirable solutions within acceptable computing time by combining GA with IHR.So GA+IHR can compete with other evolu-tion heuristic algorithms,especially for large test problems, it performs better.So GA+IHR may be of considerable practical value to the rational layout of the rectangular objects in the engineeringﬁelds,such as the wood,glass, and paper industry,the ship building industry,and textile and leather industry.Future work is to further improve the performance of GA+IHR and minimize the inﬂuence of the parameters selection,and extend this algorithm for three-dimensional rectangular packing problems.No.9ZHANG De-Fu et al.:An Improved Heuristic Recursive Strategy Based on Genetic Algorithm for (915)Table1Test problem Class1Problem category Number of items:n Optimal height Object dimensionC1(C11,C12,C13)16(C11,C13),17(C12)2020×20C2(C21,C22,C23)25(C21,C22,C23)1515×40C3(C31,C32,C33)28(C31,C33),29(C32)3030×60C4(C41,C42,C43)49(C41,C42,C43)6060×60C5(C51,C52,C53)73(C51,C52,C53)9090×60C6(C61,C62,C63)97(C61,C62,C63)120120×80C7(C71,C72,C73)196(C71,C73),197(C72)240240×160Table2Test problem Class2Problem category Number of items:n Optimal height Object dimension N1104040×40N2205030×50N3305030×50N4408080×80N550100100×100N66010050×100N77010080×100N88080100×80N910015050×150N1020015070×150N1130015070×150N12500300100×300Table3Gaps of GA+BLF,SA+BLF,Bestﬁt,HR,and GA+IHR for the test problem Class1C1C2C3C4C5C6C7Average GA+BLF4753445 4.57 SA+BLF46533344Bestﬁt11.67 6.79.9 3.87 2.93 2.5 2.23 5.69HR8.33 4.45 6.67 2.22 1.85 2.5 1.8 3.97 GA+IHR 3.33 4.44 2.22 1.67 1.110.830.83 2.06Table4Average running time of GA+BLF,SA+BLF,and GA+HRC1C2C3C4C5C6C7Average GA+BLF 4.619.2213.8359.93165.96396.463581.97604.57 SA+BLF 3.22711.06418.4452.13530.151761.0219274.413107.2 Bestﬁt0.00.00.00.000.0030.0050.0070.005 HR000.030.140.69 2.2136.07 5.59 GA+IHR0.88 1.52 2.249.5630.0465.56426.0476.55Table5Gaps of GA+BLF,SA+BLF,Bestﬁt,HR,and GA+IHR for the test problem Class2n Optimal heightGA+BLF SA+BLF BF Heuristic GA+IHRh Time(s)h Time(s)h Time(s)h Time(s)N1104040 1.02400.2445<0.01450.68 N22050519.2528.1453<0.0154 3.32 N3305052 2.65239.552<0.0151 6.18 N440808312.6838483<0.018313.09 N55010010652.31062281050.0110333.01 N6601001032611033101030.0110250.12 N7701001066711065541070.0110457.04 N8808085114285810840.018231.36 N9100150155443115517151520.01152185.94 N102001501542×10415460661520.021511154.18 N113001501558×1041553×1041520.031513763.17 N125003003134×1053126×1043060.063045864.27 Average Gap(%)-- 3.72- 3.85- 4.35- 3.46-916ACTA AUTOMATICA SINICA Vol.33References1Lodi A,Martello S,Monaci M.Two-dimensional packing problems:a survey.European Journal of Operational Re-search,2002,141(2):241∼2522Dowsland K A,Dowsland W B.Packing problems.European Journal of Operational Research,1992,56(1):2∼143Pisinger D.Heuristics for the container loading problem.European Journal of Operational Research,2002,141(2): 382∼3924Hochbaum D S,Wolfgang M.Approximation schemes for covering and packing problems in image processing and VLSI.Journal of the Association for Computing Machinery, 1985,32(1):130∼1365Leung J Y,Tam T W,Wong C S,Young G H,Chin F Y L.Packing squares into a square.Journal of Parallel and Distributed Computing,1990,10(3):271∼2756Beasley J E.An exact two-dimensional non-guillotine cutting tree search procedure.Operations Research,1985,33(1): 49∼647Hadjiconstantinou E,Christoﬁdes N.An exact algorithm for the orthogonal2D cutting problems using guillotine cuts.European Journal of Operational Research,1995,83(1): 21∼388Gilmore P C,Gomory R E.A linear programming approach to the cutting stock problem(part I).Operational Research, 1961,9:849∼8599Christoﬁdes N,Whitlock C.An algorithm for two-dimensional cutting problems.Operational Research,1977, 25(1):30∼4410Cung V D,HiﬁM,Cun B C.Constrained two-dimensional cutting stock problems a best-ﬁrst branch-and-bound algo-rithm.International Transactions in Operational Research, 2000,7(3):185∼21011Zhang De-Fu,Li Xin.A personiﬁed annealing algorithm for circles packing problem.Acta Automatica Sinica,2005, 31(4):590∼595(in Chinese)12Chazelle B.The bottom-left bin packing heuristic:an ef-ﬁcient implementation.IEEE Transactions on Computers, 1983,32(8):697∼70713Berkey J,Wang P.Two-dimensionalﬁnite bin packing algo-rithms.Journal of the Operational Research Society,1987, 38:423∼42914Kroger B.Guillotineable bin packing:a genetic approach.European Journal of Operational Research,1995,84(3): 645∼66115Jakobs S.On genetic algorithms for the packing of polygons.European Journal of Operational Research,1996,88(1): 165∼18116Liu D,Teng H.An improved BL-algorithm for genetic al-gorithm of the orthogonal packing of rectangles.European Journal of Operational Research,1999,112(2):413∼419 17Dagli C,Poshyanonda P.New approaches to nesting rectan-gular patterns.Journal of Intelligent Manufacturing,1997, 8(3):177∼19018Hopper E,Turton B.A genetic algorithm for a2D indus-trial packing puters and Industrial Engineer-ing,1999,37(1):375∼37819Hopper E,Turton B.An empirical investigation of meta-heuristic and heuristic algorithms for a2D packing problem.European Journal of Operational Research,2001,128(1): 34∼5720Wu Y L,Huang W Q,Lau S C,Wong C K,Young G H.An eﬀective quasi-human based heuristic for solving the rectan-gle packing problem.European Journal of Operational Re-search,2002,141(2):341∼35821HiﬁM,Hallah R.A best-local position procedure-based heuristic for the two-dimensional layout problem.Studia In-formatica Universalis,International Journal on Informatics–Special Issue on Cutting,Packing and Knapsacking Prob-lems,2002,2(1):33∼5622HiﬁM,Hallah R.A hybrid algorithm for the two-dimensional layout problem:the cases of regular and irregular shapes.International Transactions in Operational Research,2003, 10(3):195∼21623Burke E,Kendall G,Whitwell G.A new placement heuristic for the orthogonal stock-cutting problem.Operations Re-search,2004,52(4):655∼67124Zhang D F,Kang Y,Deng S.A new heuristic recursive algo-rithm for the strip rectangular packing puters &Operations Research,2006,33(8):2209∼221725Zhang D F,Deng A S,Kang Y.A hybrid heuristic algo-rithm for the rectangular packing problem.Lecture Notes in Computer Science,2005,3514:783∼79126Zhang D F,Liu Y J,Chen S D,Xie X G.A meta-heuristic algorithm for the strip rectangular packing problem.Lecture Notes in Computer Science,2005,3612:1235∼124127Davis L.Handbook of Genetic Algorithms.New York:Van Nostrand Reinhold,1991ZHANG De-Fu Associate professor atSchool of Information Science and Technol-ogy,Xiamen University.He received hisbachelor and master degrees in computa-tional mathematics from Xiangtan Univer-sity in1996and1999,respectively,and hisPh.D.degree in computer software and itstheory from Huazhong University of Sci-ence and Technology in2002.His researchinterest covers computational intelligence andﬁnancial data mining.Corresponding author of this paper. E-mail:dfzhangl@CHEN Sheng-Da Master student atXiamen University.He received his bach-elor degree from Jimei University in2004.His research interest is computational in-telligence.E-mail:cshengda@LIU Yan-Juan Master student in Xia-men University.She received her bache-lor degree from Shijiazhuang University ofEconomics in2004.Her research interest iscomputational intelligence.E-mail:jjj514@。

Genetic K-Means Algorithm

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 3, JUNE 1999
433
[8] H. H. Rosenbrock, “An automatic method for ﬁnding the greatest or least value of a function,” Comput. J., vol. 3, pp. 175–184, 1960. [9] C. G. Schaefer, “The ARGOT strategy: Adaptive representation genetic optimizer technique,” in Genetic Algorithms Applications: Proc. 2nd Int. Conf., 1987, pp. 50–58. [10] N. N. Schraudolph and R. K. Belew, “Dynamic parameter encoding for genetic algorithms,” Mach. Learn., vol. 9, no. 1, pp. 9–21, 1992. [11] R. J. Streifel, R. von Doenhoff, R. J. Marks II, J. J. Choi, and M. Healy, “Application of genetic algorithms to hydraulic brake system parameter identiﬁcation,” The Boeing Company, Seattle, WA, Doc. D6-81795TN. [12] E. C. K. Tsao, J. C. Bezdek, and N. R. Pal, “Fuzzy Kohonen clustering networks,” Pattern Recognit., vol. 27, no. 5, pp. 757–764, 1994. [13] D. Whitley, K. Mathias, and P. Fitzhorn, “Delta coding: An iterative search strategy for genetic algorithms,” in Proc. 4th Int. Conf. Genetic Algorithms, L. Booker and R. Belew, Eds. San Mateo, CA: Morgan Kauffman, 1991, pp. 77–84. [14] L. A. Zadeh, “Fuzzy sets,” Inf. Contr., vol. 8, pp. 338–353, 1965.

time-cost trade-off in

A genetic algorithm approach for thetime-cost trade-oﬀin PERT networksAmir Azaron *,Cahit Perkgoz,Masatoshi SakawaDepartment of Artiﬁcial Complex Systems Engineering,Graduate School of Engineering,Hiroshima University,Kagamiyama 1-4-1,Higashi-Hiroshima,Hiroshima 739-8527,Japan AbstractWe develop a multi-objective model for the time-cost trade-oﬀproblem in PERT net-works with generalized Erlang distributions of activity durations,using a genetic algo-rithm.The mean duration of each activity is assumed to be a non-increasing function and the direct cost of each activity is assumed to be a non-decreasing function of the amount of resource allocated to it.The decision variables of the model are the allocated resource quantities.The problem is formulated as a multi-objective optimal control problem that involves four conﬂicting objective functions.The objective functions are the project direct cost (to be minimized),the mean of the project completion time (min),the variance of the project completion time (min),and the probability that the project completion time does not exceed a certain threshold (max).It is impossible to solve this problem optimally.Therefore,we apply a ‘‘Genetic Algorithm for Numerical Optimizations of Constrained Problems’’(GENOCOP)to solve this multi-objective problem using a goal attainment technique.Several factorial experiments are performed to identify appropriate genetic algorithm parameters that produce the best results within a given execution time in the three typical cases with diﬀerent conﬁgurations.Finally,we compare the genetic algorithm results against the results of a discrete-time approxima-tion method for solving the original optimal control problem.Ó2004Elsevier Inc.All rights reserved.0096-3003/$-see front matter Ó2004Elsevier Inc.All rights reserved.doi:10.1016/j.amc.2004.10.021*Corresponding author.E-mail address:azaron@msl.sys.hiroshima-u.ac.jp (A.Azaron).Applied Mathematics and Computation 168(2005)1317–13391318 A.Azaron et al./put.168(2005)1317–1339Keywords:Project management and scheduling;Genetic algorithm;Multiple objective program-ming;Optimal control;Design of experiments1.IntroductionSince the late1950s,critical path method(CPM)techniques have become widely recognized as valuable tools for the planning and scheduling of large pro-jects.In a traditional CPM analysis,the major objective is to schedule a project assuming deterministic durations.However,project activities must be scheduled under available resources,such as crew sizes,equipment and materials.The activity duration can be looked upon as a function of resource availability. Moreover,diﬀerent resource combinations have their own costs.Ultimately, the schedule needs to take account of the trade-oﬀbetween project direct cost and project completion time.For example,using more productive equipment or hiring more workers may save time,but the project direct cost could increase.In CPM networks,activity duration is viewed either as a function of cost or as a function of resources committed to it.The well-known time-cost trade-oﬀproblem(TCTP)in CPM networks takes the former view.In the TCTP,the objective is to determine the duration of each activity in order to achieve the minimum total direct and indirect costs of the project.Studies on TCTP have been done using various kinds of cost functions such as linear[1,2],discrete[3], convex[4,5],and concave[6].When the cost functions are arbitrary(still non-increasing),the dynamic pro-gramming(DP)approach was suggested by Robinson[7]and Elmaghraby[8]. Tavares[9]has presented a general model based on the decomposition of the project into a sequence of stages and the optimal solution can be easily computed for each practical problem as it is shown for a real case study.Weglarz[10]stud-ied this problem using optimal control theory and assumed that the processing speed of each activity at time t is a continuous,non-decreasing function of the amount of resource allocated to the activity at that instant of time.This means that time is considered as a continuous variable.Unfortunately,it seems that this approach is not applicable to networks of a reasonable size(>10).Recently,some researchers have adopted computational optimization tech-niques,such as genetic algorithms and simulated annealing to solve TCTP.Feng et al.[11]and Chua et al.[12]proposed models using genetic algorithms and the Pareto front approach to solve construction time-cost trade-oﬀproblems.These models mainly focus on deterministic situations.However,during project implementation,many uncertain variables dynamically aﬀect activity durations,and the costs could also change accordingly.Examples of these variables are weather,space congestion,productivity level,etc.To solve prob-lems of this kind,PERT has been developed to deal with uncertainty in the project completion time.A.Azaron et al./put.168(2005)1317–13391319PERT does not take into account the time-cost trade-oﬀ.Therefore,com-bining the aforementioned concepts to develop a time-cost trade-oﬀmodel under uncertainty would be beneﬁcial to scheduling engineers in forecasting a more realistic project completion time and cost.In this paper,we develop a multi-objective model for the time-cost trade-oﬀproblem in PERT networks,using a genetic algorithm.It is assumed that the activity durations are independent random variables with generalized Erlang dis-tributions.It is also assumed that the amount of resource allocated to each activ-ity is controllable,where the mean duration of each activity is a non-increasing function of this control variable.The direct cost of each activity is also assumed to be a non-decreasing function of the amount of resource allocated to it.The problem is formulated as a multi-objective optimal control problem, where the objective functions are the project direct cost(to be minimized), the mean of the project completion time(min),its variance(min)and the prob-ability that the project completion time does not exceed a given level(max). Then,we apply the goal attainment technique,which is a variation of the goal programming technique,to solve this multi-objective problem.For the problem concerned in this paper,as a general-purpose solution method for non-linear programming problems,in order to consider the non-linearity of problems and to cope with large-scale problems,we apply the revised GENOCOP V,developed by Suzuki[13],which is a direct extension of the genetic algorithm for numerical optimizations of constrained problems (GENOCOP),proposed by Koziel and Michalewicz[14].Three factorial experiments are performed to identify appropriate genetic algorithm parameters that produce the best results within a given execution time in the three typical cases with diﬀerent conﬁgurations.Moreover,an experiment in randomized block design is conducted to study the eﬀects of three diﬀerent methods of solving this problem,including the GA,on the objective function value and on the computational time.The remainder of this paper is organized in the following way.In Section2, we extend the method of Kulkarni and Adlakha[15]to analytically compute the project completion time distribution in PERT networks with generalized Erlang distributions of activity durations.Section3presents the multi-objec-tive resource allocation formulation.In Section4,we explain the revised GENOCOP V.Section5presents the computational experiments,andﬁnally we draw conclusions from these experiments in Section6.2.Project completion time distribution in PERT networksIn this section,we present an analytical method to compute the distribution function of the project completion time in PERT networks,or in fact the distribution function of the longest path from the source to the sink node ofa directed acyclic stochastic network,where the arc lengths or activity dura-tions are mutually independent random variables with generalized Erlang dis-tributions.To do this,we extend the technique of Kulkarni and Adlakha[15], because this method is an analytical one,simple,easy to implement on a com-puter and computationally stable.Let G=(V,A)be a PERT network with set of nodes V={v1,v2,...,v m}and set of activities A={a1,a2,...,a n}.Duration of activity a2A(T a)exhibits a generalized Erlang distribution of order n a and the inﬁnitesimal generator matrix G a as:G a¼Àk a1k a10 (00)0Àk a2k a2 (00):::...::000...Àk anak ana 000 (00)2666666437777775:In this case,T a would be the time until absorption in the absorbing state.An Erlang distribution of order n a is a generalized Erlang distribution withk a1¼k a2¼ÁÁÁ¼k ana .When n a=1,the underlying distribution becomes expo-nential with the parameter k a1.First,we transform the original PERT network into a new one,in which all activity durations have exponential distributions.For constructing this net-work,we use the idea that if the duration of activity a is distributed according to a generalized Erlang distribution of order n a and the inﬁnitesimal generator matrix G a,it can be decomposed to n a exponential series of arcs with theparameters k a1,k a2,...,k ana .Then,we substitute each generalized Erlang activ-ity with n a series of exponential activities with the parameters k a1,k a2,...,k ana .Now,Let G0=(V0,A0)be the transformed network,in which V0and A0rep-resent the sets of nodes and arcs of this transformed network,respectively, where the duration of each activity a2A0is exponential with parameter k a. The source and sink nodes are denoted by s and t,respectively.For a2A0, let a(a)be the starting node of arc a,and b(a)be the ending node of arc a. Deﬁnition1.Let I(v)and O(v)be the sets of arcs ending and starting at node v, respectively,which are deﬁned as follows:IðvÞ¼f a2A0:bðaÞ¼v gðv2V0Þ,ð1ÞOðvÞ¼f a2A0:aðaÞ¼v gðv2V0Þ:ð2ÞDeﬁnition2.If X&V0,such that s2X and t2X¼V0ÀX,then an(s,t)cut is deﬁned as1320 A.Azaron et al./put.168(2005)1317–1339A.Azaron et al./put.168(2005)1317–13391321ðX,XÞ¼f a2A0:aðaÞ2X,bðaÞ2X g:ð3ÞAn(s,t)cutðX,XÞis called an uniformly directed cut(UDC),ifðX,XÞis empty.Example1.Before proceeding,we illustrate the material by an example. Consider the network shown in Fig.1.Clearly,(1,2)is a uniformly directed cut(UDC),because V0is divided into two disjoint subsets X and X,where s2X and t2X.The other UDCs of this network are(2,3),(1,4,6),(3,4,6)and(5,6).Deﬁnition3.Let D=E[F be a uniformly directed cut(UDC)of a network. Then,it is called an admissible2-partition,if I(b(a))X F,for a2F.To illustrate this deﬁnition,consider Example1again.As mentioned, (3,4,6)is a UDC.This cut can be divided into two subsets E and F.For exam-ple,E={4}and F={3,6}.In this case,the cut is an admissible2-partition,be-cause I(b(3))={3,4}X F and also I(b(6))={5,6}X F.However,if E={6} and F={3,4},then the cut is not an admissible2-partition,because I(b(3))={3,4}&F={3,4}.Deﬁnition4.During the project execution and at time t,each activity can be in one of the active,dormant or idle states,which are deﬁned as follows:(i)Active.An activity is active at time t,if it is being executed at time t. (ii)Dormant.An activity is dormant at time t,if it hasﬁnished but there is at least one unﬁnished activity in I(b(a)).If an activity is dormant at time t, then its successor activities in O(b(a))cannot begin.(iii)Idle.An activity is idle at time t,if it is neither active nor dormant at time t.The sets of active and dormant activities are denoted by Y(t)and Z(t),respec-)).tively,and X(t)=(Y(t),Z(tConsider Example 1,again.If activity 3is dormant,it means that this activ-ity has ﬁnished but the next activity,i.e.5,cannot begin because activity 4is still active.Table 1,presents all admissible 2-partition cuts of this network.We use a superscript star to denote a dormant activity.All others are active.E contains all active while F includes all dormant activities.Let S denote the set of all admissible 2-partition cuts of the network,and S ¼S [fð/,/Þg .Note that X (t )=(/,/)implies that Y (t )=/and Z (t )=/,i.e.all activities are idle at time t and hence the project is completed by time t .It is proven that {X (t ),t P 0}is a continuous-time Markov process with state space S ,refer to [15]for details.As mentioned,E and F contain active and dormant activities of a UDC,respectively.When activity a ﬁnishes (with the rate of k a ),and there is at least one unﬁnished activity in I (b (a )),it moves from E to a new dormant activities set,i.e.to F 0.Furthermore,if by ﬁnishing this activity,its succeeding ones,O (b (a )),become active,then this set will also be included in the new E 0,while the elements of I (b (a )),which one of them belongs to E and the other ones be-long to F ,will be deleted from the particular sets.Thus,the elements of the inﬁnitesimal generator matrix Q =[q{(E ,F ),(E 0,F 0)}],(E ,F )and ðE 0,F 0Þ2S ,are calculated as follows:q fðE ;F Þ;ðE 0;F 0Þg ¼k a if a 2E ;I ðb ða ÞÞ&F [f a g ;E 0¼E Àf a g ;F 0¼F [f a g ;ð4Þk a if a 2E ;I ðb ða ÞÞ&F [f a g ;E 0¼ðE Àf a gÞ[O ðb ða ÞÞ;F 0¼F ÀI ðb ða ÞÞ;ð5ÞÀP a 2E k a if E 0¼E ;F 0¼F ;ð6Þ0otherwise :ð7Þ8>>>>>>>>>><>>>>>>>>>>:In Example 1,if we consider E ={1,2},F (/),E 0={2,3}and F 0=(/),then E 0=(E À{1})[O (b (1)),and thus from (5),q{(E ,F ),(E 0,F 0)}=k 1.{X (t ),t P 0}is a ﬁnite-state absorbing continuous-time Markov process.Since q{(/,/),(/,/)}=0,this state would be an absorbing one and obviouslyTable 1All admissible 2-partition cuts of the example network1.(1,2) 5.(1,4*,6)9.(3*,4,6)13.(3,4*,6*)17.(/,/)2.(2,3) 6.(1,4,6*)10.(3,4*,6)14.(5,6)3.(2,3*)7.(1,4*,6*)11.(3,4,6*)15.(5*,6)4.(1,4,6)8.(3,4,6)12.(3*,4,6*)16.(5,6*)1322 A.Azaron et al./put.168(2005)1317–1339A.Azaron et al./put.168(2005)1317–13391323 the other states are transient.Furthermore,we number the states in S such that the Q matrix is an upper triangular matrix.We assume that the states are num-bered1,2,...,N¼j S j.State1is the initial state,namely(O(s),/);and state N is the absorbing state,namely(/,/).Let T represent the length of the longest path in the network,or the project completion time.Clearly,T=min{t>0:X(t)=N/X(0)=1}.Thus,T is the time until{X(t),t P0}gets absorbed in theﬁnal state starting from state1.Chapman–Kolmogorov backward equations can be applied to compute F(t)=P{T6t}.If we deﬁne:P iðtÞ¼P f XðtÞ¼N=Xð0Þ¼i g,i¼1,2,...,N:ð8ÞThen F(t)=P1(t).The system of diﬀerential equations for the vector P(t)=[P1(t), P2(t),...,P N(t)]T is given byP0ðtÞ¼QPðtÞ,ð9ÞPð0Þ¼½0,0,...,1 T:3.Multi-objective resource allocation problemIn this section,we develop a multi-objective model to optimally control the resources allocated to the activities in a PERT network whose activity dura-tions exhibit generalized Erlang distributions,where the mean duration of each activity is a non-increasing function and the direct cost of each activity is a non-decreasing function of the amount of resource allocated to it.We may de-crease the project direct cost,by decreasing the amount of resource allocated to the activities.However,clearly it causes the mean project completion time to be increased,because these objectives are in conﬂict with each other.Conse-quently,an appropriate trade-oﬀbetween the total direct costs and the mean project completion time is required.The variance of the project completion time should also be considered in the model,because when we only focus on the mean time,the resource quantities may be non-optimal if the project com-pletion time substantially varies because of randomness.The probability that the project completion time does not exceed a certain threshold is also impor-tant in many cases and to be considered.Therefore,we have a multi-objective stochastic programming problem.The objective functions are the project direct cost(to be minimized),the mean of project completion time(min),the variance of project completion time(min), and the probability that the project completion time does not exceed a certain threshold(max).The direct cost of activity a 2A is assumed to be a non-decreasing function d a (x a )of the amount of resource x a allocated to it.Therefore,the project direct cost would be equal to P a 2A d a ðx a Þ.The mean duration of activity a 2A ,which is equal to P n a j ¼11k a j ,is assumed to be the non-increasing function g a (x a )of the amount of resource x a allocated to it.Let U a represent the amount of resource available to be allocated to the activity a ,and L a represent the minimum amount of resource required to achieve the activity a .In reality d a (x a )and g a (x a )can be estimated using linear regression.We can collect the sample paired data of d a (x a )and g a (x a )as the dependent variables,for diﬀerent values of x a as the independent variables,from the previous similar activities or using the judgments of the experts in this area.Then,we can esti-mate the parameters of the relevant linear regression model.The mean and the variance of project completion time are given byE ðT Þ¼Z 1ð1ÀP 1ðt ÞÞd t ,ð10ÞVar ðT Þ¼Z10t 2P 01ðt Þd t ÀZ10tP 01ðt Þd t 2,ð11Þwhere P 01ðt Þis the density function of project completion time.The probability that the project completion time does not exceed the given threshold u isP ðT 6u Þ¼P 1ðu Þ:ð12ÞThe inﬁnitesimal generator matrix Q would be a function of the vector k ¼½k a j ;a 2A ,j ¼1,2,...,n a T ,in the optimal control problem.Therefore,the non-linear dynamic model isP 0ðt Þ¼Q ðk ÞP ðt Þ,P i ð0Þ¼08i ¼1,2,...,N À1,P N ðt Þ¼1:ð13ÞAccordingly,the appropriate multi-objective optimal control problem isMin f 1ðx ,k Þ¼X a 2A d a ðx a Þ,Minf 2ðx ,k Þ¼Z 10ð1ÀP 1ðt ÞÞd t ,Minf 3ðx ,k Þ¼Z 10t 2P 01ðt Þd t ÀZ10tP 01ðt Þd t 2,Max f 4ðx ,k Þ¼P 1ðu Þ1324 A.Azaron et al./put.168(2005)1317–1339s :t :P 0ðt Þ¼Q ðk ÞP ðt Þ,P i ð0Þ¼08i ¼1,2,...,N À1,P N ðt Þ¼1,g a ðx a Þ¼X n a j ¼11k a j a 2A ,x a 6U aa 2A ,x a P L aa 2A ,k a j P 0a 2A ,j ¼1,2,...,n a :ð14ÞA possible approach to solving (14)to optimality is to use the Maximum Principle (see [16]for details).For simplicity,consider solving the problem with only one of the objective functions,f 2ðx ,k Þ¼R 10ð1ÀP 1ðt ÞÞd t .Clearly,x a ¼g À1a P n a j ¼11k aj for a 2A .Therefore,we can consider k as the unique control vector of the problem,and ignore the role of x =[x 1,x 2,...,x n ]T as the other independent decision vector.Consider K as the set of allowable controls consisting of all constraints except the constraints representing the dy-namic model (k 2K ),and N -vector l (t )as the adjoint vector function.Then,Hamiltonian function would beH ðl ðt Þ,P ðt Þ,k Þ¼l ðt ÞT Q ðk ÞP ðt Þþ1ÀP 1ðt Þ:ð15ÞNow,we write the adjoint equations and terminal conditions,which areÀl 0ðt ÞT ¼l ðt ÞT Q ðk Þþ½À1,0,...,0 ,l ðT ÞT ¼0,T !1:ð16ÞIf we could compute l (t )from (16),then we would be able to minimize the Hamiltonian function subject to k 2K in order to get the optimal control k *,and solve the problem optimally.Unfortunately,the adjoint equations (16)are dependent on the unknown control vector,k ,and therefore they cannot be solved directly.If we could also minimize the Hamiltonian function (15),subject to k 2K ,for an optimal control function in closed form as k *=f (P *(t ),l *(t )),then we would be able to substitute this into the state equations,P 0(t )=Q (k )ÆP (t ),P (0)=[0,0,...,1]T ,and adjoint equations (16)to get a set of diﬀerential equa-tions,which is a two-point boundary value problem.Unfortunately,we cannot obtain k *by diﬀerentiating H with respect to k ,because the minimum of H A.Azaron et al./put.168(2005)1317–13391325occurs on the boundary of K ,and consequently k *cannot be obtained in a closed form.According to these points,it is impossible to solve the optimal control prob-lem (14),optimally,even in the restricted case of a single objective problem.Relatively few optimal control problems can be solved optimally.Therefore,we apply a genetic algorithm for numerical optimizations of constrained prob-lems (revised GENOCOP V),which is fully described in Section 4,to solve this problem,using a goal attainment method.3.1.Goal attainment methodThis method requires setting up a goal and weight,b j and c j (c j P 0)for j =1,2,3,4,for the four indicated objective functions.The c j relates the relative under-attainment of the b j .For under-attainment of the goals,a smaller c j is associated with the more important objectives.c j ,j =1,2,3,4,are generally normalized so that P 4j ¼1c j ¼1.The appropriate goal attainment formulationto obtain x *isMinz s :t :Xa 2A d a ðx a ÞÀc 1z 6b 1,Z10ð1ÀP 1ðt ÞÞd t Àc 2z 6b 2,Z 10t 2P 01ðt Þd t ÀZ10tP 01ðt Þd t 2Àc 3z 6b 3,P 1ðu Þþc 4z P b 4,P 0ðt Þ¼Q ðk ÞP ðt Þ,P i ð0Þ¼08i ¼1,2,...,N À1,P N ðt Þ¼1,g a ðx a Þ¼X n a j ¼11k a j a 2A ,x a 6U aa 2A ,x a P L aa 2A ,k a j P 0a 2A ,j ¼1,2,...,n a ,z P 0:ð17Þ1326 A.Azaron et al./put.168(2005)1317–1339Lemma 1.If x *is Pareto-optimal,then there exists a c,b pair such that x *is an optimal solution to the optimization problem (17).4.A genetic algorithm for numerical optimizations of constrained problems (revised GENOCOP V)In this section,we use the revised GENOCOP V proposed as a general purpose method for solving non-linear programming problems as deﬁned in (18).Min f ðk Þs :t :g r ðk Þ¼0,r ¼1,2,...,k 1,h r ðk Þ60,r ¼k 1þ1,k 1þ2,...,k ,L j 6k j 6U j ,j ¼1,2,...,l ,ð18Þwhere k is an l dimensional decision vector,g r (k )=0,r =1,2,...,k 1,are k 1equality constraints and h r (k )60,r =k 1+1,k 1+2,...,k ,are k Àk 1inequality constraints.These are assumed to be either linear or non-linear real-values functions.Moreover,L j and U j ,j =1,...,l ,are the lower and upper bounds of the decision variables,respectively.In order to have the same form given in (18),we reformulate the problem (17),by combining the objective functions and the state equations.We also consider a new decision vector k =[k j ;j =1,2,...,m ]T,where m ¼n þP n i ¼1n i ,instead of the original decision vectors x and k ,in the reformulated problem (19).The appropriate min–max problem is obtained as:Min f ðk Þ¼Max f z 1ðk Þ,z 2ðk Þ,z 3ðk Þ,z 4ðk Þgs :t :g r ðk Þ¼0,r ¼1,2,...,n ,L j 6k j 6U j ,j ¼1,2,...,n ,ð19Þwherez 1ðk Þ¼f 1ðx ,k ÞÀb 1c 1,z 2ðk Þ¼f 2ðx ,k ÞÀb 2c 2,z 3ðk Þ¼f 3ðx ,k ÞÀb 3c 3,z4ðkÞ¼b4Àf4ðx,kÞc4,g r ðkÞ¼g rðk rÞÀX n rj¼11k rj¼0,r¼1,2,...,n,j¼1,2,...,n randP0ðtÞ¼QðkÞPðtÞ,Pð0Þ¼½0,0,...,1 T:ð20ÞIt should be noted that in our computer program,P1(t)is obtained by solv-ing the system of diﬀerential equations(20)analytically and then the mean and the variance of project completion time are computed,numerically.The prob-lem(19)does not have the inequality constraints(h r(k)60)of problem(18). The only restriction that we have in this problem is that the elements of k vec-tor(decision variables)are selected between the given lower and upper bounds.We apply the revised GENOCOP V,developed by Suzuki[13],which is a direct extension of the genetic algorithm for numerical optimizations of con-strained problems(GENOCOP),proposed by Koziel and Michalewicz[14]. In GENOCOP V,an initial reference point is generated randomly from indi-viduals satisfying the lower and upper bounds,which is quite diﬃcult in prac-tice.Furthermore,because a new search point is randomly generated on the line segment between a search point and a reference point,the eﬀectiveness and speed of the search may be quite low.The proposed revised GENOCOP V overcomes these drawbacks by generating an initial reference point by min-imizing the sum of squares of the violated non-linear constraints and using a bisection method for generating a new feasible point on the line segment between a search point and a reference point.To be more explicit aboutﬁnding the initial reference point,for some k,Lj6 k j6U j,j=1,...,l,we use the set of violated non-linear equality constraintsI g¼f w j g wð kÞ¼0,w¼1,...,k1gð21aÞand the set of violated non-linear inequality constraintsI h¼f w j h wð kÞ>0,w¼k1þ1,...,k g:ð21bÞAn unconstrained optimization problem is formulated to minimize the sum of squares of violated non-linear constraintsMinXw2I g ðg wðkÞÞ2þXw2I hðh wðkÞÞ2ð21cÞand the optimization problem(21)is solved for obtaining one initial reference point.In the bisection method for generating a new search point,two cases are considered,in which the search points are either feasible or infeasible individuals.If search points are feasible,a new search point is generated on the line seg-ment between a search point and a reference point.If search points are infea-sible,a boundary point is found and a new point is generated on the line segment between the boundary point and a reference point.If the feasible space is not convex,the new point could be infeasible.In this case the generation of a new point is repeated if becomes feasible.putational procedures of revised GENOCOP VIn this section,the genetic algorithm for numerical optimizations of con-strained problems(revised GENOCOP V)is summarized step by step.Step0.Determine the values of the population size P,the total number of generations G,the probability of mutation P m,and the probabilityof crossover P c.Step1.Generate one or more initial reference points by minimizing the sum of squares of violated non-linear constraints.Step2.Generate the initial population consisting of P individuals.Step3.Solve the system of diﬀerential equations in(20)and compute P1(t)for each individual.The solution of the system is found as follows;ﬁrst,the eigenvalues and then the related eigenvectors of the constant coef-ﬁcient matrix Q are found.According to the eigenvectors and theeigenvalues of the system,the solution is found for each individual. Step4.Decode each individual(genotype)in the current population and cal-culate itsﬁtness(phenotype).Step5.Apply the mutation and crossover operations with the probabilities provided in step0.Step6.Generate the new population by applying the reproduction operator, based on the ranking selection.Step7.When the maximum number of iterations is reached,then go to step8.Otherwise,increase the generation number by1and then go to step3. Step8.Stop.putational experimentsTo investigate the performance of the proposed genetic algorithm method (revised GENOCOP V)for the time-cost trade-oﬀproblem in PERT networks, we consider3typical small,medium and large cases with diﬀerent conﬁgura-。

Discovering Accurate and Interesting Classification Rules Using Genetic Algorithm

Discovering Accurate and Interesting Classiﬁcation Rules Using GeneticAlgorithmJanaki Gopalan Reda Alhajj Ken BarkerAbstractDiscovering accurate and interesting classiﬁcation rulesis a signiﬁcant task in the post-processing stage of a datamining(DM)process.Therefore,an optimization problemexists between the accuracy and the interesting metrics forpost-processing rule sets.To achieve a balance,in thispaper,we propose two major post-processing tasks.Intheﬁrst task,we use a genetic algorithm(GA)toﬁnd thebest combination of rules that maximizes the predictiveaccuracy on the sample training set.Thus we obtain themaximized accuracy.In the second task,we rank the rulesby assigning objective rule interestingness(RI)measures(or weights)for the rules in the rule set.Henceforth,we propose a pruning strategy using a GA toﬁnd thebest combination of interesting rules with the maximized(or greater)accuracy.We tested our implementation onthree data sets.The results are very encouraging;theydemonstrate the applicability and effectiveness of ourapproach.Keywords:post-processing,data mining,classiﬁcationrules,rule interestingness,genetic algorithms.1IntroductionData mining is generally deﬁned as the process ofextracting previously unknown knowledge from a givendatabase.A DM process is divided into three stagesnamely,the pre-processing,mining,and the post-processingstages[1,20].The post-processing stage of the DM pro-cess involves interpretation of the discovered knowledge orsome post-processing of this knowledge.An example ofjective boosted hypothesis rule[4].Therefore,the accu-racy of the obtained results are biased by the accuracy with which these weights are obtained.Moreover,these weightsare based on one metric which is the classiﬁcation accuracy of the classiﬁer.In this paper,we propose a pruning strat-egy by extending the idea proposed by Thompson.In our strategy,we use a GA with objective rule interestingness measures(based on Freitas[7])toﬁnd the most interest-ing subset with the performance accuracy of atleast on the sample set(problem space).These measures are based on several objective metrics(including the accuracy metric) to derive interesting as well as accurate rules.Therefore, the resulting rule set from the solution of our GA method is the best combination of accurate interesting classiﬁcation rules.These rules are then tested for their accuracy on the unknown validation set(the solution space).The rest of the paper is organized as follows.Section2 describes the related work in rule-set reﬁnement for classi-ﬁcation rules.Section3discusses the implementation using a GA for this problem.In Section4,we give the experimen-tal results using the GA method.Section5is conclusions and future work.2Related WorkIn this section,we discuss the related work in rule-set reﬁnement for classiﬁcation rules.They are:1)The rule interestingness(RI)principles proposed for classiﬁcation rules;and2)ﬁnding the best set(or subset)of rules from the discovered rule-set.The task of assigning a RI measure is discussedﬁrst,followed by the discussion of deriving the best combination of accurate rules.Methods for the selection of interesting classiﬁcation rules can be divided into subjective and objective methods. Subjective methods are user-driven and domain-dependent. By contrast,objective methods are data-driven and domain-independent.A comprehensive review about subjective as-pects of RI is available[10].Piatetsky-Shapiro[11],Major and Mangano[12],and Kamber and Shinghal[13]propose objective principles for RI to include the rule quality factors of coverage,completeness,and a conﬁdence factor.Freitas (1999)[7]extended the objective RI principles[11,12,13] to include additional factors such as the disjunct size,im-balance of class distributions,attribute interestingness,mis-classiﬁcation costs,and the asymmetric nature of classiﬁca-tion rules.We consider the next problem of pruning rule sets.Pro-dromidis et al.[14]present methods for pruning classiﬁers in a distributed meta-learning system.A pre-training prun-ing is used to select a subset of classiﬁers from an ensem-ble which are then combined by a meta-learned combiner. Margineantu and Deitterich[2]use a backﬁtting algorithm for pruning classiﬁer sets.This involves choosing an ad-ditional classiﬁer to add to a set of classiﬁers by a greedy search and then checking that each of the other classiﬁers in the set cannot be replaced by another to produce a better en-semble.Thompson[4]proposes a GA to prune a classiﬁer ensemble toﬁnd the right combination of classiﬁers with-out over-ﬁtting the training set.The proposed GA uses a real-valued encoding.Each chromosome has real-valued genes,where is the number of classiﬁers in the ensemble. Each gene represents the voting weight of its corresponding classiﬁer calculated using a boosted hypothesis(WVBH) rule.Theﬁtness function consists of measuring the pre-dictive accuracy of the classiﬁer ensemble with the weights proposed by the chromosomes on a hold-out set(different from the training set).Two major conclusions are drawn:1) In a majority of the experiments that were performed with the classiﬁer sets,it was found that a subset of classiﬁers from the original classiﬁer ensemble had a better classiﬁca-tion accuracy.2)The GA method was very efﬁcient inﬁnd-ing the right set of pruned classiﬁers.Moreover,the pruned classiﬁer sets from the GA method have a better classiﬁca-tion accuracy over the pruned classiﬁers sets from earlier work[4].In this paper,we propose a pruning strategy using a GA toﬁnd the best set of interesting and accurate rules by ex-tending the idea proposed by Thompson.In the rest of the paper,weﬁrst discuss our proposed approach that employs GA to address this problem.Finally,we present the experi-mental validations along with future work.3Post-processing Rule Sets Using Genetic AlgorithmsGA’s were introduced by Holland[8],as a general model of an adaptive processes,but subsequently widely exploited as optimizers[8].Basically,a GA can be used for solving problems for which it is possible to construct an objective function(also known asﬁtness function)to estimate how a given representative(solution)ﬁts the considered environ-ment(problem).In general,the main motivation for using GAs in any data mining process is that they perform a global search and cope better with interaction than the greedy rule induction algorithms often used in data mining.Genetic algorithms can be used in the post-processing stage of the DM process.Very little work has been reported in the literature in this area.As reviewed in earlier sections, Thompson[4]proposes a GA to prune a classiﬁer ensemble efﬁciently.We implemented GAKPER,a GA based Knowledge Discovery algorithm for deriving Efﬁcient Rules to achieve our goal.In our implementation,the original dataset is di-vided into a sample set(to train)and a validation set(to test).This task is divided into two parts.In theﬁrst part, a binary encoded GA is used toﬁnd the most accurate sub-set of rules with the best classiﬁcation accuracy on the sample set(problem space).In the second part,a binary en-coded GA is used toﬁnd the most interesting subset with accuracy of at least on the sample set.Finally,the derived accurate interesting rules are tested on the unknown valida-tion set(solution space)for their accuracy.Each of these parts are described in the subsections below.3.1GAKPER ALGORITHM-PART1A binary encoded GA is used to search for the best com-bination of accurate rules.Each chromosome in the popu-lation is a subset of the classiﬁcation rules.The length of the chromosome is the number of rules in the rule set;in that,each gene represents the corresponding classiﬁcation rule.For example,if there are rules in the original rule set,then“1001100111”is a possible chromosome,where theﬁrst,fourth,ﬁfth,eight,ninth,and tenth rules from the rule set are chosen to represent the set of accurate rules.A solution in the phenotype space is only represented by a single chromosome and all possible chromosomes are valid. Hence a1-1mapping exists between the genotype and phe-notype spaces.Theﬁtness functionﬁrst measures the predictive accu-racy of the rules(represented by the chromosome)on the entire sample set.This is achieved as follows.The true class values for all instances in the sample set is stored prior to running the GA.The classes predicted by the rule set repre-senting the chromosome are known.Therefore,to classify the test instances from the sample set,theﬁtness function takes a vote of the rules from the rule set.Thus,to calculate the class of a single test instance,the class predicted by the rules representing the chromosome(based on majority vote) is matched with the original stored class value of the test in-stance.This process is repeated for all the test instances in the sample set.Finally,the best combination of rules that maximizes the predictive accuracy on the sample set(i.e., the number of correctly predicted test instances)is obtained. The maximum accuracy is denoted as.If represents the number of correctly predicted instances by the chromo-some,theﬁtness value of the chromosome is deﬁned as:(1) All these aspects are precisely encoded and implemented into the GA and all the chromosomes(potential solutions) should be awarded or punished according to the criteria stated above during the process of evolution.The outcome of several evolutions modeled by this GA generates the right set of accurate rules.The GAKPER algorithm(Part I)is presented below: ALGORITHM:GAKPER-IInput:Classification rule set Output:Set of accurate rulesMethod:1)Search for the right set of accuraterules using the GA Method as follows: 1a)Randomly create an initial populationof potential accurate rules.1b)Iteratively perform the following substeps on the population until thetermination criterion in the GA issatisfied:a)FITNESS:Evaluate fitness f(x)of eachchromosome in the populationb)NEW POPULATIONb1)SELECTION:Based on f(x)b2)RECOMBINATION:2-point Cross-over ofchromosomesb3)MUTATION:Mutate chromosomesb4)ACCEPTATION:Reject or accept new onec)REPLACE:Replace old with newpopulation in new generationd)TEST:Test problem criterium(Number of generations is>100)2)After the termination criterion issatisfied,the best chromosome in thepopulation produced during the run isdesignated as the required combinationof the accurate rules.3.2GAKPER ALGORITHM-PART IIA binary encoded GA is used to search for the best com-bination of interesting accurate rules.The representation for the chromosome is the same as described in Part1.Each chromosome in the population is a subset of the classiﬁca-tion rules.The length of the chromosome is the number of rules in the rule set,in that,each gene represents the corre-sponding classiﬁcation rule.The maximum predictive accuracy is known from the result of the GAKPER algorithm-Part I.The RI measure (or weight)proposed by Freitas is assigned to all the rules. Theﬁtness function optimizes the weights toﬁnd the best combination of interesting rules with a classiﬁcation accu-racy of at least.It is important to note that accuracy of the rules when tested on the test instances(from the sample set) are based on the weighted majority vote.Thus,a rule with a higher RI measure ranks higher in classiﬁcation over a rule with a lower RI measure.The weights of the rules rep-resenting a chromosome are denoted as. If the accuracy of the rules is greater than(or equal)to, then theﬁtness value of the chromosome is the sum of the weights,otherwise the chromosome is given a defaultﬁt-ness value.The intuitive idea of choosing this default value is:1)the value has to be greater than zero to continue the GA runs in subsequent generations,and2)since theﬁtness criteria is not satisﬁed by the respective chromosome,an ar-bitrary value is chosen to punish the leastﬁt chromosome. Theﬁtness value of the chromosome is deﬁned as:If(2)All these aspects are precisely encoded and implemented into the GA and all the chromosomes(potential solutions) should be awarded or punished according to the criteria stated above during the process of evolution.The outcome of several evolutions modeled by this GA generates the right set of accurate interesting rules.The GAKPER algorithm(Part II)is presented below:ALGORITHM:GAKPER-IIInput:Classification rule setOutput:Set of accurate interesting rules Method: (1)Assign weights for the rules in the input ruleset.(2)Search for the right set of accurateinteresting rules using the GA Method asfollows:2a)Randomly create an initial population of potential accurate interesting rules.2b)Iteratively perform the following sub steps on the population until the terminationcriterion in the GA is satisfied:a)FITNESS:Evaluate fitness f(x)of eachchromosome in the populationb)NEW POPULATIONb1)SELECTION:Based on f(x)b2)RECOMBINATION:2-point Cross-over ofchromosomesb3)MUTATION:Mutate chromosomesb4)ACCEPTATION:Reject or accept new onec)REPLACE:Replace old with new population:the new generationd)TEST:Test problem criterium(Number of generations is>100) (3)After the termination criterion is satisfied,the best chromosome in the population produced during the run is designated as the rightcombination of the accurate interesting rules. The approach is implemented as a standard GA writ-ten in C,similar to Grefenstette’s GENESIS program; Baker’s SUS selection algorithm[21]is employed;2-point crossover is maintained at60%and mutation is very low; and selection is based on proportionalﬁtness.It is impor-tant to note that this approach optimizes the predictive ac-curacy and the interesting measures of the rule set on the entire sample set,which is the problem space.4Results and DiscussionsAll the tests have been conducted using a single Proces-sor,Intel(R)Xeon(TM)UNIX machine with CPU power of 2.80GHz and cache size of512KB.The GAKPER algorithm implementation is tested onﬁve datasets.The data-splitting,that is,dividing the dataset into the sample set and validation set is performed using ran-dom sampling technique.The classiﬁcation rules are ob-tained using the sample set.Recall that,to prune the rule set to derive the set of interesting accurate rules,the fol-lowing tests are performed.The GAKPER(part I)is used to derive the subset of rules that maximizes the classiﬁca-tion accuracy on the sample set.To derive the most inter-esting subset,a RI measure or weight(based on Freitas)is assigned to the rules.GAKPER(part II)is used again,this time,to maximize the interestingness measure of the rules whose classiﬁcation accuracy on the sample set is at least .The accuracy of the derived interesting accurate rules (on the validation set)from this approach is compared with: 1)pruning the rule set using GA without assigning initial weights for the rules,and2)without any pruning methods, using the entire rule set.The results for theﬁve datasets are presented below.Data Set1:Breast Cancer Data SetThis is a real data set obtained from Tom Baker Cancer Cen-ter,Calgary,Alberta,Canada.The original dataset consists of records and attributes.Each record represents follow-up data for one breast cancer case.Breast cancer “recurred”in some of these patients after the initial occur-rence.Hence,each patient is classiﬁed as“recurrent”or “non-recurrent”,depending on his or her status.With re-spect to classiﬁcation of the dataset,there are2classes: 1)Recurrent Patients,and2)Non Recurrent Patients.The original dataset is divided into a sample set and a validation set using the random sampling technique. The sample set has instances whereas the validation set has instances.The GA based approaches,i.e., GAKPER-I and GAKPER-II,is used(on the sample set) toﬁnd the best combination of accurate interesting rules. These rules are tested on the unknown validation set for ac-curacy.The GA parameters are:,-,,,and the is.30 experiments were performed with the GA approaches and every GA experiment was run for generations in both approaches.The post-processing results on the sample set and the validation sets are presented in Table1and Table2 respectively.Table1.Post-processing Results using Dif-ferent Approaches on the Sample Set for Dataset1WithoutWeightsCorrectly Predicted(in%)80Incorrectly Predicted(in%)10Unknown(in%)10Table2.Post-processing Results using Dif-ferent Approaches on the Validation Set for Dataset1WithoutWeightsCorrectly Predicted(in%)77Incorrectly Predicted(in%)8Unknown(in%)15With Weights Entire Rule Set94940066Table4.Post-processing Results using Dif-ferent Approaches on the Validation Set for Dataset2WithoutWeightsCorrectly Predicted(in%)91Incorrectly Predicted(in%)3Unknown(in%)6using the different approaches on the sample set(the prob-lem space)for Dataset2is presented.Table4presents the accuracy results of the discovered rule sets(using the differ-ent approaches)when tested on the validation set(the solu-tion space)for Dataset2.For this dataset,we found that the accuracy of the result on the unknown validation set using different approaches is the same(as presented in theﬁrst, second and third columns in Table4).This is also depicted graphically in Fig2.Data Set3:US-CENSUS-DATASETThis data is the USCensus1990raw data set obtained from the UCI repository[19].The data was collected as part of the1990census.The original dataset consisted of cate-gorical attributes and instances.The dataset used here consists of instances and attributes derived from the original USCensus1990rawdataset.It contains classes namely,iClass=0,iClass=5,and iClass=1.Each class refers to the native country of the candidate under con-sideration.The original dataset is divided into a sample set and a validation set using the random sampling technique.The sample set has instances while the validation set has instances.The GA based approaches,i.e.,GAKPER-Iand GAKPER-II,is used(on the sample set)toﬁnd the best combination of accurate interesting rules.These rules are tested on the unknown validation set for accuracy.Finally, the same GA parameters enumerated earlier for Dataset1 is used here.The post-processing results are presented in Table5and Table6.In Table5,the accuracy of the rule sets,while pruning, using the different approaches on the sample set(the prob-lem space)for Dataset3is presented.Table6presents the accuracy results of the discovered rule sets(using the dif-ferent approaches)when tested on the validation set(the solution space)for Dataset3.It is very important to ob-serve that,for this dataset,the accuracy of the result on the unknown validation set using our approach(as presented in theﬁrst column in Table6)is much greater than the accu-racy of the results using the traditional approaches(as pre-sented in second and third columns in Table6).This is also depicted graphically in Fig3.Table5.Post-processing Results using Differ-ent Approaches on the Sample the US census dataWithoutWeightsCorrectly Predicted(in%)66Incorrectly Predicted(in%)34Unknown(in%)0WithWeightsEntireRule Set7560254000From the results,it can be observed that:1)in majority of the tests performed,a subset of rules pruned from the original set has a better performance accuracy;and2)it is possible to derive the most interesting subset with a higher classiﬁcation accuracy as compared to the original rule set. Therefore,the GA with its inherent robust search strategies is most suitable for the post-processing problem considered in this paper.5Conclusions and Future WorkIn this paper,we propose and implement a GA based methodology to derive interesting and accurate classiﬁca-tion rules from a dataset.The fundamental goal of any data mining model is to derive interesting rules.At the same time,accuracy is a key issue.Therefore,in the post-processing component,the problem of deriving interesting accurate rules is addressed. The earlier works in this area addressed the following two problems independently:1)the problem ofﬁnding the ac-curate set(or subset)of rules using pruning strategies to theoriginal rule set[4];and2)the problem of assigning sub-jective or objective RI measures to the rules to determine their interestingness[7,11,12,13].In our work,a new methodology is proposed byﬁrst assigning an objective RI measure based on Freitas[7]to the rules and using pruning strategies with a GA to search for the right set of interesting accurate rules.An alternative approach worth investigating is toﬁnd the interesting accurate rules using multi-objective genetic al-gorithms.The goal is to optimize two parameters,namely, the interesting and the accuracy metrics for the classiﬁca-tion rules simultaneously.References[1]Dorain Pyle,“Data Preparation For Data Mining”,Morgan Kaufmann,1999.[2]D.D.Margineantu and T.G.Dietterich,“PruningAdaptive Boosting”.Proceedings of the14Inter-national Conference on Machine Learning,San Fran-cisco,CA,pp.211-218,1997.[3]J.R.Quinlan,“Boosting First Order Learning”.Pro-ceedings of the14International Conference on Ma-chine Learning,1997.[4]S.Thompson,“Genetic Algorithms as Postprocessorsfor Data Mining”.Data Mining with Evolutionary Algorithms:Research Directions-Papers from the AAAI Workshop”,pp18-22,1999.[5]R.E.Schapire and Y.Freund and P.Bartlett and W.S.Lee,“Boosting the Margin:a New Explanation for the Effectiveness of V oting Methods”.Machine Learning:Proceedings of the14International Con-ference,pp.322-330,1997.[6]P.Domingos,“Knowledge Acquisition from Exam-ples via Multiple Models”.Machine Learning:Pro-ceedings of the14International Conference,pp98-106,1997.[7]A.A.Freitas,“On Rule Interestingness Measures”.Advances in Evolutionary Computation,Knowledge-Based Systems,12,1999.[8]D.E.Goldberg,“Genetic Algorithms in Search,Op-timization and Machine Learning”.Addison Wesley, Longman Publishing Co.,Inc.,Boston,MA,1989. [9]A.A.Freitas,“A Survey of Evolutionary Algo-rithms for Data Mining and Knowledge Discovery”.Advances in Evolutionary Computation,Springer-Verlag,2001.[10]B.Liu,W.Hsu,and Yiming Ma,“Integrating Classiﬁ-cation and Association Rule Mining”.Proc.of KDD, pp.80-86,1998.[11]G.Piatetsky-Shapiro,“Discovery,Analysis,and Pre-sentation of Strong Rules”.Knowledge Discovery in Databases,pp.229-248,1991.[12]J.A.Major and J.J.Mangano,“Selecting AmongRules Induced from a Hurricane Database”.Proceed-ings of AAI-93,Workshop on Knowledge Discovery in Databases,pp.28-44,1993.[13]M.Kamber and R.Shinghal,“Evaluating the Inter-estingness of Characteristic Rules”.Proceedings of the2International Conference on KDD”,pp.28-44, 1993.[14]A.L.Prodromidis and S.Stolfo,“Pruning Classiﬁersin a Distributed Meta-Learning System”.Proceedings of the KDD”,pp.151-160,1998.[15]C.M.Fonseca and P.J.Fleming,“Genetic Algo-rithms for Multi-Objective Optimization:Formula-tion,Discussion and Generalization”.Proceedings of the Fifth International Conference on Genetic Algo-rithms,pp.93-100,1985.[16]R.J.Bayardo,“Brute-Force Mining of High-Conﬁdence Classiﬁcation Rules”.Proc.of KDD, pp.123-126,1997.C.M.Fonseca and P.J.Fleming [17]Han,“CMAR:Accurate and Efﬁcient ClassiﬁcationBased on Multiple Class-Association Rules”.Proc.of IEEE-ICDM,pp.369-376,2001.[18]R.Agrawal,and R.Srikant,“Fast algorithms for min-ing association rules,”Proc.of VLDB,Santiago,Chile, September1994.[19]C.L.Blake,and C.J.Merz,“UCI Repository of ma-chine learning databases”.University of California, Department of Information and Computer Science, Irvine,CA,1998.[20]I.H.Witten and E.Frank,“Data Mining:Practical Ma-chine Learning Tools and Techniques with Java Imple-mentations”.Morgan Kaufmann,October1999. [21]J.E.Baker,Reducing Bias&Inefﬁciency in the Selec-tion Algorithm.Proc.of the International Conference on Genetic Algorithms,pp.14-21,1987.。

GeneticAlgorithms Optimization Method 最优化方法经典教材课件

• After meiotic division 2 gametes appear in the process
• In reproduction two gametes conjugate to a zygote wich will become the new individual
• Hence genetic information is shared between the parents in order to create new offspring
• GA’s are based on Darwin’s theory of evolution
• History of GA’s • Evolutionary computing evolved in the 1960’s. • GA’s were created by John Holland in the mid-70’s.
• There are more individuals born than can survive, so there is a continuous struggle for life.
• Individuals with an advantage have a greater chance for survive: survival of the fittest.
2020/7/3
Biological Background (7) – Natural selection • The origin of species: “Preservation of favourable variations and rejection of unfavourable variations.”
2020/7/3

1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。

Using a Genetic Algorithm for Optimizing the Functional Decompositionof Multiple-Valued FunctionsT. Kalganova P, E. Kochergov k, N. Strechen kP Belarussian State University of Informatics and Radioelectronics,Phone: +375172 49 19 81, Fax: +375172 49 51 06,E-Mail: jack@expert.belpak.minsk.by or pottosina@risq.belcaf.minsk.byk Institute on Problems of Criminology, Criminalistics and Forensic ExpertisePhone: +375172 23 95 54; Fax: +375172 26 72 79,E-Mail: jack@expert.belpak.minsk.byABSTRACT:The genetic algorithm which determines the good functional decomposition of multiple-valued logic functions is presented. The algorithm expands the range of searching for a best decomposition, providing the optimal column multiplicity. The possible solutions are evaluated using the gain of decomposition for multiple-valued function.1.INTRODUCTIONIn recent years, increasing attention has been given to multiple-valued (MV) circuits. This is because of their ability to reduce the number of interconnections in complicated digital circuits, to increase the information content per unit chip area, and to increase the speed of information transmission [Shousha 1995].The combinational circuits based on the look-up table type FPGA (field programmable gate array), MV PLA (multiple-valued programmable logic array), FPD (field programmable devices), PAL (programmable logic array), T-gate (multiplexer) and etc. can be designed by using the functional decomposition technique. This technique is to break the many variable function into several subfunctions in terms of decreasing the number of variables. Each of these subfunctions can be implemented independently, and are relatively easier to design.The classical functional decomposition method contains the two main phase:1. Finding the best partition of X that provides the minimum number of subfunctions, where X is the set of inputvariables.2. Encoding the subfunctions using the minimum number of r-valued variablesThe optimization of these phases leads to decreasing the size of combinational circuit. In this paper we search the best partition of X. Note that the success of functional decomposition depends on the partition of X. Moreover, the decomposition methods for r-valued functions and their systems have been poorly investigated. Therefore the problem to find the best partition of X is actual and requires the further researches especially for r-valued functions and their systems.The search for a fast algorithm to determine the best decomposition of switching (binary) functions is studies by T. Sasao [Sasao 1993]. He presented an algorithm based on testing the necessary condition of function decomposability, using reduced ordered binary decision diagrams. By performing of this algorithm we can intend the best partition of X with minimum column multiplicity. Note that this algorithm does not examine dependence of the variable permutations on the column multiplicity for given partition of X. This drawback can be removed by using genetic algorithm (GA).The objective of this research is to use such optimization technique as GA to search the best partition of X for a r-valued function. The fitness function determines the best partition for specific variable’s permutation.The multiple-terminal multiple-valued decision diagrams (MTMDD) are used instead of decomposition tables. It allows to investigate (n-1) different partition of X for one MTMDD.Thus this paper addressed the problem to find the best decomposition without considering the all decomposition variants. For this purpose the genetic algorithm is applied. The distinctive feature of this algorithm is that the possible solutions are coded by real numbers. It permits to increase the GA convergence.The rest of paper is organized as follows. In section 2 we start with describing the basic notations and definitions. We then discussed the genetic algorithm and its properties. The experimental results is explained in Section 4. the Section 5 draws a conclusion.2.DEFINITIONS AND NOTATIONSWe will use the following notation throughout.X : the set of n r -valued variables {x 1,x 2,...,x n }, where n is the number of elements of X .|X|: the number of elements of the set X .X 1 : the bound set of variables, |X 1| = n 1;X 2: the free set of variables, |X 2| = n 2;D e f i n i t i o n 1. Let X is the set of r -valued variables. {X 1, X 2} is a partition of X when X 1∩X 2=∅ and X 1∪X 2=X.D e f i n i t i o n 2. An r -valued function f(X) is said to have a generalized disjunctive decomposition with respect to X 1 if there exist r -valued functions h 1,h 2,...,h k and g such that (Figure 2-1)f(X)=g(h 1(X 1), h 2(X 1),..., h k (X 1), X 2)(1)The decomposition is said to be trivial if n 1 is 1 or n . A function that has a nontrivial generalized or simple disjunctive decomposition is said to be decomposable .D e f i n i t i o n 3. The number of different column patterns in the decomposition table is called as a column multiplicity of the decomposition, and denoted by µ.Note that the column multiplicity defines the type of disjunctive decomposition. An r -valued function f(X) is said to have a simple disjunctive decomposition if µ is less or equal to r [Abugharieh, Lee 1995; Kalganova 1996] and a generalized disjunctive decomposition if µ is less or equal to r n 1, where n 1 is the number of elements in set X 1[Kalganova 1996].D e f i n i t i o n 4. Let {X 1, X 2} be a partition of X . Let n 1=|X 1| and n 2=|X 2|. γ=min (r n 1, r r n 2)/µ is the gain of thedecomposition for a given partition.3. GENETIC ALGORITHM FOR r -VALUED VARIABLE PARTITION PROBLEMGAs provide a stochastic search technique inspired by principles of natural genetics and evolution [Holland 1975].They operate through a simulated evolution process on a population of string structures, each of which encoded a possible solution in the search space. Each string structures is called chromosome and a set of chromosomes is referred as a population.Evolution of population involved two basic steps: (1) a selection mechanism that implements a survival of the fittest strategy and (2) genetic recombination of the selected high-fittest chromosomes to produce offspring for the new generation. Recombination is effected through the two genetic operators of crossover and mutation.GAs seem to provide a highly effective balance between exploitation (using known good partial solutions) and exploration (trying new solutions) [Miller, Luchian; Bradbeer; Barclay 1994]. First we see how a solution can be mapped to a string. Then we discuss the basic operators applied in searching for an optimum partition of X .The variable partition problem leads itself to a straightforward representation by means of GA techniques. Indeed, as required for a classical GA approach, a partition of X is easily encoded by a string of integer numbers of length n, where n is the number of variables on which the r -valued function depends. But we solve the coding problem by nontraditional way. Each gene b i of chromosome b =(b 1,...,b i ,...,b n ) takes real value from the domain. The less value of b i corresponds to the variable with the less index, where b i is the gene of chromosome. For an n variable r -valued function the searchspace (the set of all possible partitions of X ) hasn-1∑ C i ni=12 elements.3.1 FITNESS FUNCTION The fitness function depends on the gain index γ. This is obtained by generalizing the decomposition algorithm based on the reduced ordered binary decision diagrams analysis to a r -valued function. Let us consider main principles to find the generalized disjunctive decomposition for an n -variable r -valued function. The algorithm consists of two phases:k h 1…h kn 2H X 1n 1X 2g(h 1(X 1), ..., h k (X 1), X 2)G Figure 2-1: Circuit diagram of f(X)=g(h 1(X 1), h 2(X 1),..., h k (X 1), X 2).1) A MTMDD for given function is generated.2) This MTMDD is tested for good decomposition.T h e o r e m. Let {X1, X2} be a partition of X. Suppose that the reduced ordered MTMDD for f(X) is partitioned into two blocks such as shown in Figure 3-1. Let t be the number of the nodes in the lower block that are adjacent to the boundary of two blocks, and µ be the column multiplicity of decomposition f(X)=g(h1(X1),..., h k(X1),X2)). Then t=µ.Each MTMDD is tested for a good decomposition using the gain γ the specific properties of which is shown in Table 1. As we can see the partition with maximal gain is useful. In our GA the gain index γ determines the evaluation function.T a b l e1 THE SPECIFIC PROPERTIES OF GAIN γ.Crossover is the main genetic operator. It operates on three randomly selected chromosomes (parents) and generates offsprings. Thus, new three offspring are obtained and are replaced the parents in the breeding pool.The sense of crossover operator is preserved: the obtained chromosomes have better values then parents. This property is essential convention of GA convergence that is to get the optimal solution corresponding to the maximum of fitness function. If this particular feature is not taken into consideration, than the search of optimal solution will bedepends on the mutation, i.e. GA would be degenerate intorandom search.Crossover creates new chromosomes (offsprings) on the basis of three initial ones (parents). Using crossover operator,calculation of new chromosomes is fulfilled by analogy with simplex-method [Bersini, Seront 1992]. Three chromosomes are randomly selected from current population at the chromosomes swapping. The number of parents is determined by α coefficient called as crossover speed.Assume that certain current population where each chromosome has two genes exists. Select three chromosomescalled as parent by random way. Chromosomes from this triadhave to be arranged in accordance with values of fitness-function. Interpret initial chromosomes as points A, B, C onsurface (Figure 3-2), where values of genes are values of coordinate. In accordance with crossover operator the forthpoint (D) is derived by building the median CE from "the worst" point (C) and continuation of median ED. First length of ED is equal CE. ED is further decreased at reaching the global extreme.If the number of genes is equal n , than described transforms are executed on the basis of mathematical expression:p i =x i +y i 2+δ(x i +y i 2-z i )i=1, n where x i , y i , z i are the i -th genes of three initial chromosomes arranged in decreasing order of fitness-function. They are correspond to the points A, B, C on Figure 3-2, p i isthe i -th gene of new chromosomes performed by crossover (it corresponds to the point D), δ is coefficient called as crossover force.The analysis of local extreme is envisaged in given algorithm. The sense of this procedure is following. If fitness-function of chromosomes from new generation does not increase, than a conclusion about reaching the global extreme or about falling into“trap” of local extreme is made. In order to go out from local extreme it needs to use the additional procedures, for example “shaking” of population [Michalewicz 1992]. However such approach requires high temporal expenditures (it needs to modeling the evolution for hundred populations). If improvement of fitness-function is lacked, two procedures are used. First one is to use “intense” mutation of current population. In given case, parents as well as offsprings mutate.Second one is based on changing the selection operator. Here new population is formed from the fittest chromosomes as well as from chromosomes changing by random way. Experimental investigations show that suggested procedure allows to go out from the local extreme and to continue the search of global extreme.3.3 MUTATIONMutation operator transforms a chromosome by means of randomly changing one or some gene’s values from the following range [0..1]. Mutation is used for a part of chromosomes from new population. The number of mutated chromosomes is defined by β coefficient called as mutation speed.Size of population is increased at the crossover and mutation execution. Here the current population contains both parents and offsptings. Decreasing the size of population is executed by selection operator that forms new population by choice of the fittest chromosomes.4. EXPERIMENTAL RESULSThe purpose of experimental investigations is to sign the minimum partition of X for r -valued function.The GA success depends on the correct choose of the size of populations crossover speed and the number of population. The structure of GA had been suggested in [Kochergov 1996]. Thus GA parameters are selected as in[Kochergov 1996], taking into consideration the computer characteristics such as the size of main memory and CPU time.The time needed to evaluate the fitness function is investigated too.interpretation of crossover Q 7LPH VFigure 4-1: Dependence the execution time on the number of X 1, n 1 (n=6, r=4)Figure 4-1 illustrate the dependence the evaluation time on thenumber of variables in X 1. S0, S1, S2 and S3 are randomlygenerated r -valued functions. The time complexity isexponentially decreased when we take into consideration thesize of X 1. Comparison the execution time of the proposedalgorithm with standard decomposition as shown in Figure 4-2.Let us assume that the time complexity of standarddecomposition method is 100%. Figure 4-2 illustrate that thetime needed to calculate the fitness function by proposedmethod is decreased as well as the size of X 1 is increased.5. CONCLUSION In this paper, we have presented a new technique to optimize the partition of X which leads to reducing the size ofsynthesized r -valued combinational circuit. The distinctivefeature of suggested algorithm is that a fast algorithm to find the best partition is applied as one of the part of GA. A possible solutions are encoded by real number instead of integer. This aspect provides the GA convergence. Each chromosome is evaluated using a fast algorithm to find the best position. So, two techniques are incorporated that allows to determine the global optimum. We applied a permutation of input variables to obtain the best results because the result of similar algorithms without permutation does not gives the global optimum. Experiments on large randomly generated circuits and other benchmarks obtained show that the procedure is very effective in finding the best partition.The future investigations direct into generalized distinguished approach to the system of r -valued functions. It permits to research the performance of new approach on other standardized large output benchmark problem.6. REFERENCESAbugharieh, S.B.; S.C. Lee 1995. A Fast Algorithm for the Disjunctive Decomposition of m -Valued Functions. Part 1.The Decomposition Algorithm. IEEE Transactions on Computers, pp. 118-125.Bersini, H.; Seront G. 1992. In search of a good crossover between evolution and optimization, in Parallel Problem Solving from Nature 2, Manderick, R. Manner Eds., Amsterdam: Elsevier, pp. 479-488.Holland J.H. 1975. Adaptation in Natural and Artificial Systems. Ann Arbor, Michigan: University of Michigan Press.Kalganova, T. 1996. Functional Decomposition Methods for Multiple-valued Logic Functions and its System. 3rd International Conference on Application of Computer Systems, Szczecin, Poland, pp.75-82.Kochergov, E.1996. Optimization of Decision Acceptance in Person Identification System due to Handwritten. 2nd International Conference on the New Information Technologies in Education, Minsk,Belarus,Vol.1, pp.371-380.Michalewicz, Z. 1992. Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlar, Berlin,Heidelberg.Miller, J.; Luchian H.; Bradbeer P.; Barclay P. 1994. Using the Genetic Algorithm to Optimizing Fixed Polarity Reed-Muller Expansions of Boolean Functions. Int. J. Electronics, Vol. 76, No. 4, pp. 601-609Sasao, T. 1993. FPGA Design by Generalized Functional Decomposition. Logic Synthesis and Optimization. Kyushu institute of technology, Iizuka, Japan.Shousha, A.H. 1995. Switched - Current CMOS Ternary Logic Circuits. Int. J. Electronics, Vol. 79, No. 5, pp. 617-625.Q 7LPH Figure 4-2: Comparison the execution time of theproposed algorithm with standard decomposition (100%) (r=4)。