基于层次分析法的软件体系结构的测量与评价

Analytic Hierarchy Process basedSoftware Architecture Measurement and EvaluationLi Wenhong Yu YongKey Laboratory of Software Engineering of Yunnan Province Key Laboratory of Software Engineering of Yunnan ProvinceYunnan University Yunnan UniversityKunming ,650091, Yunnan, China Kunming ,650091, Yunnan, China258067969@ yuy1219@Abstract：Software architecture has recently become a hot spot in software engineering research. The design of software architecture is a critical success factor for software systems.Software architecture design, similar to the design of a building, needs to use system engineering.Once the software architecture is established, we should also analyze and evaluate it. The purpose of such analysis and evaluation is to identify potential risks in the design of the architecture and thus helping developers to make decisions.In this paper, the Analytic Hierarchy Process is introduced into the evaluation and decision making of software architectures, so that developers can select the right software architecture based on the functional properties and non-functional attributes to achieve the most distinctions.At the same time, according to the basic steps and methods, Analytic Hierarchy Process model is set up to determine the judgment of hierarchy and consistency check. Finally, the process of software architecture evaluation and decision-makingis proposed.Software architecture; Analytic Hierarchy Process; Measurement; EvaluationI.I NTRODUCTIONSoftware design, similar to the design of a building, needs to use systems engineering. The structure of the building should be carefully designed in its entirety before construction takes place. Likewise, the design of the software architecture is a critical factor for the success of the software system. Software architecture provides the important standard for software engineers. At a high and abstract level, it provides software engineers a better way to understand the software and therefore leads to new approaches to develop larger and more complex software systems [1].A well designed architecture is a critical factor for the success of the software. Historical experience attests that selection of a good architecture and construction rule is essential in the design stage of a software project. In engineering projects, such as tall building construction, there are many accumulated natural rules and experiences for designers to follow suit. Software development, however, has been characterized as the masterpiece attributed to an individual’s creativity [1].Software architecture design provides importantguidance in software development: Firstly, it provides thebasis for software development. Software architecture represents a shared high-level abstraction of the system [2]. Various parties can use the architecture as a foundation for communication, leading eventually to consensus. Various interest parties (client, project leader, designer, programmer, testing, maintenance, etc.) involved in software development often emphasize on different aspects of the system. For example, end users may care about usability and reliability; the client may care about budget and schedule; the project leader focuses more on the independence and interaction of developers provided that project is on budget and on time; developers may concern more about the strategy to satisfy all the requirements, so on and so forth. Central to all interest parties lies on the software architecture, as it coordinates fully the interests and proposals of all parties and controls the functionality of the future system [8].Secondly, software architecture provides future developers guidance and constraints for feasible future implementations. It greatly affects software quality assurance. Well designed software architecture warrants the software project's long-term stability and reliability [9]. Consequently, well designed architecture helps the development of software and provides quality control. On the other hand, an imperfect architecture is doomed from the beginning and impairs the quality of the software. To a large extend, selection of the architecture determines whether the system satisfies the requirements for quality. Aspects of software quality include operational characteristics such as software functionality, performance, security and reliability. They also include a variety of other characteristics such as maintainability, adaptability, portability and reusability: these cannot be measured by directly observing system operations.In summary, software architecture evaluation and selection becomes the most critical stage in software development, because it determines the ultimate success or failure of the software project.II.A N AHP-BASED S OFTWARE A RCHITECTURE M EASUREMENTAND E VALUATION P ROPOSALThe uncertainty of software architecture lies in the following aspects, such as: Does the functionality satisfy the requirements of the users? Is it secure? Will it be on budget and on time? Are the developers able to work independently, provided that the project is on budget and on time? As for the project leader, in assessing and selecting software architecture, the decision maker most wants the following scenario: choose the right software architecture that achieves2010 2nd Asia-Pacific Conference on Information ProcessingAPCIP2010overall optimality covering both the functional aspects and non-functional aspects (performance, security, reliability, portability, reusability, maintainability, etc.), as well as satisfying the requirements of the majority of all interest parties.Therefore, in evaluating the architecture, the project leader needs to assess the system using multiple criteria such as functionality, performance, security, reliability, portability, reusability, maintainability, etc. That is to say, to achieve the goal of selecting the architecture is to solve a large-scale system that consists of multi-objective, multi-level, a complicated structure and multi-factors. We need a multi-objective decision analysis method that quantifies the decision maker’s experience; integrates quantity and quality analysis; that sorts and selects alternatives. Analytic Hierarchy Process (AHP) is exactly the efficient framework to address the above problems [3].III. Basic Idea of AHPA. The Basics of AHPAnalytic Hierarchy Process (AHP) was invented by an American scholar T. L. Saaty in the seventies of the twentieth century. It is a multi-objective decision making framework [4]. It formalizes mathematically the decision maker’s decision making process in evaluating options. The basic idea is to ask the decision maker to decompose a goal into multi-levels and multi-factors. At each level and sub-level, various factors go through pair-wise comparisons to obtain weights and sub-weights. These weights then form the basis of the optimal selection [6].Key to AHP is its mathematical formalization and systemization of a human’s decision making process, making it compelling to decision makers. It requires only limited quantitative information, but demands the decision maker a solid grasp of the nature of the problem, factors involved and relationships among these factors.The following are the main steps of AHP [3, 4]:(1) Decompose the problem into multiple levels and factors. Develop a hierarchical framework.(2) Construct the pair-wise comparison matrix for all levels in the hierarchy.(3) Calculate weights of the factors for all levels in the hierarchy.(4) Rank all the alternatives; thus providing the basis for scientific decision-making.B. AHP-based Software Architecture Decision AnalysisUsing three alternative architecture systems as an example, we illustrate how to apply AHP.1) Develop the hierarchical frameworkFirst, the decision maker needs to understand the scope of the problem, factors involved and their relationships, and the goal. Next, the decision maker develops the hierarchical framework. Based on the analysis of the aspects of software architectures, the decision maker determines the following four criteria:a) Reusability [F1]: Reusability is a hot topic in software engineering. Software reuse can reduce development cost and cycle, also increase software quality.b) Modifiability [F2]: When in use, due to all kinds of reasons, software needs to be constantly modified and improved. Modifiability measures the degree of easiness or difficulty of modifying the software.c) Testability [F3]: Software testing becomes more difficult and expensive as software systems become increasingly large and complex. Software architecture that supports software testing can better detect errors and fix them. It can also easily assemble temporary codes and modules.d) Functionality [F4]: This criterion refers to various uncertain factors that satisfy user requirements during software development.Once these criteria are chosen, the hierarchical framework is developed, as shown in Figure 1.Figure 1. A hierarchical model for the selection of software architecture2) Construct the judgment matrixOnce the hierarchical framework is developed, the decision maker needs to compare or judge the relative importance of various items at the lower levels to the corresponding item in the level immediately above, i.e., the weights. Specifically, based on the actual value of each item, and his/her subjective evaluation from field experience, the decision maker performs item-to-item pair-wise comparison, quantifying each judgment using a nine-scale as shown in TableI.TABLE IS CALE FOR PAIR-WISE COMPARISONS (TAKE THE MEAN VALUE IF IN-BETWEEN)A vs.B Extremely Very Important Slightly EqualImportant Important ImportantRelative 9 7 5 3 1 IntensityThis results a pair-wise comparison matrix as shown in Table II.3) Priority ranking and the consistency test a) Priority rankingPriority ranking is to determine and rank the relative importance of various items at the lower levels to the corresponding item in the level immediately above. The decision maker can compute the eigenvector (or priority vector) of the judgment matrix using matrix theory. The weight in the priority vector can be sorted. There are two approximating methods to compute the eigenvector: the square root method and the integral method. Both provide comparable results that meet the accuracy and precision requirements.b) Perform the consistency testConsistency refers to the logic consistency in thedecision maker’s judgment. For example, if A is much more important than C, B is slightly more important than C, then obviously A should be more important than B to maintain logical consistency in the judgment, otherwise the judgment is inconsistent and contradicts each other. The above nine-scale metric helps the decision maker quantify his/her thinking, therefore helps the construction of the comparison matrix. Only when the comparison matrices are consistent, an AHP-based conclusion (i.e., eigenvector) remains plausible. Nevertheless, due to problem complexity, diversity and unavoidable judgment bias of the decision maker, it is very difficult to guarantee full consistency of each comparison matrix, even with the help of the nine-scale metric. Therefore, to guarantee the plausibility of AHP-based conclusions, it is necessary to test the consistency of the matrices constructed. Such test is based on matrix theory. The procedure is as follows. Firstly, calculate the maximum eigenvalue λmax, then calculate the consistency index CI = (λmax - n)/(n -1). If CI = 0, the comparison matrix is fully consistent, test is over. Secondly, if the CI ≠ 0, then the random consistency ratio CR = CI/RI needs to be calculated. Here RI refers to the average random consistency index, which can be found in a reference table. If CR < 0.1, then the consistency of the comparison matrix and the priority vector are considered acceptable. Otherwise, the decision maker needs to adjust the values of elements in the matrix. The detailed calculation is omittedhere. Back to our problem of selecting the best out of three software architectures, results of priority ranking andconsistency testing are shown in Table III, Table Ⅳ, Table Ⅴ and Table Ⅵ, corresponding to each criterion. Results suggest that they have all passed the consistency checking. TABLE ⅡC RITERIA L EVEL J UDGMENT M ATRIXModifiability F25 16 ¼ 0.2518SA Overall Evaluation F1 F2 F3 F4 Priority VectorReusability F1 1 1/5 3 1/9 0.0774 Testability F3 1/3 1/6 1 1/8 0.0439 Functionality F4 9 4 8 1 0.6269TABLE IIIC RITERION R EUSABILITY F1 P RIORITY V ECTOR AND C ONSISTENCY T ESTINGλ ｍａｘ =3.0037， CI=0.00185，CR=0.0036<0.1C 1/5 1/3 1 0.1095B 1/2 1 3 0.3090 Reusability F1 A BC Priority VectorA 1 2 5 0.5816TABLE ⅣC RITERION M ODIFIABILITY F2 P RIORITY V ECTOR ANDC ONSISTENCY T ESTING λ ｍａｘ =2.9999， CI<0，CR<0.1C 1 3 1 0. 4286B 1/3 1 1/3 0.1428 Modifiability F2A B C Priority VectorA 1 3 1 0.4286TABLE ⅤC RITERION T ESTABILITY F3 P RIORITY V ECTOR ANDC ONSISTENCY T ESTING λ ｍａｘ =3， CI<0，CR<0.1C 1/2 1/2 1 0.2B 1 1 2 0.4 Testability F3 A BC Priority VectorA 1 1 2 0.44) Overall priority rankingThe overall priority vector is to determine the relative importance (or weight) of each factor in a level to the top level. To compute the overall priority of a given level, the decision maker needs to multiply the priority vector for the level immediately above by the priority vector of the current level, following the top down sequence. Consistency tests for all levels need to be conducted as well, also following the sequence from the higher level to lower level. Results of the overall priority vector of three software architectures are shown in table Ⅶ. Overall, software architecture B has the best evaluation with a value of 0.4968, higher than A and C, consequently, software architecture B is the optimal choice.Ⅳ C ONCLUSIONSSoftware architecture has recently become a hot spot in software engineering research. Further, it becomes the key technology in the development of large-scale software systems and products. The design of software architecture is a critical success factor for software systems. It sets an importantstandard for software engineers. At a high and abstract level, it provides software engineers a better way to understand the software and therefore leads to new approaches to develop larger and more complex software systems.Once the software architecture is established, we should also analyze and evaluate it. The purpose of such analysis and evaluation is to identify potential risks in the design of the architecture and thus helping developers to make decisions. This paper applies the analytic hierarchy process (AHP)principle and method to the evaluation and decision making of software architectures, allowing developers to select the right software architecture that meets their needs and achieves overall optimality account for both the functional aspects and non-functional aspects (performance, security, reliability, portability, reusability, maintainability, etc.), while satisfying the requirements of the majority of all interested parties.TABLE ⅥF UNCTIONALITY F4 P RIORITY V ECTOR ANDC ONSISTENCY T ESTINGλ ｍａｘ =3.006， CI=0.003，CR=0.006<0.1C 3 1/3 1 0. 2426B 7 1 3 0.6694 Functionality F4 A BC Priority VectorA 1 1/7 1/3 0.088In summary, this article proposes an AHP-based evaluation model for selecting software architectures. Itdemonstrates the basic procedure, including how to construct the pair-wise comparison matrices, calculate priority vectors, and perform the consistency testing.R EFERENCES[1] Sun, Changai; Jin, Maozhong; Liu, Chao. Overview of Software Architecture Research. Software Journal (J), 2002, 13 (07):1228-1237.[2] Zhang, Yousheng. Software Architecture (M). Beijing: Tsinghua University Press, 2004.[3] Yang, Zhiming. Analysis of Several Common Software Architecture Models. Computer Engineering and Design (J), 2004, 25 (8):1326-1328.[4] [U.S.] T. L. Saaty (author). Xu, Shubai et al. (translator). Analytic Hierarchy Process (AHP) [M]. Beijing: Coal Industry Press, 1988.[5] Wei, Qun; Xiong, Zhang; Zhao, Fang. Software Architecture Development Method and Application (f). Computer Engineering and Design, 2003, 24 (4):77-80.[6] Xu, Shubai. Practical Decision Making Method: AHP Principles, Tianjin: Tianjin University Press, 1988.[7] Pan, Jinping. Software System Development Technology, Xi'an: Xi'an University of Electronic Science and Technology Press, 1993.TABLE ⅦO VERALL P RIORITY V ECTORCriteria F1 F2 F3 F4 Overall Priority Vector Weights 0.0774 0.2518 0.0439 0.6269A 0.5816 0.4286 0.4 0.088 0.2256B 0.3090 0.1428 0.4 0.6694 0.4968C 0.1095 0.4286 0.2 0.2426 0.2775[8] Ben, Kerong (translator). F. Buschmann, R. Meunier, et al. (author). Pattern-Oriented Software Architecture Volume 1: A System of Patterns [M]. Beijing: Mechanical Industry Press, 2003.[9] Zhao, Huiqun; Wang, Guoren et al. Software Architecture Performance Evaluation Research [J]. Computer Science, 2003, 30 (02):144-147.[10] Wang, Xiaoxia; Song, Yu et al. Software Architecture Performance Evaluation Research [J]. Computer Engineering and Applications, 2003, 39 (9): 88-90.。