Document Sample

Outlines • Overview of MCDA – General definition – MCDM process – MCDA methods • Evaluation of WBA – Quality Attribute Relationships – Aggregation by Choquet Integral – Implementation – Case study and results What is MCDA? • Aims to give the decision-maker some tools in order to enable him to advance in solving a decision problem where several – often contradictory-points of view must be taken into account. What is MCDM? • Highly structured, disciplined and formal approach to decision making • evaluating the alternatives in the given set A against the set C of criteria • Aggregating the individual evaluations to produce global evaluation • Could be used for selection the best possible alternatives or for ranking the alternatives MCDM Process Set of Alternatives Set of Criteria C1, C2,………Cn A1 x11……..………x1n A2 x21……..………x2n . . Am xn1……..………xmn Weights wi / Importance of Criteria Aggregation Measure Overall worth of an alternative Ai Evaluation of MCDA methods • Criteria – interdependence, completeness, non-linear preferences • Weights – transparency of process, type of weights, meaning • Solution finding procedure – ranking, option • Project constraints – cost, time Evaluation of MCDA methods • Structure of problem solving process – stakeholder participant, tool for learning transparency, actors communication • Data Situation – Type of data - qualitative or quantitative – Risk/uncertainties – probabilities, thresholds, fuzzy numbers, sensitive analysis – Data processing amount – Non-substitutability Evaluation of WBA • Quality of web site is hard to evaluate – Consists of multiple criteria to be measured • Simple weighted average cannot be used to summaries the various quality measurements into a single score. • Inability to account for dependency among the quality criterion. • Tend to construct independent criteria, or criteria that are supposed to be so – Causing some bias effect in evaluation WBA Evaluation Approaches Single criteria • usability aspects(Collins, 1996; Stefani & Xenos, 2001; Hassan & Li, 2005), • content and structure (Bauer & Scharl, 2000). • accessibility (Vigo et al., 2007) WBA Evaluation Approaches Multi-criteria • WEBQEM (Olsina et al., 1999) • EWAM (Schubert & Selz, 1998) • WebQual (Barnes and Vidgen, 2002) • WAI (Miranda et al., 2006) • FQT4Web (Davoli et al., 2005) ISO/IEC 9126 Evaluation Process Stated or implied needs ISO 9126 & other technical info Requirement Managerial definition Quality requirement requirement Quality specification Requirement Definition Metric Rating level Assessment Preparation Software Selection definition criteria Development definition Products Measured Measurement value Evaluation Rated Rating value Result (acceptable or unacceptable) Assessment Quality Model e-commerce e-learning e-education e-government etc. APPLICATION DOMAIN C H A R A C T E RI S T IC S Q U A L IT Y Functionality Reliability Usability Efficiency Maintainability Portability understandability S U B C H A R A C T E R IS T IC S suitability maturity learnability time behaviour Analysability Adaptability fault tolerance operability resource changeability installability accuracy coexistence recoverability attractiveness utilisation stability interoperability expliciteness efficiency replaceability availability testability security degradability customisability compliance manageability portability traceability reliability clarity reusability compliance functionality compliance helpfulness maintainability compliance user-friendliness compliance usability compliance Indicators, scales and preferred values Quality Attributes for WBA • Define software product qualities as a hierarchy of factors, criteria and metrics. • Quality factor represents behavioral characteristics of the system • Quality criterion is an attribute of a quality factor that is related to software production and design • Quality metrics is a measure that captures some aspect of a quality criterion. Factor A is split up into Overall Quality Score three criteria a1, a2, and a3. Criteria a1 with the weight 4 is considered four times as important as criteria a2 and twice as important as Factor A Factor B Factor C criteria a3. Similarly, we can set different weight for each Criteria a1, Criteria a2, Criteria a3, factor to indicate its weight 4 weight 1 weight 2 importance. Definition of Quality Attributes Name Description Functionality The capability of the Web site to provide functions and properties which meet stated and implied needs when the site is used under specified conditions Usability The capability of the Web site to be understood, learned and liked by the user, when used under specified conditions Reliability The capability of the Web site to maintain a specified level of performance when used under specified conditions. Efficiency The capability of the site to provide appropriate performance, relative to the amount of resource used, under stated conditions Maintainability The capability of the site to be modified. Modifications may include corrections, improvements or adaptation of the site to changes in environments, and in requirements and functional specifications Portability The capability of the site to be transferred from one environment to another Quality Attributes Relationships Three types of relationships • Positive, i.e. a good value of one attribute result in a good value of the other (synergistic goals). – Relationships definitions: If characteristics A is enhanced, then characteristics B is likely to be enhanced (+) • Negative, i.e. a good value of one attribute result in a bad value of the other (conflicting goals). – Relationships definitions: If characteristics A is enhanced, then characteristics B is likely to be degraded (-) • Independent, i.e. the attributes do not affect each other. – Relationships definitions: If characteristics A is enhanced, then characteristics B is unlikely to be affected (0) Interrelationships between quality factors (Perry, 1987) Relationship Chart (Gillies, 1997) Techniques to explore the relationships Ref Attributes Purpose Techniques used Correctness, Reliability Integrity, Usability To study the relations of different Survey -questionnaire [8, 9] Efficiency, Maintainability quality goals attribute in developing Testability, Flexibility software Portability. Reusability Interoperability Performance To address the importance of [10] Adaptability design decision made during Case Study - Interview Maintainability software development Usability [11] Time to market To increase the understanding of Research Literature and Reliability, Usability software quality attributes and Survey –structured interview Correctness, Portability their relations [12] Quality attributes in 3 different To merge different view and Discussion (meeting and perspectives: management, discuss the relationships between offline discussion) developer and user perspective the quality attributes Quality Attributes Relationships for WBA What is Aggregation? • method of combining several numerical values into a single one, so that the result of aggregation takes into account in a given manner all the individual values Aggregation issues • use simple weighted average approach • methods are not transparent • assume independency • the choice of summarization method somehow should depend on the certain parameters – E.g. the kind of importance parameters (weights) and the type of dependency and interaction • the definition of the quality factors and their relationships must be clearly specified • Quasi-arithmetic means (arithmetic, geometric, harmonic, etc.) – Not stable under linear transformation and consider criteria as non interacting • Median – Typical ordinal operator – defined the middle value of the ordered list • Weighted minimum and maximum – Possible to increase one of the weights without having any change in the result • Ordered weighted averaging operators – Can express vague quantifiers 23 Properties of an aggregation operator mathematical properties – Properties of extreme values – Idempotence – Continuity – Monotonicity – Commutativity – Decomposability – Stability under the same positive linear transformation Properties of an aggregation operator behavioural properties – express the decisional behavior, interaction between criteria, interpretability of the parameters and weights on the arguments Aggregation by fuzzy integral • Different methods have been developed according to – type of information to be aggregated and – the properties have to be satisfied. 26 Definition 1: A fuzzy measure on the set X of criteria is a set function : Ƥ (X) [0,1], satisfying the following axioms i. ()=0, (X)=1. ii. A B X implies (A) (B) (A) represent the weight of importance of the set of criteria A. Additive : if (AB) = (A) + (B); A B= Superadditive: if (AB) (A) + (B); A B= Subadditive if (AB) (A) + (B); A B= If a fuzzy measure is additive, then it suffices to define n 27 coefficients (weights) ({ I}), … ({ n}) Definition 2: Let be a fuzzy measure on X. The choquet integral of a function ƒ : (X) [0,1] with respect to is defined by n C (f(x1),…. f(xn)):= (f(x(i)) - f(x(i-1))) (A(i) ) i=1 ƒ ((0)) = 0 •Fuzzy integral model does not need to assume independency •Fuzzy integral of ƒ with respect to gives the overall evaluation of an alternative 28 Importance and interaction of criteria • Problem of evaluation of student with respect to three subjects: mathematics (M), Physics (P) and literature (L). • By weighted sum (3 , 3, 2) result: 29 Solved by fuzzy measure and the choquet integral 1. Scientific subjects are more important than literature; ({M}) = ({P}) =0.45; ({L}) = 0.3 2. M and P are redundant, ({M, P}) = 0.5 < 0.45 + 0.45 3. Students equally good at scientific subjects and literature, ({L, M}) = 0.9 > 0.45 + 0.3 ({L, P}) = 0.9 > 0.45 + 0.3 4. ()=0, ({M, P, L})=1 30 Result by applying fuzzy measure: * The initial ratio of weight (3, 3, 2) is kept unchanged 31 • Number of coefficients grows exponentially with the number of criteria to be aggregated. • 3 approaches (to reduce the number of coefficients) 1. Identification based on semantics – Importance of criteria – Interaction between criteria – Symmetric criteria – Veto effects 2. Identification based learning data – Minimization of squared error – Constraint satisfaction 32 3. Combining semantics and learning Proposed solution • Apply 2-additive Choquet integral • provide the information about the relationships among criteria (redundancy or support among criteria) and the preference among alternatives • Derive fuzzy measures by constraint satisfaction Explore relationships • Techniques to explore how the different attributes are related to each other: – Experience Based Approach – Mathematical Modeling – Statistical Technique (Correlation Analysis) • measures the strength of a linear relationship among different quality factors • The main result of a correlation is called the correlation coefficient (r) Correlation Result Implementation of Choquet Integral 1. Definition of the initial preferences. 2. Convert into Choquet integral form 3. Identify threshold values. 4. If solution exists, calculate the Choquet integral, Shapley value and Interaction indices Define preference thresholds A partial weak order A overA (ranking of the webs), A partial weak order N over (ranking of the importance N of the quality factor), Quantitative intuitions about the relative importance of some quality factor A partial weak order P over the set of pairs of quality factor (ranking of interactions), Intuitions about the type and the magnitude of the interaction between some quality factor, The behavior of some quality factor as veto or favor, Etc. Convert into Choquet integral form Preferences Choquet Integral Ranking of x A x' C (u ( x )) C (u ( x ' )) C alternatives x ~ A x' C C (u ( x )) C (u ( x ' )) C Ranking of i N j (i ) ( j ) Sh criteria i ~N j Sh (i ) ( j ) Sh (weights) Ranking of pairs ij P kl I (ij ) I (kl) I of criteria ij ~ P kl I I (ij ) I ( kl) I (interactions) Sign of some Range of a I (ij ) b , a, b 1,1 interactions interactions Complementary or I (ij ) m or Redundant I (ij ) m ; m [0,1] Define preference thresholds • Three preference thresholds C, Sh & I have to be determined before the aggregation take part. • Range of : 0 to 1 • no rule to fix the , we need to compare the solutions obtain with different value of . • Once the solution exist, Choquet integral will be calculated Calculate the Choquet integral For 2-additive fuzzy measure, we have for any KX: n ( K ) ai xi a ij xi x j i 1 {i , j } X with xi 1 if iK, xi 0 otherwise. We deduce that i ai for all i, and ij ai a j aij i j aij . Calculate the Shapley value n 1 vi k 0 k ( k x \ i ,| K | k iK K ) n = total number of criteria, with k = number of elements in a sub-set (n k 1)!k! k n! Shapley index can be interpreted as a kind of average value of the contribution of element i, individual criteria, alone in all coalitions. Summation of these Shapley values for a given set of elements would represent the importance of the complete set Calculate the Interaction Index n2 I ij k ( ijK iK jK K ) k 0 k x \{i , j },| K | k With (n k 2)!k! 1 k (n 1)! n 2 k (n 1) The interaction index Iij can be interpreted as a kind of average value of the added value given by putting i and j together, all coalitions being considered. When Iij is positive (resp. negative), then the interaction is said to be positive (resp. negative). Case Study • Perform on 3 types of WBA – Academic – E-commerce – Museum • Four quality factor were evaluated – Usability,Functionality, Reliability, Efficiency • Each has different preference, importance and interaction Result for academic website Table 1. Academic websites data Univ Websites Usability (U) Functionality(F) Reliability(R) Efficiency(E) A1 76.18 61.84 60.40 69.09 A2 51.01 50.39 87.62 52.03 A3 80.08 48.49 90.86 76.11 A4 57.71 38.99 88.70 53.12 A5 71.93 82.04 83.05 85.99 A6 60.94 71.12 63.61 69.47 Table 2. Relative importances for each quality attribute Relative U F R E importance (Weight) 0.3 0.3 0.2 0.2 Table 3. Interaction between quality attribute U F R U F - R + + E - - + Table 4. Interaction preorder constraint matrix U F R U F 1 I (UF ) I R I I (UR ) 1 I I ( FR ) 1 1 I (UE ) I 1 I ( FE ) I I I ( RE ) 1 E Each row corresponding to a constraint of forma I (ij ) b ,a, b 1,1 . Table 5. The Möbius values LP MV Threshold {} 0.000000 0.000000 C= 1, {U} 0.366704 0.321697 Sh = 0.1, {F} 0.326805 0.285376 I =0.1, {R} 0.000001 0.094367 {E} 0.277765 0.298559 {U, F} -0.226804 -0.100000 {U, R} 0.137865 0.100000 {U, E} -0.100000 -0.100000 {F, R} 0.217664 0.100000 {F, E} -0.100000 -0.100000 {R, E} 0.100000 0.100000 Table 6. The Shapley values U F R E LP 0.2722348 0.2722348 0.2277652 0.2277652 MV 0.2716970 0.2353761 0.2443674 0.2485595 Table 7. The interaction indices for LP U F R U F -0.2268038 R 0.1378647 0.2176637 E -0.1000000 -0.1000000 0.1000000 Table 8. The interaction indices for MV U F R U F -0.1000000 R 0.1000000 0.1000000 E -0.1000000 -0.1000000 0.1000000 Table 9. Overall Scores Global evaluation obtained by Univ Websites WA LP MV A1 67.304 67.73138 67.32480 A2 58.350 51.26041 54.75639 A3 71.965 72.10125 74.05597 A4 57.374 52.36041 55.79382 A5 79.999 81.44097 81.17426 A6 66.234 66.63138 66.32480 Summary(1) LP MV Möbius values Very extreme value Lead to unique and does not solution necessarily lead to a unique solution Shapley value Sometimes Mostly compatible incompatible with the with the initial relative initial relative importance importance Interaction indices Satisfy the constraints Satisfy the constraints Global evaluation Leads to more Closer to the simple dispersed values weighted arithmetic mean Summary(2) Aggregation approaches Logic Arithmetic Weighted Scoring Mean Average Preference Choquet Integral (AM) (WA) (LSP) (CI) Importance / / / Veto / / Favour / / Interaction / Additive / / / / Non-additive / / Comparison with other approaches Global evaluation obtained by Choquet Integral Choquet Integral Univ Websites AM WA LSP (Interaction) (No Interaction) Rank Score Rank Score Rank Score Rank Score Rank Score Rank A1 3 66.88 3 67.3 3 66.91 3 67.32 3 67.08 3 A2 6 60.26 5 58.35 5 56.55 5 54.76 6 60.16 5 A3 2 73.89 2 71.97 2 69.61 2 74.06 2 73.18 2 A4 5 59.63 6 57.37 6 54.46 6 55.8 5 59.16 6 A5 1 80.75 1 80 1 79.76 1 81.17 1 80.66 1 A6 4 66.29 4 66.23 4 66.05 4 66.32 4 66.09 4 MIN-MAX 59.63-80.75 57.37-80 54.46-79.76 54.76-81.17 59.16-80.66 STD DEV 8.141 8.507 9.214 10.250 8.133 Conclusion • Aggregation by Choquet integral can be alternated if there is interaction exist between quality factors. • The proposed approach can be applied for non-interactive criteria as well. If there is no interaction between the criteria, then the fuzzy measure will be additive measures. • Results show that the global evaluation obtained is compatible with the weighted average method. Future works • The evaluation of WBA which cater the dynamic changes of the quality factors. – Behavior (Preferences, importance,interaction, etc.) can be change continuously. • Investigate more than 2 quality attribute interactions

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 23 |

posted: | 6/26/2012 |

language: | Latin |

pages: | 55 |

OTHER DOCS BY hcj

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.