Product quality evaluation system based on AHP fuzzy comprehensive evaluation
Xi Xi, Qiuli Qin
School of Economics and Management, Beijing Jiaotong University (China)
Received: October 2012
Accepted: March 2013
Xi, X., & Qin, Q. (2013). Product quality evaluation system based on AHP fuzzy comprehensive evaluation. Journal of Industrial Engineering and Management, 6(1), 356366. http://dx.doi.org/10.3926/jiem.685

Abstract:
Purpose: To reduce the subjective prejudice and uncertainty in evaluating product quality.
Design/methodology/approach: AHP method is used to analyze the structure of product quality evaluation problem and determine weights for evaluation criteria. After structure judge matrix, sequencing calculation and concordance examination, evaluation methods such as fuzzy synthesis evaluation are used to calculate the integrated quality evaluation result of each product.
Findings: A new model is proposed by comprehensively using AHP method, weighted comprehensive evaluation and fuzzy comprehensive evaluation. A practical example of a product has been used to illustrate the theoretical qualitative proposed evaluation model.
Practical implications: The result of this research offers a new method for the enterprises production quality management.
Originality/value: Using AHP fuzzy comprehensive evaluation method in building product quality evaluation system.
Keywords: AHP (Analytic Hierarchy Process), product quality, evaluation criteria, empirical analysis

1. Introduction
As the gradually improving of specialization level, more and more manufacturing enterprises start to purchase basic products parts manufactured by suppliers instead of producing all parts of the product by themselves. Only with high quality parts offered by suppliers, can the quality of the downstream products be guaranteed. However, because of the diverse manufacturing product varieties and the product evaluation standards, to evaluate the quality of products in a scientific way became a very difficult and complicated work. Hence, how to use scientific evaluation method to evaluate the comprehensive quality products has become a key problem of the enterprise production quality management. Bidding method and Delphi method are the most popular methods to select suppliers in Chinese enterprises at present (Shenghai, Hua & Ningsheng, 2005). These methods are qualitative, and, however, lack quantitative factors and are influenced by subjective factors greatly. Product quality can be divided into material quality, operation quality, structure quality, and many other quality aspects and each aspect can be subdivided into more specific criteria which are difficult to get scientific judgment results only through subjective judgment.
AHP is a simple way to make decisions for complex and fuzzy problems, especially the problems that cannot be quantitatively analyzed completely. Rather than prescribing a correct decision, the AHP helps decision makers find one that best suits their goal and their understanding of the problem (Shenghai et al., 2005). It provides a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions (Juntian, 2008). And fuzzy comprehensive evaluation method is a synthetical assessment method that applies fuzzy mathematical principles to evaluate things and phenomenon affected by variety of factors. A model that integrates AHP method with evaluation methods like fuzzy comprehensive evaluation method will be most likely to help in solving the problem.
This research is aimed at the characteristics of manufacturing products. It applies AHP evaluation, weighted comprehensive evaluation and fuzzy comprehensive evaluation to the process of product quality comprehensive evaluation to design a product quality evaluation model which converts the subjective judgment into scientific decisionmaking result. It offers a new method for the enterprises production quality management.
2. Analytic hierarchy process
Quantitative analysis and qualitative analysis are widely used in research. They are chosen according to the data collected and the knowledge of users in the actual application. Both the methods have played important roles in the development of social science. As a combination of qualitative analysis and quantitative analysis, fuzzy comprehensive evaluation has been widely used in the engineering technology, economic management and social activity. At present, one of the difficulties of fuzzy comprehensive evaluation research is how to convert a multi criteria problem into a single criterion form scientifically and objectively, to realize comprehensive evaluation in one dimension space (Holder, 1990). Its essence is how to properly determine the weights of evaluation criteria.
The Analytic Hierarchy Process (AHP) is a structured technique for organizing and analyzing complex decisions. Based on mathematics and psychology, it was developed by Thomas L. Saaty in the 1970s and has been extensively studied and refined since then.
The Analytic Hierarchy Process (AHP) is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have longterm repercussions. It has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives (Zeshui, 2000). It has particular application in group decision making, and is used around the world in a wide variety of decision situations, in fields such as government, business, industry, healthcare, and education.
2.1. AHP hierarchical structure model
AHP divide the problem into criteria according to the nature and the goal of the problem. It breaks down the factors into target hierarchy, standards hierarchy and scheme hierarchy according to the relationship between factors. The standards hierarchy can be broken down further to form a hierarchical structure model (as shown in Figure 1) which can be analyzed quantitatively and qualitatively to obtain the weights of importance of the lowest hierarchy criteria against the highest hierarchy criteria. AHP finds the final synthesis weights through pairwise comparisons to get objective and accurate results.
Figure 1. AHP hierarchical structure model
2.2. The procedure for using AHP
The procedure for using the AHP can be summarized as:
Model the problem as a hierarchical structure containing the decision goal, the alternatives for reaching it, and the criteria for evaluating the alternatives.
Establish priorities among the elements of the hierarchy by making a series of judgments based on pairwise comparisons of the elements. Each of these judgments is assigned a number on a scale. One common scale (adapted from Saaty) is shown in Table 1.
Intensity of importance 
Definition 
Explanation 
1 
Equal importance 
Two factors contribute equally to the objective 
3 
Somewhat more important 
Experience and judgment slightly favor one over the other 
5 
Much more important 
Experience and judgment strongly favor one over the other 
7 
Very much more important 
Experience and judgment very strongly favor one over the other. Its importance is demonstrated in practice 
9 
Absolutely more important 
The evidence favoring one over the other is of the highest possible validity 
2,4,6,8 
Intermediate values 
When compromise is needed 
Table 1. Manufacturing vs. Remanufacturing
Synthesize these judgments to yield a set of overall priorities for the hierarchy. In terms of matrix algebra, this consists of calculating the "principal vector" of the matrix by adding the members of each column to find the total. In order to normalize each column to sum to 1.0 or 100%, divide the elements of that column by the total of the column and sum them up. Then, add the elements in each resulting row and divide this sum by the number of elements in the row to get the average.
Check the consistency of the judgments. A true consistency ratio is calculated by dividing the consistency index for the set of judgments by the index for the corresponding random matrix. Saaty suggests that if that ratio exceeds 0.1 the set of judgments may be too inconsistent to be reliable. A CR of 0 means that the judgments are perfectly consistent.
Come to a final decision based on the results of this process. Numerical priorities are calculated for each of the decision alternatives. These numbers represent the alternatives' relative ability to achieve the decision goal, so they allow a straightforward consideration of the various courses of action.
3. The Fuzzy comprehensive
evaluation
method
Fuzzy comprehensive evaluation method is a synthetical assessment method that applies fuzzy mathematical principles to evaluate things and phenomenon affected by variety of factors. It regards evaluation objectives as a fuzzy set (named the Factor Set U) composed of variety of factors with different assessment levels selected. Another fuzzy set named the Evaluation Set V is employed to calculate the membership degree of each individual factor in the Evaluation Set to establish a fuzzy matrix. The quantitative evaluation value of each factor is finally determined by calculating the weight distribution of each factor in evaluation goal. It applies the fuzzy transformation theory and maximum membership degree law, and makes a comprehensive evaluation to various factors. Specific steps are as follows:
Determine the Factor Set of evaluation object, that is U= {u_{1}, u_{2}, …, u_{m}}. It is a set composed of m kinds of evaluation factors.
Determine the Evaluation Set, that is V = {v_{1}, v_{2}, …, v_{n}}. It’s a set composed of n kinds of evaluation standards.
Construct singlefactor evaluation matrix. Evaluate singlefactor and then get vector R_{i}. A singlefactor evaluation matrix R is constituted by numbers of singlefactor evaluation vector put together. There are some commonly used membership degree calculation function of singlefactor, such as “linearity lower semiladdershaped” distribution function and so on.

(1) 
In (1), R is a fuzzy relationship matrix composed of evaluation Factor Set U and the Evaluation Set V.
Determine evaluation factors weight vector E= {e_{1}, e_{2}, …, e_{m}}. There are many methods to calculate weight, such as over standard weighting method, analytic hierarchy process (AHP) and so on. Among them, over standard weighting method is commonly used in articles about all kinds of fuzzy comprehensive evaluation, which is calculated as follows:
ei=xi/ai (2)
In (2), x_{i} is the practice measure value of the i factor in distinguishing samples, and a_{i} states the statistical average value of the i factor in distinguishing standard. As the weight of fuzzy comprehensive evaluation method, it needs normalize, that is:
(3)
Comprehensive evaluation. The last results of comprehensive evaluation can be got by doing complex operations calculation between single factor weight vector E and fuzzy relationship matrix R, which is as follows.
(4)
In (4), bi is the membership degree value of evaluation samples to each evaluation standard. The determination results are usually defined according to the maximum membership degree law.
4. Establishment for product quality evaluation system
A large motorcycle specialized production enterprise with international advanced level of motorcycle product technology and equipment, built a engine manufacturing, assembling and production lines, the annual production capacity can reach millions of motorcycles. According to the study of the enterprise, the product quality evaluation system is established.
Figure 2. System structure model
The first hierarchy criterion is product comprehensive quality, divided into two second hierarchy criteria as product entity quality and product quality system. To decide which supplier who manufacture the same parts is better, the most direct way is to compare the entity quality of the parts, including passing rate, outoftolerance parts utilization rate, repair rate, rejection rate, these data are coming from the quality record of each supplier. The product quality system is divided into the third hierarchy criteria as purchasing level, management level, quality level, nonconforming product handling, quality inspection condition, manufacturing capability, packing and storage, equipment condition. The third hierarchy criteria can be further divided into the forth hierarchy criteria. The product quality evaluation system structure model as shown in Figure 2, the forth hierarchy criteria will be given in Table 1 due to the space limit.
5. The products quality evaluation model
5.1. Setting up quality evaluation hierarchy criteria
First hierarchy 
Second hierarchy 
Third hierarchy 
Forth hierarchy 

Product quality U 
Product entity quality U1 
Passing rate U11 
——————— 

Outoftolerance parts utilization rate U12 


Repair rate U13 


Rejection rate U14 


Product quality system U2 
Purchasing level U21 
Purchasing documents 
U211 

Selection of qualified supplier 
U212 

Quality assurance agreement 
U213 

Inspection methods agreement 
U214 

Raw materials inspection planning and control 
U215 

Raw materials quality record 
U216 

Methods to resolve the dispute 
U217 

Management level U22 
The degree of quality improvement 
U221 

Quality organization or institution 
U222 

Quality responsibility system 
U223 

Quality documents and plan 
U224 

Quality level U23 
Manufacturing and assembly quality 
U231 

Appearance quality 
U232 

Basic performance 
U233 

Reliability 
U234 

Nonconforming product handling U24 
Integrity of handling procedure steps of nonconforming product 
U241 

Handling measure 
U242 

Documents record 
U243 

Create file 
U244 

Quality inspection condition U25 
Inspection system perfection 
U251 

Integrity of inspection measure 
U252 

Timeliness of inspection data transmission 
U253 

Inspection personnel 
U254 

Manufacturing capability U26 
Key process 
U261 

Process audit 
U262 

Process capability 
U263 

Packing and Storage U27

Damp proofing and tightness 
U271 

Label, traceability and durability 
U272 

Warehouse area and ventilation 
U273 

Regular inspection system 
U274 

Process to file creation 
U275 

Equipment condition U28 
Applicability and accuracy 
U281 

Preventive maintenance plan 
U282 

Integrality and unity 
U283 

Equipment advancement 
U284 
Table 2. Quality evaluation hierarchy criteria
According to the comprehensive evaluation hierarchical structure, set up quality evaluation hierarchy criteria. For example, the first evaluation hierarchy criterion is U = {U1, U2}, the second evaluation hierarchy criteria are U1 = {U11, U12, U13, U14} and U2 = {U21, U22, U23, U24, U25, U26, U27, U28}. By analogy, the evaluation criteria at all hierarchies are designed, as shown in Table 2.
5.2. Judge matrix structuring, weight calculation and concordance analyzing
Pairwise comparing each pair of criteria that in
the lower hierarchy
of the same criterion, deciding the degree of importance of each
criterion to
structure the judge matrix. Calculate the weights of each criteria and
analyzing
the concordance of judge matrix. Only when judge matrix passed
concordance
examination can the weights of criteria be scientific.
5.3. The evaluation method used
The evaluation methods commonly used are direct evaluation method (total score method), weighted comprehensive evaluation method and fuzzy comprehensive evaluation method. Direct evaluation method is suitable for the basic criteria in the bottom hierarchy of the evaluation system that are easily evaluated, and is used in the evaluation for the criteria U11U14 in this system. For the evaluation of criteria U and U1, weighted comprehensive evaluation method is applied, as it’s suitable for the basic criteria that are in the middle or top hierarchy. And fuzzy comprehensive evaluation method is adopted for fuzzy criteria that are hard to evaluate directly and need subjective evaluation, thus it’s used in the evaluation for criteria U2, U21U28 and all the forth hierarchy criteria.
6. The empirical analysis
The evaluation method mentioned above is used to evaluate the quality of a product that supplier offered. According to the quality record of the supplier, we can get the scores of the criteria U11U14, as is shown in Table 3.
Criterion 
Score 
U11 
0.90 
U12 
0.68 
U13 
0.78 
U14 
0.84 
Table 3. Criteria scores of U11U14
Experts are invited to grade the forth hierarchy criteria as excellent, good, ok or bad. Then the scores matrix of forth hierarchy criteria will be determined by the proportion of each grade, as is shown in Table 4.
Criterion 
Score matrix 
U211 
(0.5,0.3,0.2,0) 
U212 
(0.4,0.1,0.2,0.3) 
U213 
(0.5,0.4,0.1,0) 
U214 
(0.3,0.4,0.1,0.2) 
U215 
(0.7,0.1,0.2,0) 
U216 
(0.6,0.2,0.1,0.1) 
U217 
(0.5,0.4,0.1,0) 
U221 
(0.3,0.3,0.2,0.2) 
U222 
(0.5,0.4,0.1,0) 
U223 
(0.2,0.2,0.5,0.1) 
U224 
(0.4,0.5,0.1,0) 
U231 
(0.6,0.4,0,0) 
U232 
(0.7,0.3,0,0) 
U233 
(0.6,0.2,0.2,0) 
U234 
(0.6,0.2,0.2,0) 
U241 
(0.4,0.2,0.3,0.1) 
U242 
(0.5,0.4,0.1,0) 
U243 
(0.6,0.2,0.2,0) 
U244 
(0.3,0.2,0.4,0.1) 
U251 
(0.5,0.4,0.1,0) 
U252 
(0.2,0.4,0.1,0.3) 
U253 
(0.4,0.2,0.4,0) 
U254 
(0.5,0.2,0.2,0.1) 
U261 
(0.5,0.2,0.3,0) 
U262 
(0.7,0.2,0.1,0) 
U263 
(0.4,0.3,0.2,0.1) 
U271 
(0.6,0.2,0.2,0) 
U272 
(0.6,0.4,0,0) 
U273 
(0.4,0.1,0.5,0) 
U274 
(0.5,0.2,0.3,0) 
U275 
(0.5, 0.5,0,0) 
U281 
(0.5,0.2,0.3,0) 
U282 
(0.6,0.3,0.1,0) 
U283 
(0.7,0.2,0.1,0) 
U284 
(0.8,0.2,0,0) 
Table 4. Criteria scores matrix
Enterprise quality department is invited to pairwise each pair of criteria that under the same criterion and in the same hierarchy. If the judge matrix gets through the concordance analyzing, then the weights can be calculate as Table 5.
Criterion 
Weight 
U 
(0.667, 0.333) 
U1 
(0.470, 0.284, 0.171, 0.074) 
U2 
(0.209, 0.148, 0.119, 0.123, 0.123, 0.081, 0.086, 0.110) 
U21 
(0.209, 0.190, 0.172, 0.141, 0.132, 0.124, 0.125, 0.116) 
U22 
(0.494, 0.239, 0.203, 0.063) 
U23 
(0.462, 0.281, 0.151, 0.107) 
U24 
(0.544, 0.224, 0.129, 0.103) 
U25 
(0.511, 0.258, 0.151, 0.080) 
U26 
(0.669, 0.257, 0.074) 
U27 
(0.411, 0.216, 0.115, 0.171, 0.088) 
U28 
(0.338, 0.205, 0.251, 0.118, 0.088) 
Table 5. Criteria Weights
The third hierarchy criteria score of product quality system are calculated with fuzzy mathematical evaluation method. The result is shown in Table 6.
Criterion 
Score 
U21 
(0.494, 0.267, 0.149, 0.090) 
U22 
(0.334, 0.316, 0.231, 0.119) 
U23 
(0.628, 0.320, 0.052, 0) 
U24 
(0.438, 0.245, 0.253, 0.065) 
U25 
(0.408, 0.354, 0.153, 0.085) 
U26 
(0.544, 0.207, 0.241, 0.007) 
U27 
(0.551, 0.258, 0.191, 0) 
U28 
(0.606, 0.212, 0.182, 0) 
Table 6. Criteria scores of U21U28
The product entity quality criteria score is calculated with weighted comprehensive evaluation method. The result is shown below:
b1= 0.813
The product quality system criteria score is calculated with fuzzy mathematical evaluation method. The result is shown below:
b2= B2×Yw =(0.490, 0.277, 0.178, 0.055)×(0.95, 0.83, 0.68, 0.30) = 0.833
Finally, the comprehensive product quality score is calculated with weighted comprehensive evaluation method. The result is shown below:
b= A×P = A ×(b1，b2)^{T} =(0.667, 0.333)×(0.813, 0.833)^{T} = 0.819
The comprehensive quality score of the product is 0.819. Other products comprehensive quality scores can be calculated as the same. Finally, the selection of the products will be made by selecting one which gets the highest score.
7. Conclusion
This study is to develop a scientific quality evaluation system which comprehensively applies AHP method, weighted comprehensive evaluation and fuzzy comprehensive evaluation, to convert subjective judgment into scientific decisionmaking result.
The main contribution of this research was the identification of the important criteria for supplier selection process. The second contribution was a development of a multicriteria decision model for evaluation and selection which is used for supplier selection. The model for supplier evaluation and selection were successfully developed by using AHP method dedicated for manufacturing enterprises. The four hierarchy of AHP model is assessing decisionmakers to identify and evaluate the supplier selection. Finally, the developed model is tested by empirical analysis. The results show the models are able to assist decisionmakers to examine the strengths and weaknesses of supplier selection. Furthermore, the proposed AHP model is significantly effective in decision making. With the use of AHP model software, the results can be transferred to a spreadsheet for easy computations and it is easier to identify and evaluate suppliers to arrive to a consensus decision.
References
Holder, R.D. (1990). Some Comments on the Analytic Hierarchy Process. Operational Research Society, 41, 10731076.
Juntian, W. (2008). Study on the acquirementmode improvement on AHP judgment information and its related decisionmaking approach. Jilin University.
Shenghai, Q., Hua, Y., & Ningsheng, W. (2005). Research for comprehensive evaluation of enterprise group partner in agile manufacturing. Enterprise Informatization, 5, 1721.
Zeshui, X. (2000). A simulationbased evaluation of several scales in the analytic hierarchy process. Systems Engineering Theory & Practice, 20, 5862.
This work is licensed under a Creative Commons Attribution 4.0 International License
Journal of Industrial Engineering and Management, 20082022
Online ISSN: 20130953; Print ISSN: 20138423; Online DL: B287442008
Publisher: OmniaScience