Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model

It is well-known that the efficiency of mixed integer linear mathematical programming depends on the model (formulation) used. With the same mathematical programming solver, a given problem can be solved in a brief calculation time using one model but requires a long calculation time using another. In this paper a new, unexpected feature to be taken into account is presented: the order of the constraints in the model can change the calculation time of the solver considerably. For a test problem, the Response Time Variability Problem (RTVP), it is shown that the ILOG CPLEX 9.0 optimizer returns a ratio of 17.47 between the maximum and the minimum calculations time necessary to solve optimally 20 instances of the RTVP, according to the order of the constraints in the model. It is shown that the efficiency of the mixed integer linear mathematical programming depends not only on the model (formulation) used, but also on how the information is introduced into the solver.


Introduction
Integer linear programming is a classical tool in practical operations research that can be applied to many problems (e.g.Salkin and Mathur, 1989) very effectively (e.g. in Corominas et al. 2008 it was applied to solve a real problem of a doi: 10.3926/jiem.2008.v1n2.p4-15 ©© JIEM, 2008-01(02): 4-15 -ISSN: 2013-0953 Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model 5 R. Pastor motorcycle assembly line and in Pastor et al. 2008 it was applied to solve the case of a woodturning company).The technique is well-known and reliable but it must be handled carefully.It is known that its efficiency depends on the model (formulation) used: with the same mathematical programming solver, a given problem can be solved in a brief calculation time using one model but requires a long calculation time using another.Therefore, as stated by Billionnet (1999, p. 105): "Given a problem with a few dozen of variables one cannot be confident that integer programming will work until it has been tried on realistic instances".
Several techniques have been used to improve the efficiency of this tool.A standard technique is the elimination of symmetries: Margot (2007), for example, presents techniques for handling symmetries in integer linear programs in which variables can take integer values, which extends previous research that dealt exclusively with binary variables.Tightening the definition of the data and introducing redundant constraints have also provided good results: Corominas et al. (2006), for example, demonstrated the importance of modelling, as well as the huge impact that redundant constraints and the elimination of symmetries have on the effectiveness of MILPs for solving the Response Time Variability Problem (RTVP), an NP-hard scheduling problem (Corominas et al., 2007); the total computation time taken to solve 20 instances dropped from 38,603 to 398 seconds and its practical limit to obtaining optimal solutions was increased from 25 to around 40 units to be scheduled.This paper argues that the order of the constraints in a model can have a considerable effect on the time that a mathematical programming solver takes to solve a problem optimally.Lets us, for example, take three sets of constraints (A, B and C) of a problem to be solved.To introduce the sets of constraints in the mathematical programming solver in the order A- B-C, A-C-B, B-A-C, B-C-A, C-A-B and C-B-A is not indifferent and can cause its efficiency to vary considerably.This new, unexpected feature that must be taken into account in mathematical programming has not been presented previously (to the best of the authors' knowledge).
For an integer programming formulation of a test problem, RTVP, it is shown that the ILOG CPLEX 9.0 optimizer returns a ratio of 17.47 between the maximum and minimum calculation times needed to solve optimally 20 instances of the RTVP, according to the order of the constraints in the model: with one permutation of the The rest of the paper is organized as follows.Section 2 presents the RTVP.Section 3 describes the computational experiment carried out.Finally, Section 4 is devoted to the conclusions.

The test problem: the Response Time Variability Problem
The Response Time Variability Problem (RTVP) occurs whenever products, clients or jobs need to be sequenced so as to minimize variability in the time between the instants at which they receive the necessary resources.It was recently defined in the literature and first presented by Corominas et al. (2007), who proved that it is NP-hard.
This problem has a broad range of real-world applications.For example, it can be used in the automobile industry to sequence the models to be produced on a mixed-model assembly line (Monden, 1983).Other contexts in which the RTVP appears are the computer multi-threaded systems and network servers (Waldspurger and Weihl, 1995), the periodic machine maintenance problem (Bar-Noy et al., 2002), the scheduling of advertising slots for television (Brusco, 2008), the design of sales catalogs (problem introduced in Bollapragada et al., 2004) and the scheduling of waste collection (Herrmann, 2007).
The abovementioned applications are examples of a very common situation in which a resource must be used successively by different units and it is important to schedule them in such a way that the different types of units share the resource in some fair manner.The RTVP proposes a measure of fairness: to minimize the variability of the distance (measured, for example, in number of slot times) between any two consecutive units of the same product (event, job or client); i.e., to have the distances between any two given consecutive units of the same product as constant as possible.
The RTVP is formulated as follows.Let be the number of products, the number of units of product and the total number of units .Let be a solution of an instance in the RTVP that consists of a circular sequence of units , where is the unit sequenced in position of sequence .For all products in which , let be the distance between the positions at which the units and of product are found (i.e. the number of positions between the units, where the distance between two consecutive positions is considered equal to 1).As the sequence is circular, position 1 comes immediately after position ; therefore, is the distance between the first unit of product in a cycle and the last unit of the same product in the preceding cycle.Let be the average distance between two consecutive units of product .For all products in which , is equal to .The objective is to minimize the .(2) to ( 11).

Computational experiment
Using the RTVP as the test problem, a computational experiment was performed in order to show that the order of the constraints in the model can have a considerable effect on the calculation time of the mathematical programming solver.The basic data used for the experiment are as follows: • The 20 instances in Computational Experiment 1 in Corominas et al. (2006) were used.In short, was between 21 and 40 units, was between 3 and 12 products, and was between 1 and 15 units.
• Sets ( 2) to ( 11), the 10 sets of constraints of the model shown in the Annex, were grouped into seven groups of constraints (G1 to G7).All possible permutations of these seven groups of constraints were carried out; therefore, 7! MILP models were generated and solved optimally (5,040 models).Table 1 shows the 10 original sets of constraints and the seven groups of constraints that were used to test the order .
The grouping of the original sets of constraints was carried out in accordance with the similarity between the characteristics of the RTVP that were modeled by the restrictions.As explained later in this paper, grouping the constraints into seven groups shows that the order of the constraints in the model has a decisive influence on the efficiency; therefore, it was not necessary to generate and solve optimally the 10! MILP models (it would indeed be highly time-consuming: solving the 7! MILP models took 4,550,156 seconds).
• The MILP models were solved to optimality using the ILOG CPLEX 9.0 optimizer and a Pentium IV PC at 3.4 GHz with 1 GB of RAM.These models were run with the same computer environment in order to the performance (calculation time) of the next run was not affected.(5) G3 (6) ( 7) G4 (8) G5 ( 9) G6 (10) (11) G7 Table 1."Grouping of the original sets of constraints".
The 5,040 MILP models were generated and solved optimally for each of the 20 test instances (a total of 100,800 models).Obviously, the number of variables and constraints of the 5,040 models were the same for each instance, because the only difference between these MILP models was the order of the seven groups of constraints.For each of the 5,040 models the sum of the calculation time of the 20 instances was obtained.Table 2 shows the permutations required by the minimum and maximum calculation times (in seconds).

Permutation of the groups Time (in seconds)
G5_G2_G3_G6_G1_G7_G4 335 G6_G5_G4_G2_G7_G3_G1 5.851 Table 2. "Best and worst permutations of the groups of constraints".
Table 2 shows the ILOG CPLEX 9.0 optimizer takes only 335 seconds for permutation G5_G2_G3_G6_G1_G7_G4, whereas for permutation G6_G5_G4_G2_G7_G3_G1 it takes 5,851 seconds.That is, in order to solve 20 instances of this test problem, the order of the constraints in the model returns a ratio of 17.47 between the maximum and the minimum calculation times of the ILOG CPLEX solver.

Conclusions
This paper presents a new, unexpected feature that should be taken into account whenever a mixed integer linear mathematical formulation must be solved optimally with a mathematical programming solver: the order of the constraints in the model can considerably change the calculation time of the solver.Because the ILOG CPLEX 9.0 optimizer is commercial software, it is very difficult for the authors to explain its performance.One hypothesis, which is not confirmed, is that the order in which the variables are handled in the search process depends on the order of the constraints in the model.Obviously, if we formalize the RTVP with another model (or we solve another problem) the difference in performance according to the order of the constraints in the model could be lower, but it could also be even higher.
Future research work will involve checking if this new, unexpected feature is also showed when another mathematical programming solver is used.The possibility to establish a formalised procedure to achieve, a priori, the best order of the constraints in the model is another area of future research.

Annex
The RTVP model to be tested is specifically as follows. Data:

Number of products
Total number of units Number of units of product to be scheduled ; it is assumed that doi:10.3926/jiem.2008.v1n2.p4-15©© JIEM, 2008 -01(02): 4-15 -ISSN: 2013-0953    Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model 6 R. Pastor sets of constraints the solver takes only 335 seconds, whereas with another one it takes 5,851 seconds.It is shown that the efficiency of the mixed integer linear mathematical programming depends not only on the model (formulation) used, but also on how the information is introduced into the solver.

For
a feasible solution.For example, the sequence (C, A, C, B, C, B, A, C) is a solution, where .Corominas et al. (2007) presented the RTVP and proposed a mixed integer linear programming (MILP) model and five heuristic algorithms for solving it.García et al. (2006) presented six metaheuristic algorithms: a multi-start, a greedy randomized adaptive search procedure (GRASP) and four variants of a discrete particle swarm optimization (PSO) algorithm.Other ten discrete PSO algorithms were proposed inGarcía-Villoria and Pastor (2009).A cross-entropy method approach was used inGarcía-Villoria et al. (2007).The Electromagnetism-like Mechanism was proposed to solve the RTVP inGarcía-Villoria and Pastor (2008a).Finally, the best heuristic results recorded to date have been obtained with a Psychoclonal algorithm(García- Villoria and Pastor, 2008b).Corominas et al. (2006) presented an improved MILP model; in our paper a slight simplification of this improved MILP model is doi:10.3926/jiem.2008.v1n2.p4-15©© JIEM, 2008 -01(02): 4-15 -ISSN: 2013-0953 Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model 8 R. Pastor considered.The model to be tested, which is presented in the Annex, consists of an objective function to be minimised, labelled (1), and 10 sets of constraints, labelled doi:10.3926/jiem.2008.v1n2.p4-15©© JIEM, 2008 -01(02): 4-15 -ISSN: 2013-0953    Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model 9 doi:10.3926/jiem.2008.v1n2.p4-15©© JIEM, 2008 -01(02): 4-15 -ISSN: 2013-0953    Variations in the efficiency of a mathematical programming solver according to the order of the constraints in the model 10 R. PastorIn order to reveal this difference in performance, a computational experiment was conducted in which the Response Time Variability Problem (RTVP) was solved using the ILOG CPLEX 9.0 optimizer.It returned a ratio of 17.47 between the maximum and the minimum calculation times necessary to solve optimally 20 instances of the RTVP, according to the order of the constraints in the model.With one permutation of the sets of constraints the solver needs only 335 seconds, whereas with another one it needs 5,851 seconds.It has thus been shown that the efficiency of the mixed integer linear mathematical programming depends not only on the model used, but also on how the information is introduced into the solver.