The University of Melbourne
School of Mathematics and Statistics
MAST 20018 – Discrete Maths and Operations Research
A/Prof. Alysson M. Costa
School of Mathematics and Statistics
Office 149 – Peter Hall Building
Last updated: October 18, 2022
Contents
1 Introduction to Operations Research 4
2 Modelling in Linear Programming 18
3 The geometry of Linear Programming 34
4 The algebra of Linear Programming 45
5 Solving Linear Programs 68
6 The simplex tableau 84
7 Duality theory 106
8 Sensitivity analysis 154
1
Chapter 1
Introduction to Operations Research
2
Operations Research
“In a nutshell, operations research (O.R.) is the discipline of applying advanced analytical methods to help make better
decisions.” INFORMS | What O.R. Is
Figure 1.1: The Operations Research approach [4]
3
Applications of Operations Research
Many applications can be found in areas such as:
Figure 1.2: Applications – Slide by Gurobi ©
4
Jobs in Operations Research
Given the multi-disciplinary nature of Operations Research, jobs in the area cover a number of different profiles. Overall, a
very employable student will have good coding, communication and mathematical skills. I have former students who have
worked at (focusing on Australian Businesses):
• Accenture
• AECOM
• Argon & Co
• Australian Department of Defence
• Biarri
• Cardinal Operations
• CSRIO
• INESC TEC – Institute for Systems and Computer Engineering, Technology and Science
• Intelligent Decision Support – Opturion Pty
• Neighbourlytics
• Scada Data Analytics
• Singapore-MIT Alliance for Research & Technology
• Transport Safety Victoria
• Victorian Centre for Data Insights
• Victoria Police
• Workforce analytics
• YPC Technologies
• For further opportunities, see Careers in Mathematics and Statistics
5
Branches of Operations Research
• Mathematical Programming,
• Dynamic Programming,
• Simulation,
• Heuristics and metaheuristics
• Stochastic Processes / Queuing Theory,
• Inventory Theory,
• Graph Theory,
• etc.
Figure 1.3: Holistic illustration of the disciplines and problems related to operations research. Image by Alex Elkjaer
Vasegaard
6
Mathematical Programming
Modelling and solving a problem using mathematics.
Figure 1.4: Subfields of Mathematical Optimisation: Retrieved from Wikipedia
7
Choosing the right model
Decision making models are mostly useful when they can be solved. Using a modelling framework allows for solving the
models with generic algorithms for that framework.
The choice of the modelling framework is often guided by the capacity of solution algorithms to solve realistic instances of
a problem.
Figure 1.5: Algorithms shed light on models
8
Linear Programming
Linear programming is a modelling framework in which constraints and objective function are linear expressions on
continuous decision variables.
= min
s.t.
≥ ,
≥ 0
Figure 1.6: Pictorial representation of a 2D Linear Program. Fig from Hartman-Baker et. al.
9
Find the cheapest diet that supply daily requirements of 2000 calories, 55g protein, 800mg calcium.
Food Serving size Energy (Cal) Protein (g) Calcium (mg) Price per serving ($)
Oatmeal 28g 110 4 2 0.30
Chicken 100g 205 32 12 2.40
Eggs 2 large 160 13 54 1.30
Whole milk 250ml 160 8 285 0.90
Cherry pie 170g 420 4 22 0.20
Pork and beans 260g 260 14 80 1.90
Example: The diet problem
Data:
• : set of foods
: cost per serving of food ,
• : set of nutrients
• = minimum required level of nutrient ,
• = maximum allowable level of nutrient .
• : amount of nutrient in food
Model:
= min
∑︁
∈
s.t.∑︁
∈
≥ min, ∈ ,∑︁
∈
≤ max, ∈ ,
≥ 0, ∈ , ∈ .
10
Solving the problem
import gurobipy as gp
from gurobipy import GRB
#DATA
Food, cal, protein, calcium, price = gp.multidict({
’oatmeal’: [110, 4, 2, .30],
’chicken’: [205, 32, 12, 2.40],
’egg’: [160, 13, 54, 1.30 ],
’milk’: [160, 8, 285, .90 ],
’pie’: [420, 4, 22, .20 ],
’pork’: [260, 14, 80, 1.90 ]})
r = {
’cal’: 2000,
’protein’: 55,
’calcium’: 800}
#MODEL
m = gp.Model(“diet”)
x = m.addVars(Food, name = ’x’)
calCons = m.addConstr( gp.quicksum( cal[f]*x[f] for f in Food) >= r[’cal’] )
calProt = m.addConstr( gp.quicksum( protein[f]*x[f] for f in Food) >= r[’protein’] )
calCalc = m.addConstr( gp.quicksum( calcium[f]*x[f] for f in Food) >= r[’calcium’] )
m.setObjective( gp.quicksum(price[f]*x[f] for f in Food ),GRB.MINIMIZE)
#Analysis
m.optimize()
# Display Solution
print(’SOLUTION:’)
for v in m.getVars():
if v.x > 1e-6:
print(” “, v.varName, v.x)
# Display optimal solution value
print(’ Cost: ’, m.objVal)
#Requirements provided
print(” Calories: “,sum( x[f].x*cal[f] for f in Food ) )
print(” Protein : “, sum( x[f].x*protein[f] for f in Food ))
print(” Calcium : “, sum( x[f].x*calcium[f] for f in Food ))
Implementation
11
Set parameter Username
Academic license – for non-commercial use only – expires 2022-08-28
Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (mac64[x86])
Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
Optimize a model with 3 rows, 6 columns and 18 nonzeros
Model fingerprint: 0x1ed81c8f
Coefficient statistics:
Matrix range [2e+00, 4e+02]
Objective range [2e-01, 2e+00]
Bounds range [0e+00, 0e+00]
RHS range [6e+01, 2e+03]
Presolve removed 0 rows and 4 columns
Presolve time: 0.00s
Presolved: 3 rows, 2 columns, 6 nonzeros
Iteration Objective Primal Inf. Dual Inf. Time
0 0.0000000e+00 3.950000e+02 0.000000e+00 0s
2 3.7821577e+00 0.000000e+00 0.000000e+00 0s
Solved in 2 iterations and 0.00 seconds (0.00 work units)
Optimal objective 3.782157676e+00
SOLUTION:
x[milk] 2.0643153526970957
x[pie] 9.62136929460581
Cost: 3.7821576763485485
Calories: 4371.265560165975
Protein : 55.0
Calcium : 800.0
[Finished in 0.5s]
12
MAST 20018 – Unimelb Handbook
Learning outcomes:
• L1 Comprehend the essential features of problems encountered in Operations Research investigations.
• L2 Develop basic skills required to construct formal mathematical models for practical optimisation problems, and
those required to analyse settings from real-world applications;
• L3 Appreciate the extent and limitations of a number of Operations Research techniques for solving real-world
problems.
Generic skills:
• GS1 Problem-solving skills: the ability to engage with unfamiliar problems and identify relevant solution strategies;
• GS2 Analytical skills: the ability to construct and express logical arguments and to work in abstract or general terms
to increase the clarity and efficiency of analysis;
• GS3 Collaborative skills: the ability to work in a team; and
• GS4 Time-management skills: the ability to meet regular deadlines while balancing competing commitments
13
MAST 20018 – Syllabus (Operations Research)
1. Mathematical modelling.
2. Linear Programming.
Algebra of linear programming.
The simplex method.
Duality Theory.
14 15
Chapter 2
Modelling in Linear Programming
16
The right level of simplification
Finding the right level of simplification is key in modelling.
—————–
On Exactitude in Science – Jorge Luis Borges, Collected Fictions, translated by Andrew Hurley [1].
…In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a
City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the
Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point
with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that
that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and
Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in
all the Land there is no other Relic of the Disciplines of Geography.
Suarez Miranda,Viajes devarones prudentes, Libro IV,Cap. XLV, Lerida, 1658.
—
17
Ceci n’est pas une pipe
We model nature and society to understand how they work. Modelling is the art of interpreting and representing reality.
Figure 2.1: Rene Magritte – The treachery of images
18
Modelling frameworks
Nonlinear Programming
Both constraints and objective function can be nonlinear expressions on the decision variables.
= min ()
s.t.
() ≤ 0,
ℎ() = 0
∈ R.
Figure 2.2: (, ) = () · cos()
19
Linear Programming
The constraints and objective function are linear expressions on continuous decision variables.
= min
s.t.
≥ ,
≥ 0
Figure 2.3: Vertices and simplex path [2]
20
Mixed-Integer Programming
The constraints and objective function are linear expressions on continuous or discrete decision variables.
= min +
s.t.
+ ≥ ,
≥ 0, ∈ R
≥ 0, ∈ Z
If all variables are binary ( ∈ {0, 1}), we have Binary Programming.
If all variables are integer ( ∈ {0, 1}), we have Integer Programming.
Figure 2.4: IP polytope with LP relaxation. Reprinted from Wikimedia Commons, 2010
21
Modelling and solving problems computationally
There are many linear programming solvers available both commercially and free for academic use.
Examples of commercial solvers (also with free academic licences):
• CPLEX,
• Gurobi,
• FICO Xpress
Examples of open source solvers:
• Cbc,
• GLPK,
The implementation of models is often made with mathematical programming languages or packages such as AMPL,
Julia/JuMP, Gurobipy. Common mathematical software, such as MATLAB also have packages for solving linear programs.
Microsoft Excel Solver also solves linear (integer) and nonlinear optimization problems.
22
Modelling in Linear Programming
23
The company Dovetail produces two kinds of matches: long and short ones. The company makes a profit of 3 (x$1,000)
for every 100,000 boxes of long matches, and 2 (x$1,000) for every 100,000 boxes of short matches. The company has
one machine that can produce both long and short matches, with a total of at most 9 (x100,000) boxes per year. For
the production of matches the company needs wood and boxes: three cubic meters of wood are needed for 100,000
boxes of long matches, and one cubic meter of wood is needed for 100,000 boxes of short matches. The company has
18 cubic meters of wood available for the next year. Moreover, Dovetail has 7 (x100,000) boxes for long matches, and
6 (x100,000) for short matches available at its production site. The company wants to maximize its profit in the next
year. It is assumed that Dovetail can sell any amount it produces.
Example: The Dovetail problem [3]
Variables:
1 ≥ 0: number of 100,000 boxes of long matches produced,
2 ≥ 0: number of 100,000 boxes of short matches produced.
Model:
Let be the maximum profit the company can obtain given such technological and resource constraints.
= max 31 + 22 (2.1)
s.t.
1 + 2 ≤ 9, (2.2)
31 + 2 ≤ 18, (2.3)
1 ≤ 7, (2.4)
2 ≤ 6, (2.5)
1, 2 ≥ 0.
24
Adelaide has two water catchment storage facilities. Storage 1 can store up to 400 megalitres per day. Storage 2 can
store up to 500 megalitres per day. Three secondary dams are supplied from these two facilities: Barossa needs at least
300 megalitres per day, Happy Valley needs at least 200 megalitres per day and Kangaroo Creek needs at least 400
megalitres per day
The distances between storage facilities and the secondary dams are (in kilometres).
Dam 1 Dam 2 Dam 3
Storage Facility 1 75 40 85
Storage Facility 2 20 55 75
Formulate a linear program that minimises the total transportation distances to meet the demands of the secondary
dams, such that capacities of the storage facilities are not violated.
A transportation problem
• – quantity transported from storage to damn .
min 7511 + 4012 + 8513 + 2021 + 5522 + 7523
such that
11 + 12 + 13 ≤ 400
21 + 22 + 23 ≤ 500
11 + 21 ≥ 300
12 + 22 ≥ 200
13 + 23 ≥ 400
11, 12, 13, 21, 22, 23 ≥ 0.
25
import gurobipy as gp
from gurobipy import GRB
#DATA
Storages = [“Storage 1″,”Storage 2”]
Damns = [“Baroosa”,”Kangaroo Creek”, “Happy Valley”]
Cap = {
“Storage 1”: 400,
“Storage 2”: 500}
Dem = {
“Baroosa”: 300,
“Kangaroo Creek”: 400,
“Happy Valley”: 200}
dist ={
(“Storage 1”, “Baroosa”): 75,
(“Storage 1”, “Happy Valley”): 40,
(“Storage 1”, “Kangaroo Creek”): 85,
(“Storage 2”, “Baroosa”): 20,
(“Storage 2”, “Happy Valley”): 55,
(“Storage 2”, “Kangaroo Creek”): 75,
}
#MODEL
m = gp.Model(“transp”)
x = m.addVars(Storages,Damns, name = ’x’)
demCons = m.addConstrs( gp.quicksum( x[s,d] for s in Storages) >= Dem[d] for d in Damns )
SupCons = m.addConstrs( gp.quicksum( x[s,d] for d in Damns) <= Cap[s] for s in Storages)
m.setObjective( gp.quicksum( dist[s,d]*x[s,d] for s in Storages for d in Damns ) )
# Analysis
m.optimize()
print(’SOLUTION:’)
print(’ Cost: ’, m.objVal)
for v in m.getVars():
if v.x > 1e-6:
print(” “, v.varName, v.x)
Implementation
26
The classical transportation problem
Data:
• – set of origins,
– capacity of origin ∈ .
• – set of destinations,
– demand of destination ∈ ,
• – distance between origin ∈ and destination ∈ ,
Variables:
• – quantity transported between ∈ and ∈ .
Model:
min
∑︁
∈
∑︁
∈
,
such that ∑︁
∈
≥ , ∈ ,∑︁
∈
≤ , ∈ ,
∈ R×+ .
27
import gurobipy as gp
from gurobipy import GRB
import random
random.seed(1)
#DATA
n = 10 #number of ’storages’
m = 50 #number of ’damns’
Storages = range(n)
Damns = range(m)
cap = {}
for s in Storages:
cap[s] = random.randint(100,1000)
dem = {}
for d in Damns:
dem[d] = random.randint(10,100)
dist = {}
for s in Storages:
for d in Damns:
dist[s,d] = random.randint(10,100)
#MODEL
m = gp.Model(“transp”)
x = m.addVars(Storages,Damns, name = ’x’)
demCons = m.addConstrs( gp.quicksum( x[s,d] for s in Storages) >= dem[d] for d in Damns )
supCons = m.addConstrs( gp.quicksum( x[s,d] for d in Damns) <= cap[s] for s in Storages)
m.setObjective( gp.quicksum( dist[s,d]*x[s,d] for s in Storages for d in Damns ) )
m.optimize()
print(’SOLUTION:’)
print(’ Cost: ’, m.objVal)
Implementation
28
A steel company must decide how to allocate next week’s time on a rolling mill, which is a machine that takes slabs
of steel and can produce either bands (at 200 tonnes/hour) or coils (at 140 tonnes/hour). Bands and coils can be sold
for $25/tonne and $30/tonne respectively. Based on currently booked orders, the company must produce at least 6000
tonnes of bands and 4000 tonnes of coils. Given that there are 40 hours of production time this week, how many tonnes
of bands and coils should be produced to yield the greatest revenue?
Production planning:
• – number of products produced, ∈ {bands, coils}
max 25 + 30
such that
200
+
140
≤ 40,
≥ 6000,
≥ 4000,
29
import gurobipy as gp
from gurobipy import GRB
#DATA
sellprice = {
“bands”: 25,
“coils”: 30,
}
#MODEL
m = gp.Model(“prod”)
b = m.addVar(name = ’b’)
c = m.addVar(name = ’c’)
m.addConstr( b/200 + c/140 <= 80)
m.addConstr( b >= 6000)
m.addConstr( c >= 4000)
m.setObjective( sellprice[’bands’]*b + sellprice[’coils’]*c , GRB.MAXIMIZE )
# Analysis
m.optimize()
print(’SOLUTION:’)
print(’ Profit: ’, m.objVal)
for v in m.getVars():
if v.x > 1e-6:
print(” “, v.varName, v.x)
#Checking
print(“Capacity constraint lhs: “, b.x/200 + c.x/140 )
Implementation
30
A generic production problem
Data:
– set of items,
– minimum demand of item ∈ ,
– profit for unit sold of item ∈ ,
– set of resources,
– capacity of resource ∈ ,
– ammount of resource ∈ used in the production of item ∈ ,
Variables:
ammount produced of item ,
Model:
Max =
∑︁
∈
subject to
∑︁
∈
≤ ∈ ,
≥ ∈ .
31
import gurobipy as gp
from gurobipy import GRB
import random
random.seed(1)
#DATA
n = 100 #number of ’products’
m = 10 #number of ’resources’
I = range(n)
J = range(m)
cap = {}
for j in J:
cap[j] = random.randint(1000,2000)
dem = {}
for i in I:
dem[i] = random.randint(1,5)
a = {}
p = {}
for i in I:
p[i] = random.randint(5,10)
for j in J:
a[i,j] = random.randint(1,3)
#MODEL
m = gp.Model(“production”)
x = m.addVars(I, name = ’x’, lb=dem)
capCons = m.addConstrs( gp.quicksum( a[i,j]*x[i] for i in I) <= cap[j] for j in J)
m.setObjective( gp.quicksum( p[i]*x[i] for i in I ), GRB.MAXIMIZE )
m.optimize()
print(’ Profit: ’, m.objVal)
for v in m.getVars():
if v.x > 1e-6:
print(” “, v.varName, v.x)
#Checking
for j in J:
print(f”Capacity constraints {j}: { sum(a[i,j]*x[i].x for i in I) } <= {cap[j]} “, )
Implementation
Bynaries, Integers…
Mixed integer programming is built on the shoulders of linear programming. What a reason to understand linear programming.
= min +
s.t.
+ ≥ ,
≥ 0, ∈ R
≥ 0, ∈ Z
32
Chapter 3
The geometry of Linear Programming
Definitions and 2D visualisation
In this section, we will define some basic concepts while using the 2D space to gain insight on their meaning before moving
to higher dimensions.
For an LP problem with n variables, a vector ∈ R is called a feasible solution if it satisfies all constraints of the
problem. The set of all feasible solutions of an LP problem is called the feasible region.
Feasible solution
The set of all feasible solutions of an LP problem is called the feasible region.
Feasible region
33
Consider the problem
∗ := max = 41 + 32
subject to
21 + 2 ≤ 40 (production machine time)
1 + 2 ≤ 30 (production packaging time)
1 ≤ 15 (market demand)
1 ≥ 0
2 ≥ 0.
What is the space of feasible region?
Example: Drawing the feasible region in 2D
34
The geometry in higher dimensions
We are interested in studying problems that often have thousands (if not millions) of variables. A 2D visualisation can not
go very far. We need to develop algebraic structures that will allow us to ‘see’ in higher dimensions.
Let (1) , …, () be a collection of points in R.
A point such that
= 1
(1) + 2 (2) + … + () ,
is a linear combination of (1) , …, () .
Linear combination
A point such that
= 1
(1) + 2 (2) + … + () ,
where 0 ≤ ≤ 1 and ∑ = 1,
is a convex combination of (1) , …, () .
Definition: Convex combination
35
The line segment joining two points , in R is given by all points in the convex combination of and .
Example: Line segment
A point in the line segment between and is given by:
= + (1 − )
for some 0 ≤ ≤ 1.
This follows the definition above with 1 = , 2 = (1 − ).
Figure 3.1: A line segment connecting points and in the plane.
36
A set ∈ R is a hyperplane if = { ∈ R | = } for some nonzero ∈ R and some ∈ R.
Hyperplanes
A set ′ ⊆ R is a (closed) halfspace if ′ = { ∈ R | ≥ } for some nonzero ∈ R and some ∈ R
Halfspaces
21 + 32 = 5 is an hyperplane in R2,
21 + 32 ≤ 5 is an hyperspace in R2,
Example:
A subset C of R is convex if for any two points , ∈ C then any point in the convex combination of and is also in
C.
Definition: Convex set
37
Hyperplanes and their half-spaces are convex sets.
Theorem: Hyperplanes and halfspaces define convex sts
Proof:
38
Let C1, . . . ,C be a collection of convex sets. Then
⋂
=1
C
is a convex set.
That is, the intersection of any finite number of convex sets is a convex set.
Theorem: The intersection of convex sets is convex
Proof:
39
A polytope is a set that can be expressed as the intersection of a finite number of closed half-spaces.
Definition: Polytopes
Show that polytopes are convex sets.
Example:
40
The graphical method for 2D LPs
We can use a graphical method to solve LP problems in two dimensions. Different level curves of the objective function can
be obtaibed by = , for fixed values of .
Figure 3.2: Level curves
The graphical method looks for the point in the feasible region that touches the level curve with maximum value of .
41
Sove the problem with the graphical method.
max = 3 + 2
s.t.
2 + ≤ 10
+ ≤ 8
, ≥ 0
Example: Graphical Solution method
• Profit z = constant i.e. 3 + 2 = , defines a line, called an iso-profit line.
• Changing the constant moves the line, but all iso-profit lines are parallel.
• Increasing the constant moves the line in the direction of improving profit.
• Increase the constant until the iso-profit line no longer intersects the feasible region. The highest constant value for
which the iso-profit line intersects the feasible region must be the optimal value, and points in the intersection are
optimal solutions.
42
For any LP, one of the three statements is true.
1. The LP is infeasible.
2. The optimal value of the LP is∞ (i.e., the LP does not have a bounded optimum).
3. A basic feasible solution exists that achieves the optimal value.
Consider the problem
∗ := max = 41 + 32
21 + 2 ≤ 40
1 + 2 ≤ 30
1 ≤ 15
41 + 32 ≥ 150
1 ≥ 0
2 ≥ 0.
Infeasible LP
Figure 3.3: Infeasible region.
There is no intersection among the constraints. No feasible solution exists.
Consider the problem
∗ := max = 41 + 32
41 + 32 ≥ 101
1 ≥ 0
2 ≥ 0.
Unbounded and optimal LPs
In this case the feasible region is unbounded. The objective function increases as we ‘head further into the unbounded
region’, we say that the problem is unbounded and no optimal solution exists.
Depending on the objective function, the optimisation problem on unbounded regions can have an optimal solution. For
example, consider max = −41 − 32.
43
Figure 3.4: Multiple optima.
44
Chapter 4
The algebra of Linear Programming
max =
∑︁
=1
111 + 122 + … + 1 ≤ 1
211 + 222 + … + 2 ≤ 2
…
…
…
11 + 22 + … + ≤
≥ 0, = 1, …,
with 1, 2, . . . , ≥ 0.
Standard form of a LP
max =
∑︁
=1
111 + 122 + … + 1 + +1 = 1
211 + 222 + … + 2 + +2 = 2
…
…
…
11 + 22 + … + + + =
≥ 0, = 1, …, +
with 1, 2, . . . , ≥ 0.
Canonical form of a LP
45
Write the model below in canonical form:
= max 31 + 22
s.t.
1 + 2 ≤ 9,
31 + 2 ≤ 18,
1 ≤ 7,
2 ≤ 6,
1, 2 ≥ 0.
Example
46
General procedure for transforming any LP problem to canonical form
1. For a minimisation problem, convert it to a maximisation problem by multiplying the objective function by −1
2. For each negative RHS coefficient < 0, multiply the corresponding functional constraint by −1 (and change the
direction of the inequality)
3. For each variable which is unrestricted in sign, introduce new variables (1) ,
(2)
≥ 0 and replace by (1) − (2)
4. For each “≥” functional constraint, introduce a surplus variable so as to obtain a “=” constraint
5. For each “=” functional constraint, introduce an artificial variable (we will learn how to deal with these intruders later)
6. For each “≤” functional constraint, introduce a slack variable
7. Now the problem is in canonical form whose basic variables are the slack and artificial variables
47
Convert the problem to canonical form.
min = 1 − 2
1 + 2 ≤ −5
1 ≥ −10
2 = 3
1 ≥ 0,
2 ∈ R
Example:
48
Matrix representation
= max
s.t.
≤ ,
≥ 0.
=
1 1
3 1
1 0
0 1
, =
[
1
2
]
, =
9
18
7
6
, =
[
3
2
]
.
= max
s.t.
[, ] = ,
≥ 0.
[, ] ==
1 1 1 0 0 0
3 1 0 1 0 0
1 0 0 0 1 0
0 1 0 0 0 1
, =
1
2
3
4
5
6
=
9
18
7
6
, =
3
2
0
0
0
0
.
49
General matrix representations of SF and CF
=
11 12 · · · 1
21 22 · · · 2
…
…
…
…
1 2 · · ·
, =
1
2
…
, =
1
2
…
,
Standard form:
max =
≤
≥ 0
Canonical form:
max =
+ = ,[
]
≥ 0.
50
Solutions in the standard and in the canonical form have a one-to-one correspondence. To any solution ∈ to
max =
≤
≥ 0
There is an equivalent solution =
[
]
to
max =
+ = ,
=
[
]
≥ 0.
51
We will often extend the definition of a model in canonical form to any problem with variables and equality
constraints such that an indentity matrix can be found among its columns.
Canonical form of a LP (Revisited)
Show that the LP problem
max = 21 − 32 + 43 + 5
21 + 02 + 13 + 34 + 05 = 6
−1 + 12 + 03 + 04 + 05 = 2
21 + 02 + 03 + 64 + 15 = 2
1, 2, 3, 4, 5 ≥ 0
is in canonical form.
Example:
The LP is in canonical form because the 3rd, 2nd and 5th columns of the coefficient matrix are, respectively,
1
0
0
,
0
1
0
,
0
0
1
Note that (0, 2, 6, 0, 2) is a feasible solution to this LP problem.
52
The space defined by linear constraints = is convex.
What if we had ≥ constraints? What if we need variables to assume negative values? What if our objective function is
a minimisation one?
Convexity of { : = }
Proof: Assume 1 and 2 are such that they respect the system of linear equations.
Consider a point 3 in the convex combination of 1 and 2:
3 = 1 + (1 − )2,with 0 ≤ ≤ 1.
Therefore, 3 = (1 + (1 − )2),
3 = 1 + (1 − )2,
3 = + (1 − ),
3 = ,
and therefore it also respects the equalities, showing that a set of equality constraints defines a convex set.
53
A point of a convex set C is said to be an extreme point or vertex of C if whenever we write
= + (1 − ),
with , ∈ C and 0 < < 1, then = = .
Geometrically, this means that there are no points and in C (different from ) with lying on the line segment
connecting and .
Extreme points
Figure 4.1: Feasible extreme points for region
Three ways to define an extreme point [3]:
• The vector x0 ∈ is called a vertex of if and only if there are independent hyperplanes in the collection
{1, . . . , +} that intersect at x0.
• The vector x0 ∈ is called a vertex of if and only if there is a hyperplane (not necessarily one of 1, . . . , +)
with corresponding halfspace + such that ⊆ + and ∩ = {x0}.
• The vector x0 ∈ is called a vertex of if and only if there are no two distinct x′, x′′ ∈ such that x0 = x′+ (1−)x′′
for some ∈ (0, 1)
54
If a linear programming problem has exactly one optimal solution, then this solution must be an extreme point of the
feasible region.
Theorem: Extreme points and optimality
Proof: We shall prove this theorem by contradiction.
Assume, to the contrary, that the problem has exactly one optimal solution, call it , that is not an extreme point of the
feasible region. Since is not an extreme point, there are two distinct feasible solutions, say and , which are not equal to
and a scalar , 0 < < 1, such that
= + (1 − ).
If we rewrite the objective function in terms of and rather than , we obtain
() = ( + (1 − )).
Hence
() =
∑︁
=1
=
∑︁
=1
[
+ (1 − )
]
=
∑︁
=1
+ (1 − )
∑︁
=1
= () + (1 − ) ().
Now, because 0 < < 1, there are only three possibilities
1. () < () < ()
2. () < () < ()
3. () = () = ()
Since is an optimal solution, the first two possibilities cannot occur (why?). Thus, the third case must hold.
But this contradicts the assertion that is the only optimal solution. Our initial assumption that is not an extreme point
must be wrong.
Prove that if a linear programming problem has more than one optimal solution, it must have infinitely many optimal
solutions. Furthermore, the set of optimal solutions is convex.
Multiple optima
55
Consider a problem in canonical form:
max =
∑︁
=1
111 + 122 + … + 1 + +1 = 1
211 + 222 + … + 2 + +2 = 2
…
…
…
11 + 22 + … + + + =
≥ 0, = 1, …, +
A basic feasible solution is a solution 1, . . . + that respects all constraints and have at least components set at
zero.
Basic Solutions
56
The LP
max 31 + 42
21 + 2 ≤ 4
1 + 22 ≤ 3
1, 2 ≥ 0,
is in standard form.
The corresponding canonical form is
max 31 + 42
21 + 2 + 3 = 4
1 + 22 + 4 = 3
1, 2, 3, 4 ≥ 0
The ‘trivial basic solution’, derived by selecting original variables at zero is = (0, 0, 4, 3). (What are the conditions
for this trivial basic solution to be feasible?)
Find another basic solution.
Example: basic feasible solutions
Suppose we select 2 and 3 to be basic. Then, the reduced system is
2 + 3 = 4
22 = 3.
This yields the basic feasible solution
= (0, 3/2, 5/2, 0).
57
A collection of vectors (1) , …, () in R is said to be linearly independent if
1
(1) + 2 (2) + … + () = 0,
implies that
1 = 2 = … = = 0.
Linear independence
The rank of the matrix refers to the number of linearly independent rows or columns in the matrix.
Rank of a matrix
Consider a system with variables and constraints:
=
≥ 0
We often partition the system in the following format:
××1 + ×−−×1 = ×1
We choose linear independent columns for . This makes a square matrix with complete rank (invertible) and,
therefore variables can be written in terms of variables as:
=
−1 + −1 , (General solution of the system)
Basic and Non-basic matrices
58
For the system = represented with Basic and Non-basic matrices and :
(×)×1 + ×(−) (−)×1 = ×1
A solution is basic if (for a full rank ), we set all variables at zero. Moreover, if all components of are ≥ 0, the
solution is a feasible basic solution.
Basic solutions in matrix format
The problem:
max 31 + 42
21 + 2 + 3 = 4
1 + 22 + 4 = 3
1, 2, 3, 4 ≥ 0
can be represented as:
= , ≥ 0
for:
=
[
2 1 1 0
1 2 0 1
]
, =
1
2
3
4
=
[
4
3
]
, =
3
4
0
0
0
0
.
1(2 = 0)
2(1 = 0)
3 = 0
4 = 0
59
Trying all possible matrices , = índices of variables in B, = índices of variables in :
• = {1, 2}, = {3, 4}
=
[
2 1
1 2
]
, =
[
1 0
0 1
]
,
for = 0, = −1 =
[
5/3
2/3
]
, (feasible basic).
• = {1, 3}, = {2, 4}
=
[
2 1
1 0
]
, =
[
1 0
2 1
]
,
for = 0, = −1 =
[
3
−2
]
, (infeasible basic).
• = {1, 4}, = {2, 3}
=
[
2 0
1 1
]
, =
[
1 1
2 0
]
,
for = 0, = −1 =
[
2
1
]
, (feasible basic).
• = {2, 3}, = {1, 4}
=
[
1 1
2 0
]
, =
[
2 0
1 1
]
,
for = 0, = −1 =
[
3/2
5/2
]
, (feasible basic).
• = {2, 4}, = {1, 3}
=
[
1 0
2 1
]
, =
[
2 1
1 0
]
,
for = 0, = −1 =
[
4
−5
]
, (infeasible basic).
• = {3, 4}, = {1, 2}
=
[
1 0
0 1
]
, =
[
2 1
1 2
]
,
for = 0, = −1 =
[
4
3
]
, (feasible basic).
