Introduction to Python - Gurobi
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Introduction to Python
Quick reminder about questions
Please ask them!
Copyright 2017, Gurobi Optimization, Inc.
A few words about Gurobi
Gurobi algorithms
• Continuous optimization
• LP: primal and dual simplex, sifting, parallel barrier, concurrent
• QP: primal and dual simplex, parallel barrier
• QCP: parallel barrier
• Discrete optimization (MIP, MIQP, MIQCP)
• Parallel branch-and-bound
• Relaxations solved with continuous algorithms
• Cutting planes and heuristics
• General
• Automatic presolve
• Identifying irreducible inconsistent subsystems (IIS)
• Automated parameter tuning tool
• …
Copyright 2017, Gurobi Optimization, Inc.
Interfacing with Gurobi
• Command-line tool
• Full-featured interactive shell
• Matrix-oriented APIs
• C, MATLAB, R
• Object-oriented APIs
• C++, Java, .NET, Python
• Modeling systems
• Commercial: AMPL, Frontline Solvers, …
• Free: JuliaOpt, PuLP, Pyomo, SolverStudio, …
Copyright 2017, Gurobi Optimization, Inc.Getting started with Python
Python in a nutshell
• Python is a powerful programming language
• Very easy to learn
• Runs on every major operating system
• Suitable for beginners and professional developers
• Clean, simple and compact syntax
• Open Source community: Tons of ready-made libraries and frameworks
• Python is also a great choice for mathematical modeling
• Many popular packages for scientific computing
• Jupyter Notebooks: Interactive graphical environment
• Language syntax looks similar to mathematical notation
Copyright 2017, Gurobi Optimization, Inc. 7Why Python?
• Gurobi provides a variety of programming interfaces
• Python
• Java
• C++
• .NET (C#, Visual Basic, …)
• MATLAB,
• R
• ...
• Gurobi is committed to all of these interfaces
• This presentation focuses on Python since it is easy to learn and use
Copyright 2017, Gurobi Optimization, Inc. 8Python installation
• The Gurobi Optimizer installation includes a small Python 2.7
distribution with a basic set of packages.
• Gurobi Optimizer works with multiple Python distributions
(including http://www.python.org).
• In this presentation, we use Anaconda Python
• Download from http://www.gurobi.com/downloads/get-anaconda
• Windows, Mac, Linux; 32/64-bit
• All basic packages in the default installation
• Easy-to-use package management (conda)
• Gurobi provides/maintains Anaconda packages
• Python 2.7 (recommended) and 3.5 (latest)
Copyright 2017, Gurobi Optimization, Inc. 9Basic concepts
• Python is an “interpreted language”
• No need for a manual compiling step before running a program.
• Performance optimization is done in the background (byte-code compilation/execution).
• Two basic ways to use Python
• Put your code in a text file (e.g. example.py) and run the command “python example.py”.
• Start the Python Interactive Shell (command “python”) and enter commands line by line.
• In this presentation we will also use Jupyter Notebook
• A rich graphical interactive environment for Python
Copyright 2017, Gurobi Optimization, Inc. 10Jupyter Notebook
• The Jupyter Notebook is a web application that
allows you to create and share documents that
contain live code, equations, visualizations and
explanatory text.
• Free software with an active open source
community
• Born out of the IPython Project in 2014
• Already included in Anaconda Python
• Simply change to your working directory and
run jupyter notebook on the command-line
• Learn more at http://jupyter.org/
Copyright 2017, Gurobi Optimization, Inc. 11Integrated development environments (IDEs)
• Many Python IDEs available: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments
• Easy to install Gurobi in other Python distributions
• Spyder is distributed with Anaconda
• Scientific Python Development EnviRonment
• Familiar for users of MATLAB, Visual Studio, etc.
• Built-in editor, file explorer, documentation browser
• Graphical interface for debugging (ipdb)
• IPython console for interactive code execution
Copyright 2017, Gurobi Optimization, Inc.Connecting Gurobi and Python
Copyright 2017, Gurobi Optimization, Inc.Connecting Python and Gurobi
• Basic Python interpreter: gurobi.bat / gurobi.sh
• Included and configured when you install Gurobi Optimizer
• Anaconda Python – automated installation
conda config --add channels http://conda.anaconda.org/gurobi
conda install gurobi
• Manual installation for other Python distributions
• Change to GUROBI_HOME directory
• Run python setup.py install
• For details, see the Gurobi Quick Start Guide: http://www.gurobi.com/documentation/
Copyright 2017, Gurobi Optimization, Inc. 14The Gurobi Python API
• Gurobi Interactive Shell is a Python shell with Gurobi extensions
• Gurobi distribution includes Python interpreter and basic modules
• Nothing extra to buy or install
• Don't need to be a Python expert to start modeling in Python
• Powerful environment for interacting with existing models
• Model file formats: .mps, .rew, .lp, .rlp, .ilp
• All file formats support built-in compression: .zip, .gz, .bz2, .7z
Copyright 2017, Gurobi Optimization, Inc.Demonstration: Solve model in Jupyter Notebook
• Goals:
• Make yourself familiar with Jupyter Notebook
• Solve a model file in MPS format
• Example file: afiro.mps (included in the examples
directory of the Gurobi installation)
• Steps:
• Import the gurobipy module
• Create model object from model file
• Solve model to optimality
• Print solution values
Copyright 2017, Gurobi Optimization, Inc. 16Log file (default: gurobi.log)
Gurobi 7.0.2 (mac64, Python) logging started Thu Feb 2 11:57:06 2017
Read MPS format model from file afiro.mps
Reading time = 0.00 seconds
AFIRO: 27 rows, 32 columns, 83 nonzeros
Optimize a model with 27 rows, 32 columns and 83 nonzeros
Coefficient statistics:
Matrix range [1e-01, 2e+00]
Objective range [3e-01, 1e+01]
Bounds range [0e+00, 0e+00]
RHS range [4e+01, 5e+02]
Presolve removed 18 rows and 20 columns
Presolve time: 0.01s
Presolved: 9 rows, 12 columns, 32 nonzeros
Iteration Objective Primal Inf. Dual Inf. Time
0 -4.8565680e+02 1.363638e+02 0.000000e+00 0s
3 -4.6475314e+02 0.000000e+00 0.000000e+00 0s
Solved in 3 iterations and 0.01 seconds
Optimal objective -4.647531429e+02
Copyright 2017, Gurobi Optimization, Inc.Log file
Variable X
-------------------------
X01 80
X02 25.5
X03 54.5
X04 84.8
X06 18.2143
X14 18.2143
X16 19.3071
X22 500
X23 475.92
X24 24.08
X26 215
X36 339.943
X37 383.943
Copyright 2017, Gurobi Optimization, Inc.Interactive shell environment
• Interactive shell provides access to all Gurobi Optimizer functionality through Python API
• read() is a global function which returns an object of type Model
• Model.optimize() and Model.printAttr() are methods (functions on the Model object)
• We can choose any name for the Model object variable:
model = read('afiro.mps') m = read('afiro.mps')
model.optimize() or m.optimize()
model.printAttr('X') m.printAttr('X')
Copyright 2017, Gurobi Optimization, Inc.Basic Gurobi concepts
• Parameters control the operation of the Gurobi solvers. They
must be modified before the optimization begins.
Model Attributes Parameters
http://www.gurobi.com/documentation/current/parameters.html
• Attributes are the primary mechanism for querying and Constraints Attributes
modifying properties of a Gurobi model.
Variables Attributes
http://www.gurobi.com/documentation/current/refman/attributes.html
Special Ordered Sets Attributes
Attributes can be associated with
• the model as a whole (e.g. the objective function value)
• variables (e.g. lower bounds)
• constraints (e.g. the right-hand side)
• SOSs (e.g. IIS membership)
Environment Global Parameter Settings
• Environments are the containers for models and global
Model 1 Local Parameter Settings
parameter settings. When a model is created inside an
environment, it creates a local copy of all global parameter
Model 2 Local Parameter Settings
settings.
• The Python API automatically provides a default environment.
Copyright 2017, Gurobi Optimization, Inc. 20Attributes
• Multiple ways to access attributes
• Methods: getAttr() and setAttr()on Model, Constr, or Var objects
• Model.getAttr()also can retrieve attributes for multiple variables or constraints
• Directly: m.ModelName
• Examples:
• print(m.getAttr('ObjVal'))
• print(m.ObjVal)
• m.ModelName = 'My Model'
• m.setAttr(m.ModelName, 'My Model')
Copyright 2017, Gurobi Optimization, Inc.Python API fundamentals #1
• Important global functions
• setParam() set (global) parameters for new models in this session
• read() create model object from a model file
• Primary objects
• Model the model
• Var a variable
• Constr a constraint
• Parameter/attribute examples:
• setParam('TimeLimit', 60) Set global parameter
• model.TimeLimit = 60 Set local parameter
• variable.VarName = 'TotalCosts' Set variable attribute
• print(constraint.RHS) Set constraint attribute
• Other important methods:
• Model.setObjective(expression) Set objective function
• Model.optimize() Start optimization
• Model.reset() Reset model state
Copyright 2017, Gurobi Optimization, Inc. 22Python API fundamentals #2
• Model summary information methods
• Model.printAttr() display attribute values in bulk, without needing a loop
• Model.printStats() display sizes, coefficient statistics, etc.
• Model.printQuality() display info about solution quality
• Model introspection methods
• Model.getVars() get variables as a list of Var objects
• Model.getConstrs() get constraints as a list of Constr objects
• Model.getRow() get LHS expression for a given constraint
• Model.getCol() get list of constraints for a given variable
• Python control statements
• for i in :
• if :
• Must indent body of each statement (at least one space or tab)
Copyright 2017, Gurobi Optimization, Inc. 23Getting help
• Use the help() function with no arguments to get general help
• You can get help on specific Gurobi objects
• help(Model) Gurobi model object (or help(m) if m is a model)
• help(Var) Gurobi variable object (or help(v) if v is a variable)
• help(Constr) Gurobi constraint object (or help(c) if c is a constraint)
• help(GRB.attr) Gurobi attributes
• You can get help on specific functions or methods
• Ex: help(read)
• Ex: help(Model.printAttr)
• You can get help on any parameter
• Ex: paramHelp('Method')
Copyright 2017, Gurobi Optimization, Inc.Demonstration – model introspection
• Launch interactive shell
• Read coins model from examples/data subdirectory of Gurobi installation:
model = read('/examples/data/coins.lp')
• Print model statistics and solve it:
model.printStats()
model.optimize()
• Display solution:
for v in :
if v. != 0:
print(v., v.)
• Notes/Hints:
• Don't forget to indent for control structures and enter blank line to end the for loop
• Use help(Model) to replace with correct method
• Use help(Var) to replace and with correct attributes
Copyright 2017, Gurobi Optimization, Inc.Console
gurobi> model = read('coins.lp')
gurobi> model.printStats()
Statistics for model Unnamed :
Linear constraint matrix : 4 Constrs, 9 Vars, 16 NZs
Variable types : 4 Continuous, 5 Integer
Matrix coefficient range : [ 0.06, 7.2 ]
Objective coefficient range : [ 0.01, 1 ]
Variable bound range : [ 50, 1000 ]
RHS coefficient range : [ 0, 0 ]
Copyright 2017, Gurobi Optimization, Inc.Console
gurobi> model.optimize()
...
Found heuristic solution: objective -0
Presolve removed 1 rows and 5 columns
Presolve time: 0.01s
Presolved: 3 rows, 4 columns, 9 nonzeros
Variable types: 0 continuous, 4 integer (0 binary)
Root relaxation: objective 1.134615e+02, 4 iterations, 0.00 seconds
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 113.46154 0 1 -0.00000 113.46154 - - 0s
H 0 0 113.4500000 113.46154 0.01% - 0s
Explored 0 nodes (4 simplex iterations) in 0.03 seconds
Thread count was 4 (of 4 available processors)
Optimal solution found (tolerance 1.00e-04)
Best objective 1.134500000000e+02, best bound 1.134500000000e+02, gap 0.0%
Copyright 2017, Gurobi Optimization, Inc.Console
gurobi> for v in model.getVars():
....... if v.x != 0:
....... print v.varname, v.x
.......
Dimes 2.0
Quarters 53.0
Dollars 100.0
Cu 999.8
Ni 46.9
Zi 50.0
Mn 30.0
Copyright 2017, Gurobi Optimization, Inc.Deciphering errors
• Errors are raised when something goes wrong
• Trace specifies where error occurred
• Last line specifies error type and may provide additional details
• Ex: IndentationError raised when block spacing is incorrect (must indent before if)
gurobi> for v in model.getVars():
....... if v.x != 0:
File "", line 2
if v.x != 0:
^
IndentationError: expected an indented block
• corresponds to interactive shell
Copyright 2017, Gurobi Optimization, Inc.Deciphering errors
• Ex: AttributeError raised when attribute is incorrect (should be VarName)
gurobi> for v in model.getVars():
....... if v.X != 0:
....... print v.Name, v.X
.......
Traceback (most recent call last):
File "", line 3, in
File "var.pxi", line 76, in gurobipy.Var.__getattr__ (../../src/python/gurobipy.c:11798)
File "var.pxi", line 121, in gurobipy.Var.getAttr (../../src/python/gurobipy.c:12310)
AttributeError: 'gurobipy.Var' object has no attribute 'Name'
• Don't worry about trace from within gurobipy
Copyright 2017, Gurobi Optimization, Inc.Additional model methods
• Attribute and parameter management
• Model.getAttr() get attribute values
• Model.setAttr() set attribute values
• Model.setParam() set parameter values for a model
• Model.getParamInfo() get current parameter values
• Importing and exporting
• Model.read() import data from a file (ex: .prm)
• Model.write() export model data to a file (ex: .sol)
• Data type is determined by file extension
• Copying and exporting
• Model.copy() create model copy, ex: n = m.copy()
• Model.relax() create model copy with integer variables relaxed
• Model.fixed() create model copy with integer variables fixed to values in the current solution
Copyright 2017, Gurobi Optimization, Inc.Additional model methods
• Algorithms
• Model.optimize() solve a model
• Model.reset() discard any solution information (next solve starts from scratch)
• Model.presolve() create new model containing presolved version of original model
• Model.computeIIS() compute IIS for an infeasible model
• Model.feasRelax() perform feasibility relaxation on a model by introducing penalty variables
and penalty objective
Copyright 2017, Gurobi Optimization, Inc.Editing and running Python programs
• Python is not just an interactive shell
• Python programs are typically stored in text files with .py extension
• Any text editor can be used to edit them (ex: notepad)
• Most programming editors have Python mode with syntax highlighting, etc.
• Comments are preceded with '#' character
• Most functionality in Python is provided by modules
• To use a module in a program, include an import statement
• Ex: import math
• To use the Gurobi interface in a program, import the gurobipy module:
from gurobipy import *
• Use command line to run program with Python interpreter distributed with Gurobi
Copyright 2017, Gurobi Optimization, Inc.program.py
# import gurobipy module
from gurobipy import *
# create empty model
m = Model()
# change model name attribute
m.ModelName = 'My First Model'
# process the pending change
m.update()
# export model to file
m.write('emptymodel.mps')
m.write('emptymodel.lp')
# print to console
print('This program creates an empty model named:', m.Modelname)
print('Model exported to emptymodel.mps and emptymodel.lp')
Copyright 2017, Gurobi Optimization, Inc.Run a Python program: Console output
$ gurobi.sh program.py
This program creates an empty model named: My Model
Model exported to emptymodel.mps and emptymodel.lp
Copyright 2017, Gurobi Optimization, Inc.Useful packages
• Common questions:
• Can Gurobi read data from … ?
• Can Gurobi output the solution as …?
• Can Gurobi be used through a web service?
• Can Gurobi solve this model using an approach that … ?
• Can Gurobi… ?
• The answer:
• Yes, most of the time since Gurobi stands on the shoulders of giants
• Python includes a large standard library
• Many more add-on libraries and frameworks are available for download
• The Python API is a full interface that includes all Gurobi features
• You can use Python to build powerful, complete Gurobi applications
Copyright 2017, Gurobi Optimization, Inc.Pandas
• Fast and efficient DataFrame and Series objects
• Read/write CSV, HDF5, Excel, SQL, plain text
• Missing data handling
• Slicing, fancy indexing, subsetting
• Merges, joins
• Split-apply-combine operations (groupby)
http://pandas.pydata.org
Copyright 2017, Gurobi Optimization, Inc.Bokeh
• Interactive visualization for web browsers
• High performance with streaming data and big data
• In-browser interactivity on otherwise static pages
• Bindings for Python, Scala, Julia, R
http://bokeh.pydata.org
Copyright 2017, Gurobi Optimization, Inc.Use our Gurobi online resources
• The Gurobi Reference Manual is freely available at
http://www.gurobi.com/documentation/current/refman/index.html
It contains detailed descriptions of our algorithms and APIs.
• We provide API examples for all major programming languages as well as
interactive examples at
http://www.gurobi.com/resources/examples/example-models-overview
Copyright 2017, Gurobi Optimization, Inc. 39Gurobi documentation
• Quick Start Guides
• One for each supported OS
• Example Tour
• 22 examples
• Examples for all APIs
• Reference Manual
• Detailed information on APIs, parameters, attributes, …
• Available online at http://www.gurobi.com/documentation
• Docs are installed in the docs subdirectory
• Sample code is installed in the examples subdirectory
Copyright 2017, Gurobi Optimization, Inc.You can also read
