Description:

This tutorial attends to the concepts of mathematical modeling and optimization. In an era where mathematical programming and machine learning are increasingly intertwined, optimization finds strong relevance. This workshop will discuss how to mathematically model real world problems and solve them using optimization tools in python.

Machine learning researchers have embraced the advances in mathematical programming allowing new types of models to be pursued [1][2]. Mathematical programming puts a premium on accuracy, speed and robustness. Future novel machine learning models and novel optimization models will only continue to grow with advancement of time.

Generally, optimization includes finding best available values of some objective function given a defined set of constraints. This field of study comes under the field of Operational Research (OR) methods, with a wide range of problem solving techniques use to improve decision making and efficiency. The areas of such application lies in financial engineering, marketing, revenue management to transportation problems. We will also discuss area where optimization finds application in machine learning scenarios and possible limitations. As part of beginner material, we will model using Linear Programming (LP) method and solve our modeled problem using convex optimization solver, cvx-opt.

The workshop will be in two phases:

1. How to mathematically model and use cvx-opt solver to model optimization problem as LP.

   • An introduction to mathematical modeling in real world scenario and methods.
   • Discussion of tools and choice of python tool, cvx-opt and discuss gains and limitations.
   • Solving static optimization problems. Hands on coding and running small applications.

2. Use the knowledge gained in 1. to real life problems.

   • Implement and execute code snippets to design L1 non linear regression tool, based on Manhattan distance. Explain its comparison to existing tool library: polyfit in numpy
   • (Medium to Advanced) If time permits. Implement and execute a dynamic optimization problem related to reinforced/Unsupervised learning. The modules are still under consideration, but not limited to portfolio optimization/stock pricing index examples.

Using Python for Optimization:

Python has numerous powerful solver packages for scientific computing. For this workshop we will use the cvx-opt package, software package for convex optimization based on the Python programming language.

Bibliography

[1] Kristin P. Bennett and Emilio Parrado-Hernández. 2006. The Interplay of Optimization and Machine Learning Research. J. Mach. Learn. Res. 7 (December 2006), 1265-1281.

[2] Sra, S. and Nowozin, S. and Wright, S.J. Optimization for Machine Learning, Neural information processing series, MIT Press, 2012.

Prerequisites:

The attendees should have basic Python programming experience. Prior knowledge of optimization is not required but good to have.

The cvx-opt package needs to be pre-installed with the required dependency package as indicated on the page. Acquaintance with numpy and matplotlib package is a plus but not mandatory.

Content URLs:

The content slides aren't available now, but will be updated in due time.

Convex Optimization and related literature can be found here : convex optimization

Cvx-opt Literature can be accessed from here: cvx-opt-user-guide

Speaker Info:

MS by Research, Graduate student, IIITB.

My research area lies in large scale optimization and application to real world problems. My research work involved applying such methods to optimize digital circuits, thereby creating Electronic Design Automation (EDA) tools. Prior to my MS, I have worked as an embedded design engineer for two years.

Speaker Links:

You can reach me on Facebook and find me on Git