Convex Optimization (MOOC) by Stephen Boyd

For many machine learning problems you can find an equivalent problem which is convex. This seems to be the method of choice for dealing with things like Support Vector Machines. Many academics seem to dislike neural networks because of how easy it is to get stuck in local minima and how hard it is to say anything useful about the result. This won't happen with techniques based on convex optimization.

Outside of machine learning solving a convex relaxation of a discrete optimization problem and then trying to round the result is one of the standard techniques in optimization. This is used all over the place, from straightforward linear relaxations for ILPs through the semidefinite relaxations of problems such as max cut. You start with an intractable problem, embed it into a tractable problem and use the result to gain information about the solutions to your original problem.

Yes!

For instance, a very simple example: When you travel from A to B you sometimes can reach that destination in the same time using different paths. "Same", in the sense of a certain threshold or uncertainty considering Real World (TM) applications. Furthermore, there can be constructed academic problems, where you have an infinite amount of solutions. Say, traveling from the north to the south pole in shortest time over the surface of a perfect globe.

Mathworks now give out student licenses for MOOCs. At least they did for some Udacity courses I took earlier this year.

Cvx is basically a convex solver that is called from matlab. Boyd wrote it. I'm not sure how easy it would be to divorce it from matlab, but there is a lot of existing instructional material built off of cvx, including examples in Boyd's book on convex optimization.

If you think you might be interested in this topic, it's worthwhile getting a matlab license. A lot of published academic code in this field is implemented with cvx.

From the web page:

Do we need to purchase a Matlab license to take this course?

You will be able to use Matlab under a special license provided to you as a course participant to use for the CVX101 course. This is a limited license for the duration of the course and is intended to be used only for course work and not for commercial purposes. Alternatively, you may choose to access Matlab via a web interface if you prefer not to install Matlab on your computer.

It's incredible how many of my colleagues work with Matlab.

Many enjoy the high integration of the environment. Thus it's somehow predestined for prototyping. I'm not sure if Octave can compete, considering the GUI features. However, python, especially ipython, is a real alternative to me. In fact, I use ipython for my prototyping which evolves into Fortran for the following implementation on the high performance cluster late on.

As one of the other commenters mentioned, Boyd or -- more likely -- his PhD/u/grads wrote CVX, so I'm not sure how much this course allows you to implement and play with ideas presented using python instead of Matlab.