Introduction

The course covers several basic and advanced topics in convex optimization. The goal of this course is to recognize and formulate problems as convex optimization problems. There will be an emphasis on developing algorithms for moderate as well as large size problems. The course provides insights that can be used in a variety of disciplines including signal and image processing, machine learning, and control systems.

The course treats:

  • Background and optimization basics;
  • Convex sets and functions;
  • Canonical convex optimization problems (LP, QP, SDP);
  • First-order methods (gradient, subgradient);
  • Second-order methods (unconstrained and constrained optimization);
The course includes a take-home lab assignment (size 1 EC = 28 hours) which can be done in groups of 2 students. Several track-dependent assignments are offered.

Preliminary knowledge

To follow the course with profit, you will need the working knowledge of linear algebra and calculus with functions in multiple variables.

Exam

The exam will be a written open-book exam. Please register yourself in Osiris for taking part in the exam. For the exam, you can bring the book (or a print-out of the pdf), copies of the slides, and a cheat sheet (1 page). No other written notes or materials are allowed.

The lab assignment is completed with a compact report and 10 minute presentation. During the last two lectures, the students are expected to present their project to their colleague students. Passing the lab assignment is compulsory for the exam grade to become valid. Moreover, the assignment is graded and counts for 20% of your final grade.

Projects

The course contains a compulsory lab assignment worth 1 EC (28 hours, 20% of your final grade). The assignment is done in groups of 2 students.

The deadline for submitting the reports is January 12, 2025. This deadline is "firm" and no deadline extensions will be granted. If you don't submit your report within this deadline you will not be allowed to present your project and you cannot pass this course.

Signing up for the lab assignment has to be done via Brightspace. To enroll, go to the "Collaboration" tab in Brightspace, and then select groups. Signing up can be done until December 2, 2024. To upload your report, go to assignments in Brightspace.

Project 1: Change Detection in Time Series Model. Dataset

Project 2: Linear Support Vector Machines. Dataset

Project 3: Multidimensional Scaling for Localization. Dataset

Project 4: MIMO Detection. Dataset

Project 5: Compressed Sensing. Dataset.

Book

Stephen Boyd and Lieven Vandenberghe, "Convex Optimization", Cambridge University Press, 2004. The pdf version of this book is freely available and it can be found online here.

Slides are based on Convex Optimization course ee364a offered at Stanford University by Prof. Boyd

Instructors

prof.dr.ir. Geert Leus (GL), dr.ir. Borbala (Bori) Hunyadi (BH) and ir. Ids van der Werf (IW).

Office hours Ids van der Werf:

Location: EEMCS high-rise 17.090

Dates:

- Thursday November 21st 1pm-2pm

- Friday November 29th 1pm-2pm

- Thursday December 5th 1pm-2pm

- Thursday December 12th 1pm-2pm

- Thursday December 19th 1pm-2pm

- Thursday January 9th 1pm-2pm

- Thursday January 16th 1pm-2pm

- Friday January 24th 1pm-2pm

Online Lectures

The Collegerama recordings can be access here.

Schedule

The schedule for 2023-2024 is as follows. Classes are on Wednesdays (15.45 - 17.30) and Fridays (10.45 - 12.30).


Date

Book Slides Video
1. Wed 13 Nov GL Introduction (functions, sets, optimization basics) Ch.1 and Appendix A of the textbook Linear algebra slides Ch.1 slides Lect. 1
2. Fri 15 Nov BH Convex sets and functions Ch.2 and Ch. 3 Ch.2 and 3 slides Lect. 2
3. Wed 20 Nov IW Convex sets and functions Ch.2 and Ch. 3 Ch.2 and 3 slides Lect. 3
4. Fri 22 Nov GL Canonical problems (LP, QP, SDP) Ch.4 Ch.4 slides Lect. 4
5. Wed 27 Nov GL Duality Ch. 5 Ch.5 slides Lect. 5
6. Fri 29 Nov BH Unconstrained minimization Ch. 9.1-9.5 Ch.9 slides Lect. 6
7. Wed 4 Dec GL Constrained minimization

Ch. 10.1, 10.2, 11.1, 11.2

Ch.10 and Ch. 11 slides Lect. 7
8. Fri 6 Dec IW Convex-cardinality problems Cardinality slides Lect. 8
9. Wed 11 Dec BH Subgradient methods Subgradients

Subgradient methods

Subgradient methods slides Lect. 9
10. Fri 13 Dec BH Subgradient methods Subgradients

Subgradient methods

Subgradient methods slides Lect. 10
11. Wed 18 Dec IW Exercises Exercises slides Matlab codes Lect. 11
12. Wed 8 Jan GL/BH/IW Exercises Lect. 12
13. Fri 10 Jan GL/BH/IW Q&A
Sun 12 Jan Deadline lab assignment report
15. Wed 15 Jan GL/BH/IW Projects Online
16. Fri 17 Jan GL/BH/IW Projects Online
Tue 28 Jan 2024 Exam 13.30-16.30

Exercises

Note that the exams are open book, but you must be very familiar with the material to be able to solve the questions in time. Train by solving many exercise questions from the book.

The book (BV) contains many exercises. In addtion, some more excercises can be found here (AE). A pdf of the Solutions Manual can probably be found on the internet. Some suggested excercise problems can be found below.

Chapter 2: BV2.5; BV2.7; BV2.12; BV2.15; AE1.1; AE1.3
Chapter 3: BV3.2; BV3.15; BV3.16; BV3.18; BV3.58; AE2.6
Chapter 4: BV4.1; BV4.11; AE3.3; AE3.7; AE3.8; AE3.13
Chapter 5: BV5.1; BV5.7; BV5.29; BV5.30; AE4.10; AE4.15; AE4.16

Previous homework excercises

Here are the past homeworks of ee4530 although the content of ee4530 is different since 2016/17. The relevant ones can be used to train for the exam. The homeworks are, however, now replaced with the mini projects. So you don't have to turn them in.

Homework 1.

Homework 2.

Homework 3.

Homework 4.

Homework 5.

Previous exams

Solutions of April 2025.

Solutions of January 2025.

Solutions of April 2024.

Solutions of January 2024.

Solutions of April 2023.

Solutions of January 2023.

Solutions of April 2022.

Solutions of January 2022.