This is an old revision of the document!

====== Lecture: Convex Optimization and Online Learning (MM25) ====== ** Language: ** English or German, as the audience requests. ** Content: ** This lecture covers basic concepts of convex analysis and programming (classes of convex sets and functions, duality, nonexpansive mappings, splitting of convex programs, etc) with a focus on the analysis of iterative algorithms for solving large-scale convex optimization problems. Particular attention is paid to techniques for online convex optimization and the prediction of individual sequences in unknown environments, which play a key role in machine learning applications. The content of the lecture is targeted at students of mathematics and scientific computing with a long-term interest in machine learning, to prepare them for more advanced topics closer to research. ** Prerequisites: ** All proofs are elementary and only require knowledge from the mandatory undergraduate courses on analysis and linear algebra. ** Registration: ** If you wish to attend the lecture and the exercises, please sign up using [[https://muesli.mathi.uni-heidelberg.de/|MÜSLI]]. <color #ed1c24>You have to be logged in to access the files listed below.</color> **Lecture Notes: ** All statements (second part) will be proven in the lecture. //Convex Analysis and Programming// * {{ :teaching:ft1819:convex:convexfunctions.pdf |Smooth Convex Functions}} * shortcut: {{ :teaching:ft1819:convex:kkt.pdf |KKT Conditions}} * {{ :teaching:ft1819:convex:projection.pdf |Projection}} (update: 22.11.18), {{ :teaching:ft1819:convex:proxmaps.pdf |Proximal Mappings}} (update: 04.12.18) * {{ :teaching:ft1819:convex:convexnonsmooth.pdf |Nonsmooth Convex Functions}} * {{ :teaching:ft1819:convex:conjugation.pdf |Conjugation}}, {{ :teaching:ft1819:convex:conjugateduality.pdf |Conjugate Duality}}, {{ :teaching:ft1819:convex:lagrangianduality.pdf |Lagrangian Duality}} * {{ :teaching:ft1819:convex:fixedpointiterations.pdf |Fixed Point Iterations}} //Online Convex Optimisation and Learning// * {{ :teaching:ft1819:convex:online-introduction.pdf |Introduction}} * {{ :teaching:ft1819:convex:experts.pdf |Learning from Experts}} * {{ :teaching:ft1819:convex:co-offline.pdf |Basic Offline Convex Optimisation}} * {{ :teaching:ft1819:convex:co-online-fo.pdf |First-Order Online Convex Optimisation}} * {{ :teaching:ft1819:convex:co-online-regularized.pdf |Regularized Online Convex Optimisation and Learning}} //Miscellany// * {{ :teaching:ft1819:convex:loss-functions.pdf |Loss Functions}}, {{ :teaching:ft1819:convex:svd.pdf |SVD}} **Exercise Sheets ** * {{ :teaching:ft1819:convex:uebungsblatt1.pdf |Sheet 1}} (TBD 31.10) * {{ :teaching:ft1819:convex:uebungsblatt2.pdf |Sheet 2}} (TBD 07.11) * {{ :teaching:ft1819:convex:uebungsblatt3.pdf |Sheet 3}} (TBD 14.11, exercise 5: 21.11) / {{ :teaching:ft1819:convex:data_ex5.zip |data_ex5.zip}}, {{ :teaching:ft1819:convex:code_ex3.5.zip |code_ex3.5.zip}} * {{ :teaching:ft1819:convex:uebungsblatt4.pdf |Sheet 4}} (TBD 21.11) * {{ :teaching:ft1819:convex:uebungsblatt5.pdf |Sheet 5}} (TBD 28.11) / {{ :teaching:ft1819:convex:code_ex5.3.zip |code_ex5.3.zip}} * {{ :teaching:ft1819:convex:uebungsblatt6.pdf |Sheet 6}} (TBD 05.12) * {{ :teaching:ft1819:convex:uebungsblatt7.pdf |Sheet 7}} (TBD 12.12) / {{ :teaching:ft1819:convex:data_ex3.zip |data_ex3.zip }}, {{ :teaching:ft1819:convex:code_ex7.3.zip |code_ex7.3.zip}} * {{ :teaching:ft1819:convex:uebungsblatt8.pdf |Sheet 8}} (TBD 19.12, exercise 4: 09.01) / {{ :teaching:ft1819:convex:data_ex8.4.zip | data_ex4.zip}}, {{ :teaching:ft1819:convex:code_ex8.4.zip | code_ex8.4.zip}} * {{ :teaching:ft1819:convex:uebungsblatt9.pdf |Sheet 9}} (TBD 09.01) * {{ :teaching:ft1819:convex:uebungsblatt10.pdf |Sheet 10}} (TBD 16.01) * {{ :teaching:ft1819:convex:uebungsblatt11.pdf |Sheet 11}} (TBD 23.01) / {{ :teaching:ft1819:convex:data_ex11.2.zip | data_ex11.2.zip}}