Course Overview

Course Code: SDSC6015

Course Name: Stochastic Optimization for Machine Learning

Semester: 2025-26, Semester A

Instructor: Dr. Lu Yu

Email: lu.yu@cityu.edu.hk

Office: LAU 16-279

Lecture Time: Thursday 19:00 – 21:50
Lecture Location: CMC Building, M3017
Office Hours: To Be Announced (TBA)
Teaching Assistants:
- Xinnian Yang (xinniyang2-c@my.cityu.edu.hk)
- Zhiyou Wu (zhiyouwu2-c@my.cityu.edu.hk)

Assessment Scheme

Component	Weight	Details
Assignments	30%	3 assignments, 10% each. Collaboration with up to 2 classmates is allowed after individual attempt, but solutions must be written individually. Late submissions incur a 10% penalty per day (up to 3 days).
Assessment Option 1	30%	Midterm Exam (30%)
Assessment Option 2	30%	Course Project (10%) + Midterm Exam (20%)
Final Exam	40%	Date TBD. Closed-book; two double-sided A4 cheat sheets are allowed.

Note: Students can choose between Assessment Option 1 or Option 2.

Week	Date	Topic	Note
Week 1	Sep 4	Introduction / Preliminaries of Stochastic Optimization
Week 2	Sep 11	Convex Function / Convex Optimization Problems
Week 3	Sep 18	Gradient Descent / Projected/Proximal Gradient Descent	Assignment 1 released (Sep 18), due Oct 1
Week 4	Sep 25	Subgradient Descent / Mirror Descent
Week 5	Oct 2	Stochastic Gradient Descent (SGD) / Momentum-based Methods
Week 6	Oct 9	Midterm Exam
Week 7	Oct 16	Adaptive Learning Rates Methods / Other Variants of SGD	Assignment 2 released (Oct 16), due Oct 29
Week 8	Oct 23	Coordinate Descent / Newton Method, Quasi-Newton Method
Week 9	Oct 30	Zero-th Order Optimization / Parallel and Distributed Optimization
Week 10	Nov 6	Nonconvex Optimization / Applications in Machine Learning
Week 11	Nov 13	Robust and Adversarial Optimization	Assignment 3 released (Nov 13), due Nov 26
Week 12	Nov 20	Advanced Topics / Final Exam Review
Week 13	Nov 27	Recap
Final Exam	TBA

Midterm Exam Date: October 9 (Thursday)
Final Exam Date: TBD
Grading Concerns: Requests for re-evaluation of graded work must be made within one week of the grade release date.
Missed Tests: If a test is missed for a valid reason (e.g., medical reasons), documentation must be submitted via AIMS and the instructor notified immediately. If approved for the midterm, the final exam weight will be adjusted to 70%. Prior instructor approval is required for non-medical reasons.

Required Textbook: None.
Recommended References:
- Convex Optimization: Algorithms and Complexity, by Sébastien Bubeck
- Convex Optimization, by Stephen Boyd and Lieven Vandenberghe
- Introductory Lectures on Convex Optimization, by Yurii Nesterov
- First-Order and Stochastic Optimization Methods for Machine Learning, by Guanghui Lan
Course Website: All course information, readings, assignments, and announcements will be posted on Canvas.