ECE 901: Learning From Small Data

Special topics course offered in Fall 2021 at UW-Madison

Instructor: Ramya Korlakai Vinayak, email: ramya[at]ece[dot]wisc[dot]edu

Class time: T/Th 9:30AM--10:45AM CT. Location: ENGR HALL 2540.

Course Description: In this course, we will explore the modern topics in machine learning with a focus on learning from small data. This course will have two parts. In the first part, we will focus on building the core theoretical tools to understand advanced ML topics, like application of polynomial approximations, method of moments, concentration inequalities, and convex geometry (in particular linear and semi-definite programs). We will also define and compare the use of KL-divergence, total variation distance, and Wasserstein-p distances; learn how to use hypothesis testing problem set-up to derive lower bounds in learning problems. In the second part of the course, we will apply these tools to understand in-depth a collection of selected recent papers on learning from sparse and small data.

Requirements: Probability e.g., ECE 730, STAT 709 or equivalent; Linear Algebra MATH 541 or equivalent; OR instructor’s permission.

Learning Outcomes:

• Become comfortable delving into details of machine learning papers with theoretical analysis.
• Learn to identify the main tools used in analysis that involve long proofs.
• Learn to critically read and review research papers.
• Understand how core topics in machine learning theory, optimization, statistics and signal processing are used to derive modern results in machine learning.
• Work together with peers, learn to discuss proof details and learn how to provide constructive feedback to others.