CS 369L: Algorithmic Perspective on Machine Learning

Instructor: Moses Charikar (Office hours: by appointment, Gates 462.)

Time/location: 1:30-4:20pm on Wednesdays in Green Earth Sciences 150.

Discussion: We will use Piazza for discussion and announcements. Sign up here:
piazza.com/stanford/fall2015/cs369l

Course description:
Many problems in machine learning are intractable in the worst case, and pose a challenge for the design of algorithms with provable guarantees. In this course, we will discuss several success stories at the intersection of algorithm design and machine learning, focusing on devising appropriate models and mathematical tools to facilitate rigorous analysis.

Prerequisites:
Students will be expected to have a strong background in algorithms, probability and linear algebra.

Course requirements:
Two problem sets. Students taking the course for a letter grade will also do a research-oriented final project (either a survey or original research).

Tentative Syllabus
The following topics will be discussed in the class:
(This is intended to be a representative sample. New topics may be substituted to reflect current research in this area)

• Nonnegative matrix factorization
  • comparison to SVD, algorithms via separability, topic models
• Tensor Decomposition
  • tensor rank, border rank and the rotation problem, generalized eigenvalues, HMMs, community detection, cumulants and independent component analysis
• Sparse Recovery
  • uniqueness and uncertainty principles, pursuit algorithms, Prony’s method, compressed sensing
• Dictionary Learning
  • full rank dictionaries, overcomplete dictionaries
• Mixture Models and Clustering
  • expectation maximization, clustering in high dimensions, method of moments
• Matrix Completion
  • nuclear norm, atomic norm, convex relaxations

Reference:

• Ankur Moitra's course notes .

Related courses:

• Algorithmic Aspects of Machine Learning at MIT by Ankur Moitra: Fall 2013, Spring 2015