Syllabus and Course Schedule

[Previous offerings: Spring 2021, Fall 2020]


This table will be updated regularly through the quarter to reflect what was covered, along with corresponding readings and notes.
DateEventDescriptionMaterials and Assignments
Week 1
9/21 Lecture 1
  • Introduction.
Class Notes
9/21 Assignment Problem Set 0 released.
9/23 Lecture 2
  • Supervised learning setup. LMS.
Class Notes
  • Supervised Learning [pdf] (Sections 1-3)
9/23 Assignment Problem Set 1 will be released. Due Thursday, 10/7 at 11:59pm
Week 2
9/28 Lecture 3
  • Weighted Least Squares. Logistic regression. Newton's Method.
Class Notes
  • Supervised Learning [pdf] (Sections 4, 5, and 7)
9/30 Lecture 4
  • Dataset split; Exponential family. Generalized Linear Models.
Class Notes
  • Supervised Learning [pdf] (Sections 6, 8, and 9)
  • Live Lecture Notes [pdf]
10/1 Section 1
  • Friday TA Lecture: Linear Algebra Review.
Notes
  • Linear Algebra Review and Reference [pdf]
  • Linear Algebra, Multivariable Calculus,
    and Modern Applications
    (Stanford Math 51 course text) [pdf]
  • Friday Section Slides [pdf]
10/1 Project Project proposal due 10/1 at 11:59pm.
Week 3
10/5 Lecture 5
  • Gaussian discriminant analysis. Naive Bayes.
Class Notes
  • Generative Algorithms [pdf] (Section 1)
  • Live Lecture Notes [pdf]
10/7 Lecture 6
  • Naive Bayes, Laplace Smoothing.
Class Notes
  • Naive Bayes and Laplace Smoothing [pdf] (Section 2)
  • Live Lecture Notes [pdf]
10/7 Assignment Problem Set 2 will be released. Due Thursday, 10/21 at 11:59pm
10/8 Section 2
  • Friday TA Lecture: Probability Theory Review.
Notes
  • Probability Theory Review [pdf]
  • The Multivariate Gaussian Distribution [pdf]
  • More on Gaussian Distributions [pdf]
  • Friday Section Slides [pdf]
Week 4
10/12 Lecture 7
  • Kernels. SVM.
Class Notes
  • Kernel Methods [pdf]
  • Live Lecture Notes [pdf]
10/14 Lecture 8
  • Neural Networks 1.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
10/15 Section 3
  • Friday TA Lecture: Python/Numpy Tutorial.
Notes
  • Python Review Code
  • Friday Section Slides [pdf]
Week 5
10/19 Lecture 9
  • Neural Networks 2. Backpropagation.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
10/21 Lecture 10
  • Bias - Variance. Regularization. Feature / Model selection.
Class Notes
  • Regularization and Model Selection [pdf]
  • Some Calculations from Bias Variance (Addendum) [pdf]
  • Bias-Variance and Error Analysis (Addendum) [pdf]
10/21 Assignment Problem Set 3 will be released. Due Thursday, 11/4 at 11:59pm
10/22 Section 4
  • Friday TA Lecture: Midterm Review.
Notes
  • Friday Section Slides [pdf]
10/22 Project Project milestones due 10/22 at 11:59pm.
Week 6
10/26 Lecture 11
  • K-Means. GMM (non EM). Expectation Maximization.
Class Notes
  • Unsupervised Learning, k-means clustering. [pdf]
  • Mixture of Gaussians [pdf]
  • The EM Algorithm [pdf]
10/28 Lecture 12
  • GMM (EM). Factor Analysis.
Class Notes
  • Factor Analysis [pdf]
10/28 Midterm The midterm details TBD.
10/29 Section 5
  • Friday TA Lecture: Evaluation Metrics.
Notes
Week 7
11/2 Lecture 13
  • PCA, ICA.
Class Notes
  • Principal Components Analysis [pdf]
  • Independent Component Analysis [pdf]
11/4 Lecture 14
  • Weak supervised / unsupervised learning.
Class Notes
  • Introduction to weak supervision
  • ICA and weak supervision
11/4 Assignment Problem Set 4 will be released. Due Thursday, 11/18 at 11:59pm
11/5 Section 6
  • Friday TA Lecture: Deep Learning (ConvNets).
Notes
Week 8
11/9 Lecture 15
  • Self-supervised learning (Language Models & Image Models).
Class Notes
  • Self-Supervised Learning
11/11 Lecture 16
  • ML Advice.
Class Notes
11/12 Section 7
  • Friday TA Lecture: Decision Trees + Boosting.
Notes
Week 9
11/16 Lecture 17
  • Basic concepts in RL, value iteration, policy iteration.
Class Notes
  • Basic RL concepts, value iterations, policy iteration [pdf] (Sections 1 and 2)
11/18 Lecture 18
  • Model-based RL, value function approximator.
Class Notes
  • Model-based RL and value function approximation [pdf] (Sections 3 and 4)
11/19 Section 8
  • Friday TA Lecture: On Critiques of ML.
Class Notes
Thanksgiving Break
Week 10
11/30 Lecture 19
  • Policy Search.
12/2 Lecture 20
  • Societal impact.
12/2 Project Project final report + poster due 12/2 at 11:59pm.
12/3 Section 9
  • Friday TA Lecture: Learning Theory.
Class Notes
  • Learning theory [pdf]
Other Resources
  1. All lecture videos can be accessed through Canvas.
  2. Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
  3. Previous projects: A list of last year's final projects can be found here.
  4. Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
  5. Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
  6. Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
  7. The Matrix Cookbook: quick reference for matrix identities, approximations, relations, etc.