Syllabus and Course Schedule

[Previous offerings: Fall 2020, Spring 2020]


This table will be updated regularly through the quarter to reflect what was covered, along with corresponding readings and notes.
DateEventDescriptionMaterials and Assignments
3/29 Lecture 1
  • Introduction.
Class Notes
3/31 Lecture 2
  • Supervised learning setup. LMS.
Class Notes
  • Supervised Learning[pdf](Sections 1-3)
  • Live Lecture Notes (draft)[pdf]
3/31 Assignment Problem Set 0 released.
4/2 Section 1
  • Friday TA Lecture: Linear Algebra Review.
Notes
  • Linear Algebra Review and Reference [pdf]
  • Linear Algebra, Multivariable Calculus,
    and Modern Applications
    (Stanford Math 51 course text) [pdf]
  • Friday Section Slides [pdf]
4/5 Lecture 3
  • Weighted Least Squares. Logistic regression. Newton's Method.
Class Notes
  • Supervised Learning [pdf] (Sections 4, 5, and 7)
  • Live Lecture Notes (draft)[pdf]
4/7 Lecture 4
  • Dataset split; Exponential family. Generalized Linear Models.
Class Notes
  • Supervised Learning [pdf] (Sections 6, 8, and 9)
  • Live Lecture Notes (draft)[pdf]
4/7 Assignment Problem Set 1 will be released. Due Wednesday, 4/21 at 11:59pm
4/9 Section 2
  • Friday TA Lecture: Probability Theory Review.
Notes
  • Probability Theory Review [pdf]
  • The Multivariate Gaussian Distribution [pdf]
  • More on Gaussian Distributions [pdf]
  • Friday Section Slides [pdf]
4/12 Lecture 5
  • Gaussian discriminant analysis. Naive Bayes.
Class Notes
  • Generative Algorithms [pdf] (Section 1)
  • Live Lecture Notes [pdf]
4/14 Lecture 6
  • Naive Bayes, Laplace Smoothing.
Class Notes
  • Naive Bayes and Laplace Smoothing [pdf] (Section 2)
  • Live Lecture Notes [pdf]
4/16 Project Project proposal due 4/16 at 11:59pm.
4/16 Section 3
  • Friday TA Lecture: Python/Numpy Tutorial.
Notes
  • Python Review Code[pdf, source]
  • Friday Section Slides [pdf]
4/19 Lecture 7
  • Kernels. SVM.
Class Notes
  • Kernel Methods [pdf]
  • Live Lecture Notes [pdf]
4/21 Lecture 8
  • Neural Networks 1.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
4/21 Assignment Problem Set 2 will be released. Due Wednesday, 5/5 at 11:59pm
4/23 Section 4
  • Friday TA Lecture: Evaluation Metrics.
Notes
  • Friday Section Slides [pdf]
4/26 Lecture 9
  • Neural Networks 2. Backpropagation.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
4/28 Lecture 10
  • Bias - Variance. Regularization. Feature / Model selection.
Class Notes
  • Regularization and Model Selection [pdf]
  • Some Calculations from Bias Variance (Addendum) [pdf]
  • Bias-Variance and Error Analysis (Addendum) [pdf]
  • Live Lecture Notes [pdf]
4/30 Section 5
  • Friday TA Lecture: Deep Learning (ConvNets).
Notes
  • Friday Section Slides [pdf, ppt]
5/3 Lecture 11
  • K-Means. GMM (non EM). Expectation Maximization.
Class Notes
  • Unsupervised Learning, k-means clustering. [pdf]
  • Mixture of Gaussians [pdf]
  • The EM Algorithm [pdf]
  • Live Lecture Notes (draft)[pdf]
5/5 Lecture 12
  • GMM (EM). Factor Analysis.
Class Notes
  • Lagrange Multipliers Review [pdf]
  • Factor Analysis [pdf]
  • Live Lecture Notes (draft)[pdf]
  • Addendum Notes[pdf]
5/5 Assignment Problem Set 3 will be released. Due Wednesday, 5/19 at 11:59pm
5/7 Project Project milestones due 5/7 at 11:59pm.
5/7 Section 6
  • Friday TA Lecture: Midterm Review.
5/10 Lecture 13
  • Factor Analysis and PCA.
Class Notes
  • Principal Components Analysis [pdf]
  • Independent Component Analysis [pdf]
  • Live Lecture Notes (draft)[pdf]
5/12 Lecture 14
  • Weak supervised / unsupervised learning.
Class Notes
  • Introduction to weak supervision [slides]
  • ICA and weak supervision [draft]
5/13 Midterm The midterm details TBD.
5/14 Section 7
  • Friday TA Lecture: Decision Trees + Boosting.
Notes
5/17 Lecture 15
  • ML advice.
5/19 Lecture 16
  • ML Advice. Self-supervised learning (Language Models & Image Models).
5/19 Assignment Problem Set 4 will be released. Due Wednesday, 5/26 at 11:59pm
5/21 Section 8
  • Friday TA Lecture: On Critiques of ML.
5/24 Lecture 17
  • Basic concepts in RL, value iteration, policy iteration.
Class Notes
  • Basic RL concepts, value iterations, policy iteration [pdf] (Sections 1 and 2)
5/26 Lecture 18
  • Model-based RL, value function approximator.
Class Notes
  • Model-based RL and value function approximation [pdf] (Sections 3 and 4)
5/28 Section 9
  • Friday TA Lecture: Learning Theory.
6/2 Lecture 19
  • Societal impact.
6/2 Project Project final report + poster (optional) due 6/2 at 11:59pm.
Other Resources
  1. All lecture videos can be accessed through Canvas.
  2. Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
  3. Previous projects: A list of last year's final projects can be found here.
  4. Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
  5. Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
  6. Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.