Syllabus and Course Schedule

[Previous offerings: Spring 2021, Fall 2020]


This table will be updated regularly through the quarter to reflect what was covered, along with corresponding readings and notes.
DateEventDescriptionMaterials and Assignments
Week 1
9/21 Lecture 1
  • Introduction.
Class Notes
9/21 Assignment Problem Set 0 released.
9/23 Lecture 2
  • Supervised learning setup. LMS.
Class Notes
  • Supervised Learning [pdf] (Sections 1-3)
9/23 Assignment Problem Set 1 will be released. Due Thursday, 10/7 at 11:59pm
Week 2
9/28 Lecture 3
  • Weighted Least Squares. Logistic regression. Newton's Method.
Class Notes
  • Supervised Learning [pdf] (Sections 4, 5, and 7)
9/30 Lecture 4
  • Dataset split; Exponential family. Generalized Linear Models.
Class Notes
  • Supervised Learning [pdf] (Sections 6, 8, and 9)
  • Live Lecture Notes [pdf]
10/1 Section 1
  • Friday TA Lecture: Linear Algebra Review.
Notes
  • Linear Algebra Review and Reference [pdf]
  • Linear Algebra, Multivariable Calculus,
    and Modern Applications
    (Stanford Math 51 course text) [pdf]
  • Friday Section Slides [pdf]
10/1 Project Project proposal due 10/1 at 11:59pm.
Week 3
10/5 Lecture 5
  • Gaussian discriminant analysis. Naive Bayes.
Class Notes
  • Generative Algorithms [pdf] (Section 1)
  • Live Lecture Notes [pdf]
10/7 Lecture 6
  • Naive Bayes, Laplace Smoothing.
Class Notes
  • Naive Bayes and Laplace Smoothing [pdf] (Section 2)
  • Live Lecture Notes [pdf]
10/7 Assignment Problem Set 2 will be released. Due Thursday, 10/21 at 11:59pm
10/8 Section 2
  • Friday TA Lecture: Probability Theory Review.
Notes
  • Probability Theory Review [pdf]
  • The Multivariate Gaussian Distribution [pdf]
  • More on Gaussian Distributions [pdf]
  • Friday Section Slides [pdf]
Week 4
10/12 Lecture 7
  • Kernels. SVM.
Class Notes
  • Kernel Methods [pdf]
  • Live Lecture Notes [pdf]
10/14 Lecture 8
  • Neural Networks 1.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
10/15 Section 3
  • Friday TA Lecture: Python/Numpy Tutorial.
Notes
  • Python Review Code
  • Friday Section Slides [pdf]
Week 5
10/19 Lecture 9
  • Neural Networks 2. Backpropagation.
Class Notes
  • Deep Learning [pdf]
  • Live Lecture Notes [pdf]
10/21 Lecture 10
  • Bias - Variance. Regularization. Feature / Model selection.
Class Notes
  • Regularization and Model Selection [pdf]
  • Some Calculations from Bias Variance (Addendum) [pdf]
  • Bias-Variance and Error Analysis (Addendum) [pdf]
  • Lecture slides: bias and variance [pdf]
  • Lecture slides: lasso regression [pdf]
  • Lecture slides: ridge regression [pdf]
10/21 Assignment Problem Set 3 will be released. Due Thursday, 11/4 at 11:59pm
10/22 Section 4
  • Friday TA Lecture: Midterm Review.
Notes
  • Friday Section Slides [pdf]
10/22 Project Project milestones due 10/22 at 11:59pm.
Week 6
10/26 Lecture 11
  • Decision Trees. Boosting
Class Notes
  • Decision trees slides [pdf]
  • Overfitting decision trees slides [pdf]
  • The EM Algorithm slides [pdf]
10/28 Lecture 12
  • K-Means. GMM (non EM). Expectation Maximization. Factor Analysis.
Class Notes
  • Unsupervised Learning, k-means clustering. [pdf]
  • Mixture of Gaussians [pdf]
  • The EM Algorithm [pdf]
  • Factor Analysis [pdf]
  • Kmeans slides [pdf]
10/28 Midterm For midterm details, please see this post on Ed
10/29 Section 5
  • Friday TA Lecture: Evaluation Metrics.
Notes
  • Friday Section Slides [pdf]
Week 7
11/2 Lecture 13
  • EM. GMM.
Class Notes
  • The EM Algorithm [pdf]
  • Principal Components Analysis [pdf]
  • Independent Component Analysis [pdf]
  • EM Slides [pdf]
  • GMM Slides [pdf]
11/4 Lecture 14
  • PCA. Types of learning.
Class Notes
11/4 Assignment Problem Set 4 will be released. Due Saturday, 11/20 at 11:59pm
11/5 Section 6
  • Friday TA Lecture: Deep Learning (ConvNets).
Notes
Week 8
11/9 Lecture 15
  • ML Advice
Class Notes
11/11 Lecture 16
  • Unsupervised learning. Reinforcement learning.
Class Notes
11/12 Section 7
  • Friday TA Lecture: Ensembling Techniques
Notes
  • Ensembling Techniques [pdf]
Week 9
11/16 Lecture 17
  • Basic concepts in RL, value iteration, policy iteration.
Class Notes
  • Basic RL concepts, value iterations, policy iteration [pdf] (Sections 1 and 2)
  • Live Lecture Notes [pdf]
11/18 Lecture 18
  • Model-based RL, value function approximator.
Class Notes
  • Model-based RL and value function approximation [pdf] (Sections 3 and 4)
  • Live Lecture Notes (Spring 2021) [pdf]
11/19 Section 8
  • Friday TA Lecture: On Critiques of ML.
Class Notes
Thanksgiving Break
Week 10
11/30 Lecture 19
  • Fairness, algorithmic bias, explainability, privacy
12/2 Lecture 20
  • Fairness, algorithmic bias, explainability, privacy
12/2 Project Project final report + poster due 12/2 at 11:59pm.
12/3 Section 9
  • Friday TA Lecture: Learning Theory.
Class Notes
  • Learning theory [pdf]
Other Resources
  1. All lecture videos can be accessed through Canvas.
  2. Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
  3. Previous projects: A list of last year's final projects can be found here.
  4. Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
  5. Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
  6. Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
  7. The Matrix Cookbook: quick reference for matrix identities, approximations, relations, etc.