Week 1 
9/21 
Lecture 1 

Class Notes

9/21 
Assignment 
Problem Set 0 released.

9/23 
Lecture 2 
 Supervised learning setup. LMS.

Class Notes
 Supervised Learning [pdf] (Sections 13)

9/23 
Assignment 
Problem Set 1 will be released. Due Thursday, 10/7 at 11:59pm

Week 2 
9/28 
Lecture 3 
 Weighted Least Squares. Logistic regression. Newton's Method.

Class Notes
 Supervised Learning [pdf] (Sections 4, 5, and 7)

9/30 
Lecture 4 
 Dataset split; Exponential family. Generalized Linear Models.

Class Notes
 Supervised Learning [pdf] (Sections 6, 8, and 9)
 Live Lecture Notes [pdf]

10/1 
Section 1 
 Friday TA Lecture: Linear Algebra Review.

Notes
 Linear Algebra Review and Reference [pdf]
 Linear Algebra, Multivariable Calculus,
and Modern Applications (Stanford Math 51 course text) [pdf]
 Friday Section Slides [pdf]

10/1 
Project 
Project proposal due 10/1 at 11:59pm. 
Week 3 
10/5 
Lecture 5 
 Gaussian discriminant analysis. Naive Bayes.

Class Notes
 Generative Algorithms [pdf] (Section 1)
 Live Lecture Notes [pdf]

10/7 
Lecture 6 
 Naive Bayes, Laplace Smoothing.

Class Notes
 Naive Bayes and Laplace Smoothing [pdf] (Section 2)
 Live Lecture Notes [pdf]

10/7 
Assignment 
Problem Set 2 will be released. Due Thursday, 10/21 at 11:59pm

10/8 
Section 2 
 Friday TA Lecture: Probability Theory Review.

Notes
 Probability Theory Review [pdf]
 The Multivariate Gaussian Distribution [pdf]
 More on Gaussian Distributions [pdf]
 Friday Section Slides [pdf]

Week 4 
10/12 
Lecture 7 

Class Notes
 Kernel Methods [pdf]
 Live Lecture Notes [pdf]

10/14 
Lecture 8 

Class Notes
 Deep Learning [pdf]
 Live Lecture Notes [pdf]

10/15 
Section 3 
 Friday TA Lecture: Python/Numpy Tutorial.

Notes
 Python Review Code
 Friday Section Slides [pdf]

Week 5 
10/19 
Lecture 9 
 Neural Networks 2. Backpropagation.

Class Notes
 Deep Learning [pdf]
 Live Lecture Notes [pdf]

10/21 
Lecture 10 
 Bias  Variance. Regularization. Feature / Model selection.

Class Notes
 Regularization and Model Selection [pdf]
 Some Calculations from Bias Variance (Addendum) [pdf]
 BiasVariance and Error Analysis (Addendum) [pdf]
 Lecture slides: bias and variance [pdf]
 Lecture slides: lasso regression [pdf]
 Lecture slides: ridge regression [pdf]

10/21 
Assignment 
Problem Set 3 will be released. Due Thursday, 11/4 at 11:59pm

10/22 
Section 4 
 Friday TA Lecture: Midterm Review.

Notes
 Friday Section Slides [pdf]

10/22 
Project 
Project milestones due 10/22 at 11:59pm. 
Week 6 
10/26 
Lecture 11 

Class Notes
 Decision trees slides [pdf]
 Overfitting decision trees slides [pdf]
 The EM Algorithm slides [pdf]

10/28 
Lecture 12 
 KMeans. GMM (non EM). Expectation Maximization. Factor Analysis.

Class Notes
 Unsupervised Learning, kmeans clustering. [pdf]
 Mixture of Gaussians [pdf]
 The EM Algorithm [pdf]
 Factor Analysis [pdf]
 Kmeans slides [pdf]

10/28 
Midterm 
For midterm details, please see this post on Ed

10/29 
Section 5 
 Friday TA Lecture: Evaluation Metrics.

Notes
 Friday Section Slides [pdf]

Week 7 
11/2 
Lecture 13 

Class Notes
 The EM Algorithm [pdf]
 Principal Components Analysis [pdf]
 Independent Component Analysis [pdf]
 EM Slides [pdf]
 GMM Slides [pdf]

11/4 
Lecture 14 

Class Notes

11/4 
Assignment 
Problem Set 4 will be released. Due Saturday, 11/20 at 11:59pm

11/5 
Section 6 
 Friday TA Lecture: Deep Learning (ConvNets).

Notes

Week 8 
11/9 
Lecture 15


Class Notes

11/11 
Lecture 16 
 Unsupervised learning. Reinforcement learning.

Class Notes

11/12 
Section 7 
 Friday TA Lecture: Ensembling Techniques

Notes
 Ensembling Techniques [pdf]

Week 9 
11/16 
Lecture 17 
 Basic concepts in RL, value iteration, policy iteration.

Class Notes
 Basic RL concepts, value iterations, policy iteration [pdf] (Sections 1 and 2)
 Live Lecture Notes [pdf]

11/18 
Lecture 18 
 Modelbased RL, value function approximator.

Class Notes
 Modelbased RL and value function approximation [pdf] (Sections 3 and 4)
 Live Lecture Notes (Spring 2021) [pdf]

11/19 
Section 8 
 Friday TA Lecture: On Critiques of ML.

Class Notes

Thanksgiving Break 
Week 10 
11/30 
Lecture 19 
 Fairness, algorithmic bias, explainability, privacy


12/2 
Lecture 20 
 Fairness, algorithmic bias, explainability, privacy


12/2 
Project 
Project final report + poster due 12/2 at 11:59pm. 
12/3 
Section 9 
 Friday TA Lecture: Learning Theory.

Class Notes

Other Resources
 All lecture videos can be accessed through Canvas.
 Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
 Previous projects: A list of last year's final projects can be found here.
 Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
 Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
 Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
 The Matrix Cookbook: quick reference for matrix identities, approximations, relations, etc.
