9/14 
Lecture 1



9/16 
Lecture 2

 Supervised learning setup. LMS.

Class Notes
 Supervised Learning [pdf](Sections 13)

9/16 
Assignment 
Problem Set 0 released. Due Tuesday, 9/22 at 11:59pm

9/19 
Section 1

 Friday TA Lecture: Linear Algebra Review.

Class Notes
 Review of Linear Algebra [pdf]
 Linear Algebra Review and Reference [pdf]
Prerequisite Reading
 Linear Algebra, Multivariable Calculus,
and Modern Applications (Stanford Math 51 course text) [pdf]

9/21 
Lecture 3

 Weighted Least Squares. Logistic regression. Newton's Method.

Class Notes
 Supervised Learning [pdf] (Sections 4, 5, and 7)

9/23 
Lecture 4

 Perceptron. Exponential family. Generalized Linear Models.

Class Notes
 Supervised Learning [pdf] (Sections 6, 8, and 9)

9/23 
Assignment 
Problem Set 1 will be released. Due Wednesday, 10/7 at 11:59pm

9/25 
Section 2

 Friday TA Lecture: Probability Theory Review.

Probability Theory Review [pdf]
The Multivariate Gaussian Distribution [pdf]
More on Gaussian Distribution [pdf]
Section slides [pdf]

9/28 
Lecture 5

 Gaussian discriminant analysis.

Class Notes
 Generative Algorithms [pdf] (Section 1)

9/30 
Lecture 6

 Naive Bayes, Laplace Smoothing.

Class Notes
 Naive Bayes and Laplace Smoothing [pdf] (Section 2)

10/2 
Section 3

 Friday TA Lecture: Python/Numpy Tutorial.

Slides [pdf]
Python Tutorial Notebook [link, jupyter notebook]

10/2 
Project 
Project proposal due 10/2 at 11:59pm. 
10/5 
Lecture 7


Class Notes

10/7 
Lecture 8


Class Notes

10/7 
Assignment 
Problem Set 2 will be released. Due Wednesday, 10/21 at 11:59pm

10/9 
Section 4

 Friday TA Lecture: Deep Learning.

Slides [pdf]

10/12 
Lecture 9

 Neural Networks 2. Backpropagation.

Class Notes

10/14 
Lecture 10

 Bias  Variance. Regularization. Feature / Model selection.

Class Notes
 Bias  Variance [pdf]
 Regularization and Model Selection [pdf]
 Some Calculations from Bias Variance (Addendum) [pdf]
 BiasVariance and Error Analysis (Addendum) [pdf]
 Double Descent (Optional Reading) [link]
 Hyperparmeter Tuning and Cross Validation [canvas video]

10/16 
Section 5

 Friday TA Lecture: Evaluation Metrics.

Slides [pdf]

10/19 
Lecture 11

 KMeans. GMM (non EM). Expectation Maximization.

Class Notes
 Unsupervised Learning, kmeans clustering. [pdf]
 Mixture of Gaussians [pdf]
 The EM Algorithm [pdf]
 Live lecture notes (spring quarter) [old draft]

10/21 
Lecture 12

 GMM (EM). Factor Analysis.

Class Notes
 Lagrange Multipliers Review [pdf]
 Factor Analysis [pdf]
 Live lecture notes [draft]

10/21 
Assignment 
Problem Set 3 will be released. Due Wednesday, 11/4 at 11:59pm

10/23 
Section 6

 Friday TA Lecture: Midterm Review.

Slides [pdf]

10/23 
Project 
Project milestones due 10/23 at 11:59pm. 
10/26 
Lecture 13


Class Notes
 Principal Components Analysis [pdf]
 Independent Component Analysis [pdf]
 Live lecture notes (spring quarter) [old draft, in lecture]

10/28 
Lecture 14

 Weak supervised / unsupervised learning.

Class Notes

10/29 
Midterm 
The midterm details TBD.

11/2 
Lecture 15


Class Notes

11/4 
Lecture 16

 Advice for applying machine learning.

Class Notes
 Advice for applying machine learning. [pdf]

11/4 
Assignment 
Problem Set 4 will be released. Due Wednesday, 11/18 at 11:59pm

11/9 
Lecture 17

 Basic RL concepts, value iterations, policy iteration.

Class Notes
 Basic RL concepts, value iterations, policy iteration [pdf] (Sections 1 and 2)

11/11 
Lecture 18

 Modelbased RL and value function approximation.

Class Notes
 Modelbased RL and value function approximation [pdf] (Sections 3 and 4)

11/16 
Lecture 19

 Policy search. REINFORCE.

Class Notes

11/18 
Lecture 20



11/18 
Project 
Project final report due 11/18 at 11:59pm. 
Other Resources
 All lecture videos can be accessed through Canvas.
 Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
 Previous projects: A list of last year's final projects can be found here.
 Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
 Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
 Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
