Lecture 1 |
6/24 |
- Introduction and Logistics
- Review of Linear Algebra
|
Class Notes
- Introduction [pptx]
- Linear Algebra (section 1-3) [pdf]
|
Lecture 2 |
6/26 |
- Review of Matrix Calculus
- Review of Probability
|
Class Notes
- Linear Algebra (section 4) [pdf]
- Probability Theory [pdf]
- Probability Theory Slides [pdf]
|
Lecture 3 |
6/28 |
- Review of Probability and Statistics
- Setting of Supervised Learning
|
Class Notes
- Supervised Learning [pdf]
- Probability Theory [pdf]
|
Lecture 4 |
7/1 |
- Linear Regression
- Gradient Descent (GD), Stochastic Gradient Descent (SGD)
- Normal Equations
- Probabilistic Interpretation
- Maximum Likelihood Estimation (MLE)
|
Class Notes
- Supervised Learning (section 1-3) [pdf]
|
Lecture 5 |
7/3 |
- Perceptron
- Logistic Regression
- Newton's Method
|
Class Notes
- Supervised Learning (section 5-7) [pdf]
|
Lecture 6 |
7/5 |
- Exponential Family
- Generalized Linear Models (GLM)
|
Class Notes
- Supervised Learning (section 8-9) [pdf]
|
Lecture 7 |
7/8 |
- Gaussian Discriminant Analysis (GDA)
- Naive Bayes
- Laplace Smoothing
|
Class Notes
- Generative Algorithms [pdf]
|
Lecture 8 |
7/10 |
- Kernel Methods
- Support Vector Machine
|
Class Notes
- Kernel Methods and SVM [pdf]
|
Lecture 9 |
7/12 |
|
Class Notes
Optional
- The Multivariate Gaussian Distribution [pdf]
- More on Gaussian Distribution [pdf]
|
Lecture 10 |
7/15 |
- Neural Networks and Deep Learning
|
Class Notes
- Deep Learning (skip Sec 3.3) [pdf]
Optional
|
Lecture 11 |
7/17 |
|
|
Lecture 12 |
7/19 |
- Bias and Variance
- Regularization, Bayesian Interpretation
- Model Selection
|
Class Notes
- Regularization and Model Selection [pdf]
|
Lecture 13 |
7/22 |
- Bias-Variance tradeoff (wrap-up)
- Uniform Convergence
|
Class Notes
- Bias Variance Analysis [pdf]
- Statistical Learning Theory [pdf]
|
Lecture 14 |
7/24 |
- Reinforcement Learning (RL)
- Markov Decision Processes (MDP)
- Value and Policy Iterations
|
Class Notes
- Reinforcement Learning and Control (Sec 1-2) [pdf]
|
Lecture 15 |
7/26 |
- RL (wrap-up)
- Learning MDP model
- Continuous States
|
Class Notes
- Reinforcement Learning and Control (Sec 3-4) [pdf]
|
Lecture 16 |
7/29 |
Unsupervised Learning
- K-means clustering
- Mixture of Gaussians (GMM)
- Expectation Maximization (EM)
|
Class Notes
- K-means [pdf]
- Mixture of Gaussians [pdf]
- Expectation Maximization (Sec 1-2, skip 2.1) [pdf]
|
Lecture 17 |
7/31 |
- EM (wrap-up)
- Factor Analysis
|
Class Notes
- Expectation Maximization (Sec 3) [pdf]
- Factor Analysis [pdf]
|
Lecture 18 |
8/2 |
- Factor Analysis (wrap-up)
- Principal Components Analysis (PCA)
- Independent Components Analysis (ICA)
|
Class Notes
- Principal Components Analysis [pdf]
- Independent Components Analysis [pdf]
|
Lecture 19 |
8/5 |
- Maximum Entropy and Exponential Family
- KL-Divergence
- Calibration and Proper Scoring Rules
|
Class Notes
|
Lecture 20 |
8/7 |
- Variational Inference
- EM Variants
- Variational Autoencoder
|
Class Notes
|
Lecture 21 |
8/9 |
|
Class Notes
- Evaluation Metrics [pptx]
|
Lecture 22 |
8/12 |
- Practical advice and tips
- Review for Finals
|
Class Notes
|
Lecture 23 |
8/14 |
|
Class Notes
|
Final |
8/16 |
|
|
Other Resources
- Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
- Previous projects: A list of last year's final projects can be found here.
- Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
- Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
- Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
|