Statistical Machine Learning

Statistical foundations of machine learning for undergraduate and graduate students.

Course Schedule

Session Day Date Topic Details References
1 Sat 2 Esfand Mathematical Foundations Course overview and mathematical review
2 Mon 4 Esfand
3 Sat 9 Esfand
4 Mon 11 Esfand
5 Sat 16 Esfand Introduction to Supervised Learning Input/output data, data types, numerical encoding, model definition, model complexity, parametric vs. nonparametric models, loss functions, bias–variance tradeoff ESL 2.2–2.3
6 Mon 18 Esfand Linear Models Simple linear regression, coefficient estimation, inference, model assessment ISL 3.1
7 Sat 23 Esfand Linear Models Multiple linear regression, coefficient estimation ISL 3.2
8 Mon 25 Esfand Linear Models Nonlinear feature expansion, model limitations ESL 5.1; ISL 3.3, 7.1–7.3
9 Sat 15 Farvardin Linear Classification Logistic regression ISL 4.1–4.3
10 Mon 17 Farvardin Generative Models Linear Discriminant Analysis (LDA) ISL 4.4–4.5
11 Sat 22 Farvardin Model Evaluation & Selection Bias–variance tradeoff, model complexity ESL 7.2–7.5
12 Mon 24 Farvardin Model Selection Subset selection methods ESL 3.3
13 Sat 29 Farvardin Regularization Ridge regression (L2) ESL 3.4
14 Mon 31 Farvardin Regularization Lasso regression (L1) ISL 3.4
15 Sat 5 Ordibehesht Complexity Estimation Cross-validation (K-fold), Bootstrap ESL 7.10–7.11
16 Mon 7 Ordibehesht Tree-Based Models Decision trees and regression trees ISL 8.1; ESL 9.2
17 Sat 12 Ordibehesht Tree-Based Models Pruning, strengths and weaknesses ISL 8.1; ESL 9.2
18 Mon 14 Ordibehesht Support Vector Machines Maximum margin classifier, SVC, SVM ESL 12.1–12.3; ISL 9.1–9.3
19 Sat 19 Ordibehesht Support Vector Machines Kernel trick, multiclass SVM ESL 12.3; ISL 9.4–9.5
20 Mon 21 Ordibehesht Neural Networks Perceptron and neural network foundations ISL 10.1–10.2
21 Sat 26 Ordibehesht Neural Networks Multilayer perceptron (MLP) and architectures ISL 10.3–10.5
22 Mon 28 Ordibehesht Optimization Stochastic Gradient Descent (SGD) ESL 11.4
23 Sat 2 Khordad Ensemble Methods Bagging, Random Forest ESL 15.1–15.4
24 Mon 4 Khordad Ensemble Methods AdaBoost ESL 10.1–10.4
25 Sat 9 Khordad Ensemble Methods Gradient Boosting ESL 10.10
26 Mon 11 Khordad Unsupervised Learning K-means, Gaussian Mixture Models (GMM), Soft K-means ESL 14.3
27 Sat 16 Khordad Dimensionality Reduction Principal Component Analysis (PCA) ESL 14.4
28 Mon 18 Khordad Dimensionality Reduction Independent Component Analysis (ICA) ESL 14.5
29 Sat 23 Khordad Nonlinear Manifold Learning Isomap ESL 14.9
30 Mon 25 Khordad Reserved
31 Sat 30 Khordad Reserved
32 Mon 1 Tir Reserved

Final Exam

15 Tir – 14:00