[CSED490X] Recent Trends in Machine Learning: A Large-Scale Perspective

Instructor: Jungtaek Kim (Email: jtkim [at] postech [dot] ac [dot] kr)

Class time: Wednesday 15:30 - 16:20

Teaching assistant: Dayoung Kong (Email: dayoung.kong [at] postech [dot] ac [dot] kr)

---

Schedule

Date	Contents	Materials
February 23, 2022	Course introduction Brief introduction on artificial intelligence, machine learning, and deep learning Brief introduction on automatic differentiation	Course Introduction
March 02, 2022	Machine learning Three ingredients: data, a machine learning model, and a learning algorithm Integration of all ingredients	Basics of Machine Learning
March 09, 2022	No lecture due to presidential election day
March 16, 2022	Large-scale datasets Automatic differentiation frameworks Machine learning accelerators	Basics of Machine Learning Tools
March 23, 2022	Transformer (https://arxiv.org/abs/1706.03762)	Transformer
March 30, 2022	BERT (https://arxiv.org/abs/1810.04805)	BERT
April 06, 2022	GPT (https://openai.com/blog/language-unsupervised/) GPT-2 (https://openai.com/blog/better-language-models/) GPT-3 (https://arxiv.org/abs/2005.14165)	A Series of GPTs
April 13, 2022	No lecture due to midterm exams period
April 20, 2022	XLNet (https://arxiv.org/abs/1906.08237) RoBERTa (https://arxiv.org/abs/1907.11692) DistilBERT (https://arxiv.org/abs/1910.01108) T5 (https://arxiv.org/abs/1910.10683) Turing-NLG (Microsoft Blog)	XLNet, RoBERTa, DistilBERT, T5, Turing-NLG
April 27, 2022	Vision Transformer (https://arxiv.org/abs/2010.11929) Swin Transformer (https://arxiv.org/abs/2103.14030)	Vision Transformer & Swin Transformer
May 4, 2022	Special Lecture by Kakao Brain CLIP	Special Lecture 1 by Kakao Brain
May 11, 2022	Special Lecture by Kakao Brain DALL·E	Special Lecture 2 by Kakao Brain
May 18, 2022	Special Lecture by Kakao Brain DALL·E 2	Special Lecture 3 by Kakao Brain
May 25, 2022	Ethical issues in large-scale models Environmental issues in large-scale models	Ethical and Environmental Issues in Large-Scale Models
June 01, 2022	No lecture due to local election day
June 08, 2022	No lecture due to final exams

Last updated: May 25, 2022

---