CS 598 PEN

Subject offerings of new and developing areas of knowledge in computer science intended to augment the existing curriculum. See Class Schedule or departmental course information for topics and prerequisites. Course Information: May be repeated in the same or separate terms if topics vary.

Recent progress in open-source pretrained large language models (LLMs) have opened up new exciting opportunities for researchers to explore creative ideas, even when they may lack extensive resources for pretraining. This course delves into them through lectures and student-led discussions. We will cover continual pretraining, instruction fine-tuning, preference learning, alignment, efficiency optimization, evaluation, and so on. Though this course is primarily designed for graduate students, motivated undergraduates with suitable backgrounds are also welcome. Prior research experience in related fields (such as natural language processing, machine learning, vision, etc.), strong skills for paper reading and presentation, proficiency in Python and modern deep learning frameworks are assumed. For up-to-date information about CS course restrictions, please view the following link for restrictions and release dates: http://go.cs.illinois.edu/csregister.