Skip to main content

Natural Language Processing Fall 2025

Natural Language Processing (NLP) was traditionally a research field heavily reliant on partially supervised machine learning to tackle language tasks. However, the landscape shifted dramatically with the advent of large language models (LLMs), popularized by tools like ChatGPT. Unlike earlier models, LLMs are trained using self-supervised learning, exhibiting remarkable emergent behavior and tackling a wide range of tasks they were never explicitly trained on. This demonstrates that unsupervised learning is scalable and capable of achieving zero-shot performance, where models perform tasks with little to no task-specific examples.

This course delves into language models and representation learning for NLP with a focus on large language models. We will explore key components such as model architecture, effective training strategies, and inference techniques, highlighting their applications across diverse natural language processing tasks. As NLP rapidly evolves, LLMs have become a cornerstone of artificial intelligence research and development.

At the conclusion of the course, the student is expected to gain an understanding of the machine learning models and algorithms used to create large language models including training and inference for representation learning, embedding models, sentence encoders, generative language models, autoregressive language models, fine-tuning and instruction tuning of language models.

Instructor

Teaching Assistants

  • Jackon de Faria, jgd5, Office hour: TBD.
  • Enze Jiang, eja42, Office hour: TBD.
  • Wanying Tian, wta55, Office hour: TBD.

Asking for help

  • Ask for help on the discussion forum
  • Instructor office hours: Thu 8:30-9:30am (starts on Sept 21); Zoom link on Coursys discussion forum
  • No emails to the TAs and strictly emails about personal matters to the instructor
  • Always post to the the discussion forum instead of email. If you have to email use your SFU email address only.

Time and place

  • Wed 9:30-10:20am Blusson Hall BLU10011
  • Fri 8:30am-10:20am Blusson Hall BLU10011
  • Last day of classes: Dec 2

Calendar

Textbook

  • No required textbook. Online readings provided in Syllabus.

Grading

  • Submit homework source code and check your grades on Coursys
  • Programming setup homework: HW0 due on Sep 17 (2%)
  • Four programming homeworks. Due dates: HW1 on Sep 29, HW2 on Oct 13, HW3 on Nov 3, HW4 on Nov 17 (10% each)
  • In class midterm: Oct 22 (25%)
  • Participation: Helping other students on the discussion board in a positive way (5%)
  • Final Project Proposal: Due on Nov 10 (5%)
  • Final Project: Due on Dec 5 (23%)
  • Final Project Poster Session:
    • Time: Dec 5 TBD.
    • Location: TBD