Skip to main content

Natural Language Processing Fall 2024

Natural Language Processing (NLP) was traditionally a research field heavily reliant on partially supervised machine learning to tackle language tasks. However, the landscape shifted dramatically with the advent of large language models (LLMs), popularized by tools like ChatGPT. Unlike earlier models, LLMs are trained using self-supervised learning, exhibiting remarkable emergent behavior and tackling a wide range of tasks they were never explicitly trained on. This demonstrates that unsupervised learning is scalable and capable of achieving zero-shot performance, where models perform tasks with little to no task-specific examples.

This course delves into language models and representation learning for NLP. We will explore key components such as model architecture, effective training strategies, and inference techniques, highlighting their applications across diverse natural language processing tasks. As NLP rapidly evolves, LLMs have become a cornerstone of artificial intelligence research and development.

Instructor

Teaching Assistants

  • Yanxin Shen, ysa291, Office hour: Fridays 3pm-4pm on Zoom (link available on Coursys discussion forum).

Asking for help

  • Ask for help on the discussion forum
  • Instructor office hours: Thu 8:30-9:30am (starts on Sept 21); Zoom link on Coursys discussion forum
  • No emails to the TAs and strictly emails about personal matters to the instructor
  • Always post to the the discussion forum instead of email. If you have to email use your SFU email address only.

Time and place

  • Wed 9:30-10:20am Blusson Hall BLU10011
  • Fri 8:30am-10:20am Blusson Hall BLU10011
  • Last day of classes: Dec 3

Calendar

Textbook

  • No required textbook. Online readings provided in Syllabus.

Grading

  • Submit homework source code and check your grades on Coursys
  • Programming setup homework: HW0 due on Sep 16 (2%)
  • Four programming homeworks. Due dates: HW1 on Oct 1, HW2 on Oct 15, HW3 on Nov 5, HW4 on Nov 29 (10% each)
  • In class midterm: Oct 23 (25%)
  • Participation: Helping other students on the discussion board in a positive way (5%)
  • Final Project Proposal: Due on Nov 15 (5%)
  • Final Project: Due on Dec 6 (23%)
  • Final Project Poster Session:
    • Time: Dec 6 Poster session: 10:00am-12:30pm.
    • Location: HAL 126