Link Search Menu Expand Document
CSCI 2952-N

Course Schedule

Paper reading list and presenters

Jan 27, Thu
Course Overview (slides)
Chen Sun
Feb. 1, Tue
Recap: CNNs and Transformers (slides)
Chen Sun
  1. Presentation signup sheet
  2. Paper nomination form
Feb. 3, Thu
Overview: Self- and Cross-modal Learning (slides)
Chen Sun
  1. (Background) How to Read a CS Research Paper by Philip Fong
  2. (Background) How to do research by Bill Freeman
  3. (Background) How to do write a good paper by Bill Freeman
  4. (Background) Novelty in Science by Michael Black
  5. (Background) Self-supervised learning: The dark matter of intelligence by Yann LeCun and Ishan Misra
Feb. 4, Fri
Due Presentation signup
Feb. 8, Tue
The Unreasonable Effectiveness of Data (Reading survey / Questions / Slides)
Jorge, Koyena, Yipu
  1. Revisiting Unreasonable Effectiveness of Data in the Deep Learning Era
  2. A ConvNet for the 2020s
  3. (Background) The Unreasonable Effectiveness of Data
  4. (Background) Training data-efficient image transformers & distillation through attention
  5. (Background) Exploring Randomly Wired Neural Networks for Image Recognition
  6. (Background) NAS evaluation is frustratingly hard
  7. (Background) The bitter lesson
Feb. 10, Thu
Semi-supervised Learning (Reading survey / Questions / Slides)
Cheng-You, Vivek
  1. Mean teachers are better role models
  2. MixMatch: A Holistic Approach to Semi-Supervised Learning
  3. (Background) Semi-Supervised Classification with Graph Convolutional Networks
  4. (Background) Inductive Representation Learning on Large Graphs
  5. (Background) Transfer Learning in a Transductive Setting
Feb. 15, Tue
Transfer Learning (Reading survey / Questions / Slides)
Changcheng, Gabriel, Kangping
  1. Big Transfer (BiT): General Visual Representation Learning
  2. Rethinking Pre-training and Self-training
  3. (Background) A Survey on Transfer Learning
  4. (Background) Transfusion: Understanding Transfer Learning for Medical Imaging
  5. (Background) Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks
  6. (Background) A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
  7. (Background) Rethinking ImageNet Pre-training
Feb. 17, Thu
Few-shot Learning (Reading survey / Questions / Slides)
Anessa, Reza, Yong
  1. Matching Networks for One Shot Learning
  2. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
  3. (Background) Prototypical Networks for Few-shot Learning
  4. (Background) Learning to Learn (Blog)
  5. (Background) Meta-Learning: Learning to Learn Fast (Blog)
  6. (Background) A Closer Look at Few-shot Classification
  7. (Background) Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
Feb. 22, Tue
University holiday, no class
Feb. 24, Thu
Multitask Learning (Reading survey / Questions / Slides)
Amir, Hyuk, Jinwoo, Leonard
  1. Taskonomy: Disentangling Task Transfer Learning
  2. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Section 1, 2, 4)
  3. (Background) UberNet: Training a Universal Convolutional Neural Network
  4. (Background) On the Opportunities and Risks of Foundation Models
  5. (Background) ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Mar. 1, Tue
AI Safety (Reading Survey / Questions / Slides)
Anna, Mason, Will Yang
  1. Concrete Problems in AI Safety
  2. (Background) Deep reinforcement learning from human preferences
  3. (Background) AI safety via debate
  4. (Background) Avoiding Side Effects By Considering Future Tasks
  5. (Background) Objective Robustness in Deep Reinforcement Learning
Mar. 3, Thu
Transformer and its variants (Reading Survey / Questions / Slides1 / 2)
George Zerveas, Kai
  1. Big Bird: Transformers for Longer Sequences
  2. Synthesizer: Rethinking Self-Attention in Transformer Models
  3. (Background) Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
  4. (Background) MLP-Mixer: An all-MLP Architecture for Vision
  5. (Background) Linformer: Self-Attention with Linear Complexity
  6. (Background) Highly accurate protein structure prediction with AlphaFold
Mar. 8, Tue
Vision Transformers (1) (Reading Survey / Questions / Slides)
Chace, Justin, Shijie
  1. Swin Transformer
  2. On the Relationship between Self-Attention and Convolutional Layers
  3. (Background) ViViT: A Video Vision Transformer
  4. (Background) VideoBERT: A Joint Model for Video and Language Representation Learning
  5. (Background) Video Action Transformer Network
Mar. 10, Thu
Vision Transformers (2) (Reading Survey / Questions / Slides)
Avi, George Hu, Peilin
  1. End-to-End Object Detection with Transformers
  2. Perceiver: General Perception with Iterative Attention
  3. (Background) TrackFormer: Multi-Object Tracking with Transformers
  4. (Background) MaX-DeepLab: End-to-End Panoptic Segmentation With Mask Transformers
  5. (Background) Perceiver IO: A General Architecture for Structured Inputs & Outputs
  6. (Background) Episodic Transformer for Vision-and-Language Navigation
Mar. 11, Fri
Due Final project signup
Mar. 14, Mon
Due Mid-term feedback
Mar. 15, Tue
Self-supervised Learning for NLP (Reading Survey / Questions / Slides)
Catherine, William Jurayj, William Rudman
  1. Language Models are Few-Shot Learners
  2. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
  3. (Background) REALM: Retrieval-Augmented Language Model Pre-Training
  4. (Background) SpanBERT: Improving Pre-training by Representing and Predicting Spans
  5. (Background) RoBERTa: A Robustly Optimized BERT Pretraining Approach
  6. (Background) Human Language Understanding & Reasoning
  7. (Background) Do Large Language Models Understand Us?
Mar. 17, Thu
Self-supervised Learning for Images (Reading Survey / Questions / Slides)
Sijie, Tian, Vadim
  1. BEiT: BERT Pre-Training of Image Transformers
  2. Representation Learning with Contrastive Predictive Coding
  3. (Background) Dimensionality Reduction by Learning an Invariant Mapping
  4. (Background) Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision
  5. (Background) Masked Autoencoders Are Scalable Vision Learners
  6. (Background) Deep Clustering for Unsupervised Learning of Visual Features
  7. (Background) Towards the Generalization of Contrastive Self-Supervised Learning
  8. (Background) Bootstrap your own latent: A new approach to self-supervised Learning
Mar. 22, Tue
Invited Learning Structured Models of the World
Thomas Kipf
  1. (Background) Object-Centric Learning with Slot Attention
  2. (Background) Conditional Object-Centric Learning from Video
Mar. 24, Thu
Project proposal (Master deck)
Mar. 29, Tue
Spring break
Mar. 31, Thu
Spring break
Apr. 5, Tue
Self-supervised Learning for Videos (Reading Survey / Slides)
Bader, Ce, Trevor
  1. Time-Contrastive Networks: Self-Supervised Learning from Video
  2. Learning image representations tied to ego-motion
  3. (Background) Simulation as an engine of physical scene understanding
  4. (Background) Learning correspondence from the cycle-consistency of time
  5. (Background) Learning Temporal Dynamics from Cycles in Narrated Video
Apr. 7, Thu
Representation Learning for RL (Reading Survey / Slides)
Aditya, Calvin, Haotian
  1. CURL: Contrastive Unsupervised Representations for Reinforcement Learning
  2. Decision Transformer: Reinforcement Learning via Sequence Modeling
  3. (Background) R3M: A Universal Visual Representation for Robot Manipulation
  4. (Background) Understanding the World Through Action
  5. (Background) Learning Latent Plans from Play
  6. (Background) Learning Latent Dynamics for Planning from Pixels
  7. (Background) Control-Aware Representations for Model-based Reinforcement Learning
  8. (Background) Shaping Belief States with Generative Environment Models for RL
  9. (Background) Goal-Aware Prediction: Learning to Model What Matters
Apr. 8, Fri
Due Project proposal
Apr. 12, Tue
Invited Multimodal Learning (Slides)
Arsha Nagrani
  1. (Background) Attention Bottlenecks for Multimodal Fusion
  2. (Background) Speech2Action: Cross-modal Supervision for Action Recognition
Apr. 14, Thu
3D Computer Vision (Reading Survey / Slides)
Arman, Jiahao, Mikhail, Rao
  1. MarrNet: 3D Shape Reconstruction via 2.5D Sketches
  2. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
  3. (Background) Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
  4. (Background) Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
  5. (Background) PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Apr. 19, Tue
Generative Modeling (Reading Survey / Slides)
Michal, Nate, Yuanhao
  1. Neural Discrete Representation Learning
  2. Diffusion Models Beat GANs on Image Synthesis
  3. (Background) PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows
  4. (Background) Variational Graph Auto-Encoders
  5. (Background) What are Diffusion Models?
  6. (Background) Zero-Shot Text-to-Image Generation
  7. (Background) Denoising Diffusion Probabilistic Models
Apr. 21, Thu
Data and model bias (Reading Survey / Slides)
Arun, Ghulam, Kunal, Pinar
  1. Beyond Accuracy: Behavioral Testing of NLP models with CheckList
  2. Equality of Opportunity in Supervised Learning
  3. (Background) Measuring and Reducing Gendered Correlations in Pre-trained Models
  4. (Background) Comparing Human and Machine Bias in Face Recognition
Apr. 26, Tue
Model interpretability (Reading Survey / Slides)
Amanda, Usha, Zachary
  1. Do Vision Transformers See Like Convolutional Neural Networks?
  2. Acquisition of Chess Knowledge in AlphaZero
  3. (Background) BERT rediscovers the classical NLP pipeline
  4. (Background) A Primer in BERTology: What We Know About How BERT Works
Apr. 28, Thu
Future Prediction, Causality (Reading Survey / Slides)
Alexander, Heejun, Peisen, Tiancheng
  1. Attention over learned object embeddings enables complex visual reasoning
  2. PHYRE: A New Benchmark for Physical Reasoning
  3. (Background) Machine Theory of Mind
  4. (Background) Shaking the foundations: delusions in sequence models for interaction and control
Apr. 29, Fri
Due Presentation Slot Signup
May 3, Tue
Final project office hours
May 5, Thu
Final project office hours
May 10, Tue
Final project presentations (Slides)
May 12, Thu
Final project presentations (Slides)
May 13, Fri
Due Project submission
May 22, Fri
Due Post-semester feedback
Other
Student Nominated Readings
  1. (Physics-informed ML) Physics-informed neural networks
  2. (Operator Learning) Learning nonlinear operators via DeepONet
  3. (Biologically-Inspired Learning) Training Spiking Neural Networks Using Lessons From Deep Learning