Link Search Menu Expand Document
CSCI 2952-N

Advanced Topics in Deep Learning

Time and Location

In-class participation is required unless otherwise approved by the instructor.

Lectures will be recorded for asynchronous viewing.

Resources

About

Welcome to CSCI 2952N! This course aims at preparing graduate-level students the research knowledge they need to apply Deep Learning techniques for their own research. Over the past few years, there has been tremendous success in developing unified neural architectures that achieve state-of-the-art performance on language understanding (GPT), visual perception (ViT), and even protein structure prediction (AlphaFold, 2024 Nobel Prize in Chemistry). The unified neural architecture design, when trained on huge amounts of data (the whole Internet!) on tens of thousands of GPUs, have given rise to ChatGPT, StableDiffusion, Genie, MusicGen, and Cosmos. Researchers and entrepreneurs are working hard to transfer the success into our daily lives, from intelligent agents, next-generation search engines, to healthcare and robotics.

Our course aims to help students understand the basic building blocks that lead to the success of these advanced deep learning models, from the perspectives of mathematical tools that guide the high-level designs of the learning paradigms, to the nuanced but crucial details of neural architecture and data engineering. Towards the end of the seminar, we hope students will have their own answers about: (1) What gives rise to the success of the current “AI” systems, better models, bigger and higher-quality data, stronger hardwares, or a mixture of everything?; (2) Should you become an AI researcher? What are the interesting problems that are left unsolved?; (3) What the latest and upcoming AI models enable you to pursue your own career?

The course is organized as a combination of instructor (and his students) led overview lectures, paper reading, student presentations, and invited guest lectures. It also requires the students to work on a final project that explores a novel direction they choose along the line of the papers we cover.

Learning Goals

This is a seminar course aimed for PhD students and students who would like to further pursue a career that utilizes deep learning either in academia or in industry.

Students are expected to feel comfortable reading 3-4 research papers per week (6 to 8 hours). They are expected to propose a final group project idea with contributions sufficient for a premier Machine Learning Conference (e.g. NeurIPS, ICLR) workshop or above. This class may be used for capstone.

Students who complete this course will:

  • Be familiar with recent trends of deep learning techniques and explore their applications to your own research.
  • Develop skills for critically reading research papers, identifying their high-level insights and limitations.
  • Build in-depth knowledge in one or more areas of active research directions.
  • Understand the evolution of research ideas over time, how some prominent research directions withstand the test of time (or do not)
  • Obtain hands-on experience proposing and implementing a novel research idea.
  • Build up your own philosophy when encountered with tons of new papers on a daily basis, “hyped-up” new results advocated by the researchers themselves, and news coverage about “AI” in the popular media.

Grading

  • 40% Final project
  • 20% Mini projects
  • 20% In-class presentation
  • 20% Paper reading and participation

For details please refer to the syllabus.

Academic Integrity & Collaboration Policy

Academic dishonesty will not be tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Brown Academic and Student Conduct Codes.
Discussion of course material with your classmates is both permitted and encouraged. However, showing, copying, or other sharing of actual code or verbatim answers to written questions is forbidden. This policy will be enforced.

Diversity & Inclusion

Our intent is that this course provides a welcoming environment for all students who satisfy the prerequisites. All members of the CS community, including faculty and staff, are expected to treat one another in a professional manner. If you feel you have not been treated in a professional manner by any of the course staff, please contact either the instructor, Ugur Cetintemel (Dept. Chair), Tom Doeppner (Vice Chair) or Laura Dobler (diversity & inclusion staff member). We will take all complaints about unprofessional behavior seriously.
Brown welcomes students from all around the country and the world, and their unique perspectives enrich our learning community. To empower students whose first language is not English, an array of support is available on campus, including language and culture workshops and individual appointments. For more information, contact the English Language Learning Specialists at ellwriting@brown.edu.

Accomodations

Brown University is committed to full inclusion of all students. Please inform the instructor if you have a disability or other condition that might require accommodations or modification of any of these course procedures. You may email the instructor, come to office hours, or speak with him after class, and your confidentiality is respected. We will do whatever we can to support accommodations recommended by SEAS. For more information contact Student and Employee Accessibility Services (SEAS) at 401-863-9588 or SEAS@brown.edu. Students in need of short-term academic advice or support can contact one of the deans in the Dean of the College office.

Mental Health

Being a student can be very stressful. If you feel you are under too much pressure or there are psychological issues that are keeping you from performing well at Brown, we encourage you to contact Brown’s Counseling and Psychological Services (CAPS). They provide confidential counseling and can provide notes supporting extensions on assignments for health reasons.