CS 7643: Deep Learning

Instructional Team

Zsolt Kira

Zsolt Kira
Instructor

Overview

Deep learning is a sub-field of machine learning that focuses on learning complex, hierarchical feature representations from raw data. The dominant method for achieving this, artificial neural networks, has revolutionized the processing of data (e.g. images, videos, text, and audio) as well as decision-making tasks (e.g. game-playing). Its success has enabled a tremendous amount of practical commercial applications and has had significant impact on society.

In this course, students will learn the fundamental principles, underlying mathematics, and implementation details of deep learning. This includes the concepts and methods used to optimize these highly parameterized models (gradient descent and backpropagation, and more generally computation graphs), the modules that make them up (linear, convolution, and pooling layers, activation functions, etc.), and common neural network architectures (convolutional neural networks, recurrent neural networks, etc.). Applications ranging from computer vision to natural language processing, and decision-making (reinforcement learning) will be demonstrated. Through in-depth programming assignments, students will learn how to implement these fundamental building blocks as well as how to put them together using a popular deep learning library, PyTorch. In the final project, students will apply what they have learned to real-world scenarios by exploring these concepts with a problem that they are passionate about.

CS 7643, originally created at Georgia Tech five years ago, was rebuilt with the support of Facebook for on-campus students in Spring 2020.

This course counts towards the following specialization(s):
Machine Learning

Machine Learning Specialization Elective

Course Goals

  • Describe the major differences between deep learning and other types of machine learning algorithms.
  • Explain the fundamental methods involved in deep learning, including the underlying optimization concepts (gradient descent and backpropagation), typical modules they consist of, and how they can be combined to solve real-world problems.
  • Differentiate between the major types of neural network architectures (multi-layered perceptions, convolutional neural networks, recurrent neural networks, etc.) and what types of problems each is appropriate for.
  • Select or design neural network architectures for new data problems based on their requirements and problem characteristics, and analyze their performance.
  • Describe some of the latest research being conducted in the field and open problems that are yet to be solved.

Sample Syllabus

Fall 2020 syllabus and schedule (PDF)

Note: Sample syllabi are provided for informational purposes only. For the most up-to-date information, consult the official course documentation.

Before Taking This Class...

Suggested Background Knowledge

It is recommended that students have a good mathematical background (linear algebra, calculus, and probabilities & statistics) and at least an introductory course in Machine Learning (e.g. equivalent to CS 7641). Strong programming skills (specifically Python) are necessary to complete the assignments.

Technical Requirements and Software
  • High-speed Internet connection
  • Laptop or desktop computer with a minimum of a 2 GHz processor and 2 GB of RAM
  • Windows for PC computers OR Mac iOS for Apple computers
  • Complete Microsoft Office Suite or comparable and ability to use Adobe PDF software (install, download, open and convert)
  • Mozilla Firefox, Chrome browser, and/or Safari browsers (Chrome required for on-boarding quiz)

Academic Integrity

All Georgia Tech students are expected to uphold the Georgia Tech Academic Honor Code. This course may impose additional academic integrity stipulations; consult the official course documentation for more information.