Advanced Deep Learning
EECS E6691 - Topics and Data Driven Analysis and Computation
Advanced Deep Learning
Columbia University Course
Zoran Kostic, Ph.D., Dipl. Ing., Professor of Professional Practice, zk2172(at)columbia.edu
Electrical Engineering Department, Data Sciences Institute, Columbia University in the City of New York
Course in a nutshell:
Advanced theory and practice of Deep Learning. Applications and projects.
Description: Advanced (second) course on Deep Learning
Bulletin Description: Regularized autoencoders, sparse coding and predictive sparse decomposition, denoising autoencoders, representation learning, manifold perspective on representation learning, structured probabilistic models for deep learning, Monte Carlo methods, training and evaluating models with intractable partition functions, restricted Boltzmann machines, approximate inference, deep belief networks, deep learning in speech and object recognition.
Detailed Description for Spring 2023
Instructor: Dr. Mehmet Kerem Turkcan mkt2126 (at) columbia.edu
This is an advanced-level course in which the students study topics in deep learning. It is required that students had previously taken a first-course in deep learning. The course consists of: (i) lectures on state-of-the-art architectural and modeling concepts, (ii) assignments, (iii) exam, and a (iv) final project. The course will address topics beyond material covered in the first course on Deep Learning (such as ECBM E4040), with applications of interest to students. In 2023, the main subject of the lectures will be object detection.
Students entering the course have to have prior experience with deep learning and neural network architectures including Convolutional Neural Nets (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memories (LSTMs), and autoencoders. They need to have working knowledge of coding in Python, Python libraries, Jupyter notebook, TensorFlow both on local machines and on Google Cloud, and of GitHub or similar code hosting tools. The framework and associated tools which will be the focus of this course are PyTorch and Google Cloud. Students have to be self-sufficient learners and to take an active role during classroom activities.
There will be a few (3-4) assignments throughout the semester focusing on coding. In the second half of the course, there will be a midterm exam comprised of multiple-choice questions.
Final projects need to be documented in a conference-style report, with code deposited in a GitHub repository. The code needs to be documented and instrumented such that the instructor can run it after a download from the repository. A Google Slides presentation of the project suitable for a poster presentation is required.
Prerequisites
(i) Machine Learning (taken previously, or in parallel with this course).
(ii) ECBM E4040 Neural Networks and Deep Learning, or an equivalent neural network/DL university course taken for academic credit.
(iii) The course requires an excellent theoretical background in probability and statistics, and linear algebra.
Students are strongly advised to drop the class if they do not have an adequate theoretical background and/or previous experience with programming of deep learning models. It is strongly advised (the instructor’s requirement) that students take no more than 12 credits of any coursework (including this course and project courses) during the semester while this course is being taken.
Registration
The enrollment is limited to several dozen students. The instructor’s permission is required to register. Students interested in the course need to populate the SSOL waitlist, and MUST also populate the questionnaire. The instructor will move the students off of the SSOL waitlist after reviewing the questionnaire.
(Tentative) Grading for the course (2023 Spring)
Assignments: 30%
Midterm Exam (Delivered at Week 11): 30%
Project (Final report & Code Repository): 40%
(Potential) Class Contribution: x
Content
Convolutional and recurrent neural networks.
Analytical study and software design.
Sevearal assignments in Python and in PyTorch
Significant project.
Pursuing deeper exploration of deep learning.
Organization
Lectures:
Presentation of material by instructors and guest lecturers
Student Presentations
Every student contributes several presentations on the subject of interest
Assignments:
Combination of analytical and programming assignments
Projects:
Team-based
Students with complementary backgrounds
Significant design
Reports and presentations to Columbia and NYC community
Best projects could qualify for publications and/or funding
Prerequisites:
Required: knowledge of linear algebra, probability and statistics, programming, machine learning, first course in deep learning.
Prerequisite courses: ECBM E4040 or similar
Time:
Spring 2022 - Advanced Deep Learning (EECS E6691 TPC - Topics in Data-driven Analysis and Computation)
Spring 2021 - Advanced Deep Learning (EECS E6691 TPC - Topics in Data-driven Analysis and Computation)
Project Areas
Smart cities
Medical
Autonomous vehicles
Environmental
Physical data analytics
Finance
Books, Tools and Resources
BOOKS:
2022 software platform:
PyTorch as the main framework, Google TensorFlow, Google Cloud, Python, bitbucket
2022 software:
2022 Spring Projects
Pix2Pix Image-to-Image Translation with Conditional Adversarial Networks
Jump rope counter
Black Box Adversarial Attack with Style Information
Learning Signed Distance Function for 3D Shape Representation (DeepSDF)
Representation learning without any labeled data
Comparison of Self-Supervised Models for Music Classification
Subcellular localization of proteins using deep learning
Image Descriptions Generator
Predicting remaining surgery duration
Vision Transformer
Adversarial Audio Synthesis
PlaNet - Latent Dynamics from Pixels
RecoNET: Understanding What happens in Videos
2021 Spring Projects
Multi-Graph Graph Attention Network for Music Recommendation
3D Facial Reenactment from 2D Video
Temporal Fusion Transformers for Time Series Forecasting
Deep Reinforcement Learning for Environmental Policy
Stochastic & Split Convolutions for High Resolution Image Classification (Integrated with EfficientNet)
Temporal Fusion Transformers for Time Series Forecasting
Forecasting Corn Yields with Semiparametric CNNs
Pose Estimation + Instance Segmentation
Speaker Independent Speech Separation
Generalized Autoencoder-based Transfer Learning for Structural Damage Assessment
Deep Reinforcement Learning for Environmental Policy
End-to-end object detection with Transformers
Exploring latent space of InfoGAN
Multi-Graph Graph Attention Network for Music Recommendation
3D Facial Reenactment from 2D Video
Pose Estimation + Instance Segmentation
2018-2020 Projects
See list of projects under E6040 link
Course sponsored by equipment and financial contributions of:
NVidia GPU Education Center, Google Cloud, IBM Bluemix, AWS Educate, Atmel, Broadcom (Wiced platform); Intel (Edison IoT platform), Silicon Labs.