Deep Learning on the Edge
EECS E6692 Topics in Data Driven Analysis and Computation
Deep Learning on the Edge
Columbia University Course
Zoran Kostic, Ph.D., Dipl. Ing., Professor of Professional Practice, zk2172(at)columbia.edu
Electrical Engineering Department, Data Sciences Institute, Columbia University in the City of New York
Course in a Nutshell
Theory and Practice of Deep Learning on the Edge, with Labs Using Nvidia Jetson Nano Devices.
Description
This is an advanced-level course with labs in which students build and experiment with deep-learning models which they implement on a low-power GPU edge computing device. The topics covered by the course are: (*) architectures of low power GPU devices; (*) algorithms and DL models suitable for edge implementation; (*) CUDA language; (*) pre-processing to condition the data; (*) labeling for ground-truth annotation; (*) profiling techniques; (*) connectivity between edge devices and cloud computing servers; (*) real-life data for experimentation; (*) comparison of performance for a variety of methods; (*) comparison of performance of edge-computing and cloud computing approaches.
The labs will use NVIDIA Jetson Nano as the edge device.
From year-to-year, the course may revolve around a contemporary application, such as smart city intersection or monitoring of patient health.
The course is interactive and requires students’ active participation in every session, by presenting, interpreting and implementing deep-learning papers and models. The study of concepts is accompanied by lab sessions where students will bring to realization models under consideration, and experiment with them. There will be half-a-dozen lab sessions. There will be a final project, preceded by a proposal. Final projects need to be documented in a conference-style report, with code deposited in a github repository. The code needs to be documented and instrumented such that the instructor can run it after a download from the repository. A google-slide presentation of the project (suitable for poster version) is required. A web hosted record/documentation of both labs and projects is expected.
Students have to be self-sufficient learners and be ready for hands-on coding. They have to take an active role during the lab activities.
Prerequisites
(i) Machine Learning (taken previously for academic credit).
(ii) Students entering the course have to have prior experience with deep learning and neural network architectures including Convolutional Neural Nets (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memories (LSTMs), and autoencoders. They need to have working knowledge of coding in Python, Python libraries, Jupyter notebook, Tensorflow, both on local machines and on the Google cloud, and of Bitbucket, Github or similar.
(iii) ECBM E4040 Neural Networks and Deep Learning, or an equivalent neural network/DL university course taken for academic credit. Whereas the quality of online ML and DL courses (coursera, udacity, eDx) is outstanding, many takers of online courses do the hands-on coding assignments superficially and therefore do not gain practical coding skills which are essential to participation in this advanced course.
(iv) The course requires an excellent theoretical background in probability and statistics and linear algebra.
Students are strongly advised to drop the class if they do not have adequate theoretical background and/or previous experience with programming deep learning models. It is strongly advised (the instructor’s requirement) that students take no more than 12 credits of any coursework (including projects) during the semester while this course is being taken.
Registration
The enrollment is limited to several dozen students. Instructor’s permission is required to register. Students interested in the course need to populate the SSOL waitlist, and MUST also populate a questionnaire (to be announced). The instructor will move the students off of the SSOL waitlist after reviewing the questionnaire.
Content
Analytical study and software design.
Several assignments in Python and in PyTorch
Significant project.
Pursuing deeper exploration of deep learning.
Organization
Lectures
Labs (Assignments)
Student Presentations
Every student contributes several presentations on the subject of interest
Projects:
Team-based
Students with complementary backgrounds
Significant design
Reports and presentations to Columbia and NYC community
Best projects could qualify for publications and/or funding
Prerequisites:
Required: knowledge of linear algebra, probability and statistics, programming, machine learning, first course in deep learning.
Prerequisite courses: Some combination of a first course in Neural Networks/Deep Learning (Columbia University ECBM E4040 or similar), Internet of Things, embedded systems, projects in NN/DL
Time:
Spring 2023 - https://doc.sis.columbia.edu/#subj/EECS/E6692-20241-001/
Spring 2022
Project Areas
Smart cities
Medical
Autonomous vehicles
Environmental
Physical data analytics
Books, Tools and Resources
BOOKS:
Development platforms and software:
NVIDIA Jetson Nano
PyTorch as the main framework, Google TensorFlow, Google Cloud, Python, bitbucket
2022 Spring Projects
Real-Time Chess Bot
Sign Language Translation
Fall Detection
Conditional GAN Compression
Deployment of a DL model on Edge Devices
Sleep Apnea Detection
Object tracking Using RL
Small-Footprint Keyword Spotting
Conditional GAN Compression
Course sponsored by equipment and financial contributions of:
Amazon AWS, NVidia GPU Education Center, Google Cloud, IBM Bluemix, AWS Educate, Atmel, Broadcom (Wiced platform); Intel (Edison IoT platform), Silicon Labs.