Deep Learning on the Edge

EECS E6692 Topics in Data Driven Analysis and Computation                            

Deep Learning on the Edge

Columbia University Course 

Zoran Kostic, Ph.D., Dipl. Ing.,  Professor of Professional Practice, zk2172(at)columbia.edu

Electrical Engineering Department, Data Sciences Institute, Columbia University in the City of New York

Course in a Nutshell

Theory and Practice of Deep Learning on the Edge, with Labs Using Nvidia Jetson Nano Devices.

Description

This is an advanced-level course with labs in which students build and experiment with deep-learning models which they implement on a low-power GPU edge computing device. The topics covered by the course are: (*) architectures of low power GPU devices; (*) algorithms and DL models suitable for edge implementation; (*) CUDA language; (*) pre-processing to condition the data; (*) labeling for ground-truth annotation; (*) profiling techniques; (*) connectivity between edge devices and cloud computing servers; (*) real-life data for experimentation; (*) comparison of performance for a variety of methods; (*) comparison of performance of edge-computing and cloud computing approaches.

The labs will use NVIDIA Jetson Nano as the edge device.

From year-to-year, the course may revolve around a contemporary application, such as smart city intersection or monitoring of patient health.

The course is interactive and requires students’ active participation in every session, by presenting, interpreting and implementing deep-learning papers and models. The study of concepts is accompanied by lab sessions where students will bring to realization models under consideration, and experiment with them. There will be half-a-dozen lab sessions. There will be a final project, preceded by a proposal. Final projects need to be documented in a conference-style report, with code deposited in a github repository. The code needs to be documented and instrumented such that the instructor can run it after a download from the repository. A google-slide presentation of the project (suitable for poster version) is required. A web hosted record/documentation of both labs and projects is expected.

Students have to be self-sufficient learners and be ready for hands-on coding. They have to take an active role during the lab activities.

Prerequisites

(i) Machine Learning (taken previously for academic credit).

(ii) Students entering the course have to have prior experience with deep learning and neural network architectures including Convolutional Neural Nets (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memories (LSTMs), and autoencoders. They need to have working knowledge of coding in Python, Python libraries, Jupyter notebook, Tensorflow, both on local machines and on the Google cloud, and of Bitbucket, Github or similar.

(iii) ECBM E4040 Neural Networks and Deep Learning, or an equivalent neural network/DL university course taken for academic credit. Whereas the quality of online ML and DL courses (coursera, udacity, eDx) is outstanding, many takers of online courses do the hands-on coding assignments superficially and therefore do not gain practical coding skills which are essential to participation in this advanced course.

(iv) The course requires an excellent theoretical background in probability and statistics and linear algebra. 

Students are strongly advised to drop the class if they do not have adequate theoretical background and/or previous experience with programming deep learning models. It is strongly advised (the instructor’s requirement) that students take no more than 12 credits of any coursework (including projects) during the semester while this course is being taken.

Registration

The enrollment is limited to several dozen students. Instructor’s permission is required to register. Students interested in the course need to populate the SSOL waitlist, and MUST also populate a questionnaire (to be announced). The instructor will move the students off of the SSOL waitlist after reviewing the questionnaire.

2022 Spring Projects

• Real-Time Chess Bot

• Sign Language Translation

• Fall Detection

• Conditional GAN Compression

• Deployment of a DL model on Edge Devices

• Sleep Apnea Detection

• Object tracking Using RL

• Small-Footprint Keyword Spotting

• Conditional GAN Compression

Course sponsored by equipment and financial contributions of:

• Amazon AWS, NVidia GPU Education Center, Google Cloud, IBM Bluemix, AWS Educate, Atmel, Broadcom (Wiced platform); Intel (Edison IoT platform), Silicon Labs.