# Neural Networks and Deep Learning

**Neural Networks and Deep Learning**

**Columbia University course ECBM E4040**

**Zoran Kostic****, Ph.D., Dipl. Ing., Professor of Professional Practice, zk2172(at)columbia.edu**

**Electrical Engineering Department, ****Data Sciences Institute,**** **** Columbia University in the City of New York**

**Course in a nutshell: **

** Theoretical underpinnings and practical aspects of Neural Networks and Deep Learning. Convolutional and Recurrent Neural Networks. Focus on applications and projects.**

**Bulletin Description:**

Developing features & internal representations of the world, artificial neural networks, classifying handwritten digits with logistics regression, feedforward deep networks, back propagation in multilayer perceptrons, regularization of deep or distributed models, optimization for training deep models, convolutional neural networks, recurrent and recursive neural networks, deep learning in speech and object recognition.

**Time:**

Fall 2021: To register into the course: (i) students need to get onto the Columbia SSOL waitlist, and (ii) populate this form (using CU UNI email). Those students who satisfy the requirements will be moved from the waitlist into the registered list by early September. The course can be taken by undergraduate and graduate student. See additional course information here.

Spring 2021, Fall 2020, Fall 2019, Fall 2018, Fall 2017, Fall 2016.

**Prerequisites****:**

Required prerequisites: knowledge of linear algebra, probability and statistics, programming. Strongly recommended: machine learning.

Suggested prerequisite courses: (BMEB W4020) or (BMEE E4030) or (ECBM E4090) or (EECS E4750) or (COMS W4771) or similar.

**Organization**

Lectures:

Presentation of material by instructors and guest lecturers

Homeworks:

Combination of analytical and programming assignments

Projects:

Team-based

Students with complementary backgrounds

Significant design

Reports and presentations to Columbia and NYC community

Best could qualify for publications and/or funding

Industry participation:

Project definition and sponsoring

Weekly presentations

Interaction with students through mentoring

**Content**

Introduction to neural networks.

Convolutional and recurrent neural networks.

Focuses on the intuitive understanding of deep learning.

Review of underpinning theory - linear algebra, statistics, machine learning.

Analytical study and software design.

Three-four assignments in Python and one DL framework (Tensorflow or PyTorch)

Significant project.

Enables further exploration of key concepts in deep learning.

**Typical Syllabus**

Introduction to Course E4040

Introduction to Deep Learning (DL)

Introduction to DL computing Resources

Machine Learning Algorithms

t-SNE Data Visualization

Universal approximation theorem - Visual proof

Algebra review

Deep Feed-forward Networks

Back Propagation

Optimization

Convolutional Neural Nets (CNNs)

CNN applications

CNN Examples

Regularization

Practical Methodology for Deep Learning

Recurrent Neural Nets (RNNs)

RNN applications

Deel Learning Applications

Autoencoders

Generative Networks GANs, Variational Encoders

Generative Models

Guest Lectures

Deep Learning Trends

**Books, Tools and Resources**

BOOKS:

2021, 2020, 2019, 2018, 2017 software platform:

Google TensorFlow, Google Cloud, Python, Github

2016 software platform:

Amazon AWS cloud tools, Code development on github, bitbucket

**Project Areas**

Medical

Autonomous cars

Environmental

Smart cities

Physical data analytics

**2021 ****Fall ****Projects**

Composing Music With Recurrent Neural Networks

A New Backbone for Hyperspectral Image Reconstruction

A Recurrent Latent Variable Model for Sequential Data

Custom DenseNet models for Tiny ImageNet Classification

Deep compressive autoencoder for action potential compression in large-scale neural recording

Deep Networks with Stochastic Depth

DeepPainter: Painter Classification Using Deep Convolutional Autoencoders

Fairness without Demographics through Adversarially Reweighted Learning

FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction

MoEL: Mixture of Empathetic Listeners

Neural Distance Embeddings for Biological Sequences

Neural Distance Embeddings for Biological Sequences

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

Residual Attention Network for Image Classification

Social GAN for Human Driving Trajectories Prediction

Spectral Representations for Convolutional Neural Networks

Squeeze-and-Excitation Networks Code avaiable

Towards Accurate Binary Convolutional Neural Network Xiaofan Lin, Cong Zhao, Wei Pan, (2017NIPS)

Video-based human emotion recognition

**2021 Spring Projects**

A deep learning framework for financial time series using stacked autoencoders and long-short term memory

A Neural Algorithm of Artistic Style

A neural attention model for speech command recognition

BinaryConnect: Training Deep Neural Networks with Binary Weights during Propagations

Composing Music With Recurrent Neural Networks

Conditional Generative Adversarial Net

Deep Double Descent: Where Bigger Models and More Data Hurt

Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2

Deep Learning for Price Prediction of Cryptocurrencies

DeepPaint -- Classification of Paintings using Convolutional Autoencoder

Densely Connected Convolutional Networks

Enhancing Detection of Steady-State Visual Evoked Potentials Using Deep Learning

From 2D to 3D: Kidney Tumor Segmentation Challenge

GRUV: Algorithmic Music Generation using Recurrent Neural Networks

Highway Networks

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Neural Networks for Automated Essay Grading

Recurrent Neural Networks to Create Comprehensive 3D Models of Granular Media in YADE

Residual Attention Network for Image Classification

Singing Voice Separation from Monaural Recordings Using Deep Recurrent Neural Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

**2020 Fall Projects**

Custom DenseNet models for Tiny ImageNet Classification

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

A neural attention model for speech command recognition

CNN-Generated Images Are Surprisingly Easy to Spot... for Now

Composing Music With Recurrent Neural Networks

Custom DenseNet models for Tiny ImageNet Classification

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Deep Learning and the Cross-Section of Stock Returns: Neural Networks Combining Price and Fundamental Information

Image Super-Resolution Using Deep Convolutional Networks

Machine learning methods for crop yield prediction and climate change impact assessment in agriculture

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

PoseCNN: A convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

Random Erasing Data Augumentation

Residual Attention Network for Image Classification

Rethinking Model Scaling for Convolutional Neural Networks - EfficientNet

Semi-Supervised Learning with Graph Convolutional Neural Networks

Time series forecasting of petroleum production using deep LSTM recurrent networks

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

**2019 Projects**

A deep learning framework for financial time series using stacked autoencoders and long-short term memory

Adversarial Autoencoder Assisted Artifact Reduction of Ballistocardiogram in Simultaneous EEG-fMRI Recordings

Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks

Automated Gleason Grading of Prostate Cancer Tissue Microarrays via Deep Learning

Deep learning-based feature engineering for stock price movement prediction

Deformable Convolutional Networks:(Expanding the receptive field through Deformable Convolution)

Development and validation a RNN model to teach machines to read and comprehend

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Multi-Digit Number Recognition from Street View Imagery Using Deep Convolutional Neural Networks

One-Shot Video Object Segmentation for Mobile Vision Applications

Pelee: A Real-Time Object Detection System on Mobile Devices

Residual Attention Network for Image Classification

Short Term Electricity Consumption Forecasting in Residential Building with a Selected Auto-regressive Features&ConvLSTM Neural Network Method

Show and Tell: A Neural Image Caption Generator

SSD: Single Shot MultiBox Detector

Stock market's price movement prediction with LSTM neural networks

Towards Accurate Binary Convolutional Neural Network

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Why Should I Trust You? Explaining the Predictions of Any Classifier

World Models

**2018 Projects**

A deep learning framework for relationship extraction from articles using long-short term memory and named entity recognition

A Neural Algorithm of Artistic Style

A Neural Representation of Sketch Drawings

Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks

Backprop KF: Learning Discriminative Deterministic State Estimators

Deep contextualized word representations

Dynamic Routing Between Capsules

Gesture Recognition

Learned in Translation: Contextualized Word Vectors

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

Multi-Digit Number Recognition from Street View Imagery Using Deep Convolutional Neural Networks

Neural Networks for Automated Essay Grading

Parallel Multi-Dimensional LSTM,With Application to Fast Biomedical Volumetric Image Segmentation

PixelGAN Autoencoders

Prevention of catastrophic forgetting in Neural Networks for lifelong learning

Semantic Image Inpainting with Deep Generative Models

Towards Accurate Binary Convolutional Neural Network

Universal Style Transfer via Feature Transforms

Unsupervised Image-to-Image Translation Networks

**2016 Projects**

Striving for Simplicity: The All Convolutional Net

A Combined Semi-supervised Learning mechanism for Video Data via Deep Learning

A Neural Algorithm of Artistic Style

Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network

Colorful Image Colorization

Deep Networks with Stochastic Depth

Highway Networks

Image Super-Resolution Using Deep Convolutional Networks

Learning to Protect Communications with Adversarial Neural Cryptography

Singing Voice Separation from Monaural Recordings Using Deep

Recurrent Neural Networks

Spatial Transformer Networks

Spoken Language Understanding Using Long-Short Term Memory Neural Networks

Striving for Simplicity: The All Convolutional Net

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

**Course sponsored by equipment and financial contributions of:**

NVidia GPU Education Center, Google Cloud, IBM Bluemix, AWS Educate, Atmel, Broadcom (Wiced platform); Intel (Edison IoT platform), Silicon Labs.