AIE 604

AIE 604 : Deep Learning Applications

M.Sc in Artificial Intelligence Systems

(by Prof. Mohab Mangoud)

  1. The purpose of the Deep Learning applications course is to present to learners more details about one of the latest engineering applications of AI and deep learning in daily basis. This course will rather lead to profitable roles in IT, engineering computer vision, healthcare, FinTech, e-commerce, and other industries. Deep Learning is one of the most highly sought-after skills in AI. In this course, students will also learn the foundations of Deep Learning, understand how to build advanced ANN, successful machine learning projects for engineers. Students will learn Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization and Generative AI.

Syllabus: More details, about DEEP LEARNING:

  • Variability models, deformation model, stochastic model.
  • Supervised Learning: classification.
  • CNN-Convolutional Neural Networks
  • Properties of CNN representations: invertibility, stability,
  • Localization, Regression, and Embeddings, DrLim, and inverse
  • Object Detection techniques 
  • Sequential NN, RNN / LSTM
  • Deep Generative modeling (AE, VAE, GANs)
  • Examples and Applications with Coding.

Course Grading

  • Survey paper / presentation                     (20%)
  • HWs:                10% +10% +10%+10%=    (40%)                                               
  • Final Project                                                (40%)
 

Course outlines

Lecture 1(w1)  [Feb 10]:  Course overview and Objectives  Introduction to Deep learning + Lab1: Python + TF + Keras + Matlab : (chapter 1) 

Lecture 2(w2) : [Feb 17]: Neural Networks I :  Basics/ Perceptron / MLP / different types of Activation (Chapter 2)

_______________________

 ( W : Feb 18= 1st Ramadan 1447) Start of Ramadan working hours

Lecture 3(w3) : [Feb 24 ]  Neural Networks II / Modeling / Training/ Loss function / Back-propagation / improve training speed and accuracy / Stochastic gradient descent /Batches / Xavier/He initialization

  Final project proposal (*** 5%)

Lecture 4(w4) : [Mar 3]    

– HW1 (10%): Build a basic neural network models (6 examples) / Lab1

Lecture 5(w5)  :  [Mar 10 ] Convolutional Neural Networks (CNN I) CNN model / Compare to fully connected NN / Build a CNN by choosing the grid size, padding, stride, depth, and pooling.  (chapter 3)

Lecture 6(w6) : [Mar 17 ] Convolutional Neural Networks  II: Pre- trained Models/ Transfer learning/ Data Augmentation /  AlexNet, VGG, GoogLeNet, ResNet, etc / Deep learning literature talks about many image classification topologies like AlexNet, VGG-16 and VGG-19, Inception, and ResNet. This week, learn how these topologies are designed and the usage scenarios for each.  (chapter 4) 

– HW2 (10%) : Build a CNN model / application – / Lab2   

_______________________

Sun 20 March – 24 March : Eid Alfitr Holiday – end of Ramadan working hours

_______________________
Lecture 7(w7) :  [Mar 31]  Object Detection – Two stage detectors  – R-CNN, SDD, YOLO, one stage detector. 
Lecture 8(w8) :  [Apr 7]  RNN : Recurrent neural networks (RNN) and their application to natural language processing (NLP). LSTM Long short term memory (LSTM).

Lecture 9 (w9) : [Apr 14] Students seminar# 1:  Survey paper
# 1 (5 papers presentations) (20%)

Lecture 10(w10) : [April 21] Deep Generative Modeling 1 (AE, VAE)

Lecture 11(w11) :  [April 28] Deep Generative Modeling 2 (GANs) +Transformer + LLM]

HW3 – (10%) : OD/ RNN/ GANs  – Lab 3 

  • Lecture 14(w12) : [May 5] Deep reinforcement learning I (DRL)

HW4 (10%) : DRL  – Lab 4 

Last day of Classes      M May 11

T [ May 12] Final Project presentation + report / final paper submission (40%)

 

______________________________________________________

Previous Course Projects (Examples)

Class (2021) 

Class (2022)

Class (2023) 

Class (2024)

Class (2025) 

____________________________________________________________

Tutorials

https://cs231n.github.io/python-numpy-tutorial/

https://github.com/cs231n/gcloud

https://www.tensorflow.org/tutorials

https://deeplearning.cs.cmu.edu/F20/document/recitation/IDL_Recitation1.pdf

Textbooks

This is a selection of optional textbooks you may find useful

Deep Learning
Dive Into Deep LearningBy Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. SmolaPDF, 2020
Deep Learning
Deep LearningBy Ian Goodfellow, Yoshua Bengio, Aaron CourvilleOnline book, 2017
 

Software tools

  1. keras.io
  2. tensorflow.org
  3. mathworks.com/products/deep-learning.html

The Neural Network Zoo