YSC3221 Computer Vision and Deep Learning

Semester 2, 2017/2018
Yale-NUS College





Description:

Images and videos are everywhere. Using your mobile phone, it becomes easy to snap a picture or to record video. Yet, how can we automatically extract the rich visual information from those images/videos? This is the task computer vision attempts to solve. The goal of computer vision is to make computers work like human visual perception, namely, to understand and recognize the world through visual data. One important technique in computer vision is deep learning. Deep learning is able to extract features and to infer the visual information from the features automatically and accurately. This course will focus on the fundamentals of deep learning and its applications to computer vision.


Prerequisite: Programming skill in python, and maths (linear algebra, calculus, statistics/probability).
Textbook:
  1. Computer Vision: Models, Learning, and Inference, by S.J.D. Prince.
  2. Deep Learning, by Ian Goodfellow and Yoshua Bengio and Aaron Courville
The ebook versions are accessible through NUS library. Note, we will use the books loosely (some, if not many, topics are taken from other sources).
Instructor: Robby T. Tan (robby.tan [att] yale-nus.edu.sg)

Lecture Schedule:

The following schedule will not be strictly followed.


Date
Topic
Lecture Note
January 15 1. INTRODUCTION

Additional resources:
  • Brief introduction to computer vision: youtube

Lecture 1
January 18 2. FACE DETECTION: FEATURES + BOOSTING

Reading:
Additional resources:
Lecture 2
January 22 3. FACE DETECTION: ADABOOST CLASSIFICATION

January 25 Class Cancelled

January 29 Class Cancelled

February 1 4. FACE DETECTION: INTEGRAL IMAGE + CASCADE

February 5 5. FEATURES AND DESCRIPTORS: HoG + SIFT

Reading:
  • Textbook Chapter 13: Image Preprocessing and Feature Extraction, particularly Sec. 13.1 (per-pixel transformations), and Sec. 13.3.3 (Histogram of Oriented Gradients)
  • HoG for Human Detection: pdf | youtube 1 | youtube 2

Lecture 5
Assignment 1
February 8 6. SIFT (Part 1)

Reading:
  • Textbook: Chapter 13, Sec. 13.2 (edges, corners, and interest points), Sec. 13.3 (descriptors)
  • SIFT [PDF]
  • Matrix calculus: wikipedia

Additional resources:
Slide 6
Lecture 6
February 12 7. SIFT (Part 2)

February 15 8. IMAGE STITCHING

Reading:
Additional resources:
  • Automatic Panoramic Image Stitching Using Invariant Features: pdf
  • Image Alignment and Stitching: A Tutorial: pdf
Lecture 8
February 19 9. CAMERA GEOMETRY

Reading:
  • Textbook chapter 14 (Pinhole Camera): sect. 14.1 (the pinhole camera), 14.3 (homogeneous coordinates),
Lecture 9
February 22 10. CAMERA CALIBRATION + TWO-VIEW GEOMETRY

Reading:
  • Textbook chapter 14 (Pinhole Camera): sect. 14.2 (three geometric problems), sect. 14.4 (learning extrinsic parameters), sect. 14.5 (learning intrinsic parameters)

RECESS WEEK

March 5 11. DEPTH ESTIMATION

March 8 12. NEURAL NETWORK: INTRODUCTION

March 12 13. NEURAL NETWORK: BACKPROPAGATION | DEEP LEARNING: OPTIMIZATION

March 15 14. CONVOLUTIONAL NEURAL NETWORKS

March 19 15. DEEP GENERATIVE NETWORKS: AUTOENCODERS

March 22 18. GENERATIVE ADVERSARIAL NETWORKS

March 26 19. RECURRENT NEURAL NETWORKS

March 29 20. DEEP BELIEF NETWORKS

April 2 21. IMAGE EDITING: IMAGE INPAINTING

April 5 22. IMAGE EDITING: GRADIENT SPACE MANIPULATION

April 9 23. OPTICAL FLOW 1

April 12 24. OPTICAL FLOW 2

April 16 25. LOW LEVEL VISION

April 19 25. REVIEW + QA SESSION






Syllabus: