Assignment 1: Panoramic Image Stitching

Deadline: October 13, 2023 at 5pm



Read carefully: This is an individual assignment. Academic integrity must be stricly followed. Copying from other's code or froms any sources is not allowed. Exchanging codes is not allowed. Software will be used to detect any form of source code plagiarism. The maximum score for completing part 1 until part 5 is 80. More scores are given for submission on part 6. Your submitted code must be grouped/separated into the same parts in the instructions. In your submission, you must provide us with all necessary libraries. You are not allowed to use any toolboxes (if you are not sure, you should ask). If you use separate files in your submission, you must zip them to one file. The submission must not be in separate files in different times (it must be submitted together). The deadline is a strict deadline, so please prepare and plan early and carefully. Late submission will be deducted 10 points (out of 100) for every 24 hours. Zero marks for late submissions of more than 5 days.

Programming language: Python version 3. Submission: Jupyter Notebook (any other format won't be accepted). In your Jupyter notebook file, you must compile the codes and show the results there in the file.
See the frequently asked question page if you have any doubt about the instructions: FAQs

Part 1: 2D Convolution + SIFT

  1. Write a program for the image 2D convolution. It should be able to process any grayscale input image, and operate using the following kernels:
    • Sobel kernel,
    • Gaussian kernel,
    • The 5 Haar-like masks (note, for this part, the user should be able to set the scale of the masks)
  2. Show the results of the convolution with each of the kernels above on this image: URL.
  3. Write your conclusions on the outputs of each of the kernels, and explain how the size of the kernel influences the outputs.
  4. Download the images from here: link. Extract SIFT keypoints from im01.jpg using any publicly available SIFT code. Show the keypoints on the images, like shown below (similar but not necessarily the same):
    input

Part 2: Homography

For this part, you must write your own code for computing the homography matrix and transforming an image.
  1. Download h1.jpg and h2.jpg. Write a code to show a GUI (graphical user interface) where the user can click 4 points on an image. Let the user to choose 4 points on h1.jpg and 4 points on h2.jpg
    input input
  2. Compute the homography matrix from h1.jpg to h2.jpg, and show the homography matrix. Use the homography matrix to tranform h1.jpg to h2.jpg, and show the result:
    input
  3. Compute the homography matrix from h2.jpg to h1.jpg, and show the homography matrix.
  4. Use the second homography matrix to tranform h2.jpg to h1.jpg, and show the result:
    input
  5. Download the two images below: simpsons.jpg and bus.jpg. Ask the user to select the points for transforming the simpsons image to the advertisement area of the bus. Compute the homography, transform the image, and show the transformation result, similar to:

    input

Part 3: Manual Homography + Sticthing

  1. Load im01.jpg and im02.jpg. Let the user to choose 4 points on im01.jpg and 4 points on im02.jpg
    input input
  2. Compute the homograhy matrix, and stitch the images together using the computed homography matrix and show the result:
    input
  3. Explain why there is an effect of double edges in the overlapping regions between im01.jpg and im02.jpg

Part 4: Homography + RANSAC

  1. Load im01.jpg and im02.jpg. Based on SIFT, find matches between the two images. Show all the matches, similar to:
    input
  2. Compute the best homography matrix using RANSAC, and show all the inlier matches (matches that support your best homography matrix):
    input
  3. Stitch the images together using the best homography matrix and show the result:
    input input

Part 5: Basic Panoramic Image

For this part and the next part, you can use any set of images (even images taken by yourself) that support your implementation. You must include the images in your submission.
  1. Using SIFT and RANSAC for computing the homography matrices, create a program to generate panoramic images given ordered images. Using as many provided images as you can, show the output of your panoramic image.
  2. The figure below is generated from 3 images and is just an example. You must use as many images as you can.
    input

Part 6: Advanced Panoramic Image

You will receive more scores if you include the following implementation in your submission (the score depends on the complexity of the problems and your solutions). To be graded, you must provide some explanation about your implementation and results.
  1. Handling unordered images
  2. Showing the error accumulation problem and implementing the bundle adjustment for solving the problem
  3. Warping, straigthening, gain compensation and multiband blending
  4. Other problems that are beyond the basic panoramic image in Part 5.
Note, for the first three problems, see paper 1 (Automatic Panoramic Image Stitching using Invariant Features) and for the paralax problem see paper 2 (Parallax-tolerant Image Stitching). No consultation is provided for this part.

Submission:

Submit your Jupyter notebook file (containing: codes, results and answers of the questions) via Canvas. Your codes must be grouped/separated based on the parts above. Deadlines are strict. Late submission will be deducted 10 points (out of 100) for every 24 hours. Zero marks for late submissions of more than 5 days.