CA3 and CA4: Deep Learning (CEG5304)

Deadline: April 14, 2023 at 5pm


Read carefully: This is an individual assignment. Academic integrity must be strictly followed. Copying from other's code/text or from anyone or any source is not allowed (unless stated clearly in the instructions). Exchanging codes/text is not allowed. Software will be used to detect any form of source code plagiarism. Your submitted code must be grouped/separated into the same parts in the instructions. In your submission, you must provide us with all necessary libraries, so that we can compile your code. You are not allowed to use any toolboxes unless stated in the instructions explicitly (if you are not sure, you should ask). If you use separate files in your submission, you must zip them to one file. The submission must not be in separate files in different times (it must be submitted together once). The deadline is strict, so please prepare and plan early and carefully. Late submission will cause score deduction (assuming 100 is the full score, there will be deduction of 10 points in the first 24 hours, 30 points in the next 24 hours, 60 points in the next 24 hours, and 100 points beyond this).

CA3: Style Transfer using Image-to-Image Translation

monet

Figure 1: Style transfer using Image-to-Image Translation

In this assignment, your tasks is to translate from one image to another image, as examples shown in Fig. 1.

CA3-Part 1: PyTorch Code

  1. Download the data from: Kaggle: I’m Something of a Painter Myself.
  2. Use the data to implement an appropriate algorithm to transfer an image into Monet’s styles. You are allowed to use publicly available code. However, it must be based on PyTorch. Moreover, you must cite the source properly in your code and your report.

CA3-Part 2: Report

Write a report in PDF based on the below instructions. You must write down the instruction numbers and also the instructions themselves on your report along with your answers (so that we can identify your answer rather easily).
  1. Write your name and student ID.
  2. Draw the architecture/pipeline. If you use the publicly available code, you must state it clearly and provide the link of the code.
  3. Explain how the method works by discussing the pipeline.
  4. Test your model using the 3 provided images (in folder CA3 Part 2-4 of the provided zip file), and show the results in your report. Here is the provided zip file: download.
  5. Test your model using the 3 provided images (in folder CA3 Part 2-5 of the provided zip file), and show the results in your report.
  6. Compare your results with the real monet paintings (available in the dataset), and state any drawbacks of your results.
  7. Provide any possible solutions for the stated drawbacks or any possible improvements for your method. Provide justifications for your solutions.

CA3-Part 3: Video

  1. Create a video presenting your report with duration of maximum 10 minutes.
  2. In the video, you must use powerpoint to show your slides.
  3. In the video, you must show your face next to your powerpoint slides while you're presenting.
  4. The video must be in the AVI or MP4 format.
  5. The presentation must follow the contents of your report.
* Any recording tool is fine, but you might want to use Zoom to record your presentation.


CA4: Semantic Segmentation

segmentation

Figure 2: Semantic Segmentation

In this assignment, your task is to segment an input image semantically based on the classes, as examples shown in Fig. 2.

CA4-Part 1: PyTorch Code

  1. Find from the internet or create your best model/network for semantic segmentation. You are allowed to use any publicly available code. However, it must be based on PyTorch. Moreover, you must cite the source properly in your code and your report.
  2. Train your network using the The Oxford-IIIT Pet Dataset dataset. If you are using your own GPU, you need to download the dataset from: The Oxford-IIIT Pet Dataset website.

CA4-Part 2: Report

Write a report in PDF based on the below instructions. You must write down the instruction numbers and also the instructions themselves on your report along with your answers (so that we can identify your answer rather easily).
  1. Write your name and student ID.
  2. Draw the architecture/pipeline. If you use the publicly available code, you must state it clearly and provide the link of the code.
  3. Explain how the method works by discussing the pipeline.
  4. Compute the mIoU of your segmentation results with respect to the corresponding ground-truths.
  5. Test your model using the provided 3 pet images (in folder CA4 Part 2-5 of the provided zip file) in different styles, and show the results in your report. Here is the provided zip file: download.
  6. For the previous instruction, if there are any wrong segmentations, discuss why they are wrong.
  7. Test your model using the provided 3 pet images (in folder CA4 Part 2-7 of the provided zip file) that are taken in nighttime, and show the results in your report.
  8. For the previous instruction, if there are any wrong segmentations, discuss why they are wrong.
  9. Provide possible solutions for the nighttime segmentation problem. Provide justifications for your solutions. The more details your solution is, the more points you will get.

CA4-Part 3: Video

  1. Create a video presenting your report with duration of maximum 10 minutes.
  2. In the video, you must use powerpoint to show your slides.
  3. In the video, you must show your face next to your powerpoint slides while you're presenting.
  4. The video must be in the AVI or MP4 format.
  5. The presentation must follow the contents of your report.
* Any recording tool is fine, but you might want to use Zoom to record your presentation.


Tips for GPU Resources

Running on GPU is more efficient for training your models. If you do not have GPU resources, you may consider Google Colab.

Submission

Submit your code, report and video via Canvas by the deadline.