|Labs4: Restoration of old films (assessed!)|
This assignment is comprised of two parts:
You can now upload your submissions on the moodle page.
The goal of this practical is to design and create an algorithm that can be used to restore old films. This is assessed and the deadline is set to Monday, Feb 21st at 23:59.
In their lifetime, films may suffer deterioration due to environmental hazards such as humidity and dust, chemical instabilities and improper storage. They can also get damaged during development and processing. This leads to flicker, camera shaking, blotches, scratches and other artefacts. Old archive footage, such as film of the 1900's, presents those defects: you will have to find ways to correct them. We will deal with four kinds of features visible on films. Each of them constitutes a part in the coursework. This is an exciting topic, and a lot of research has been carried out on this problem. The interested reader can find more information about how the restoration process is done in the literature (for example see papers here, here and here). For the purpose of this coursework assignment, however, simpler solutions are equally accepted. For those of you that want to go beyond the simplest possible solution, some of the subquestions allow one to earn additional bonus points for particularly good solutions.
Detection of scene cuts (10%)
The first part of the coursework consists in detecting scene cuts, which are not defects of the film, but we'll say it's the warm-up and it might also help you for the following sections. By scene cuts we mean moments when the scene changes. They will simply appear as hard transitions since old films don't display fancy effects to switch from a scene to another. Your first task is to write a simple algorithm to detect those transitions. In your output video, you will have to overlay text on the image that will inform the viewer that a scene cut occured.
Correction of global flicker (15% + up to 10% bonus)
Intensity flicker is a common artefact in old films. It is an unnatural temporal fluctuation in perceived image intensity that does not originate from the original scene. In other words, the image will appear suddenly darker/brighter and won't have a constant global luminosity over time. Intensity flicker can be caused by dust, chemical processing, aging of film, copying, and in the case or earlier film cameras, variations in shutter time. To illustrate, we'll have a look at the following consecutive pictures of a film:
It might sometimes not be obvious by looking at still shots, but you will understand better when looking at actual footage. In those three consecutive shots, the image gets suddenly brighter. Your second task is to write an algorithm that can reduce intensity flicker. As you will see in the videos, the flicker is not only global and can be local which is a harder problem to solve. Bonus points will be given to solutions that attempt to reduce the local flicker as well.
Correction of blotches (25% + up to 10% bonus)
Blotches are a common type of artefact that manifests itself as disturbing bright or dark spots caused by dirt and by the loss of the gelatin covering the film, due to aging effects and bad film quality. Let's have a look:
Here I pointed out the most noticeable ones, but there are many more and it is more obvious once more when looking at the movie. You will have to write an algorithm that detects those blotches and removes then. That implies inpainting the part of the image that the blotches were covering, or in other words fill the holes. Bonus points will be given for particularly stable and accurate solutions.
Correction of camera shake (30% + up to 10% bonus)
This is quite self-explanatory: the image just seems to shake. Your last task is to write an algorithm that aligns the frames together in order to remove that unwanted motion. Your output video should be comparatively shake-free. Bonus points will be given for very general or particularly efficient solutions.
Convert a video into a sequence of images
As mentionned before, you will have to convert your chosen video into a sequence of images, eg. footage.avi into a sequence from footage_0000.png to footage_0150.png. It is advised to extract png images, and not jpeg, to avoid further compression.
It is slightly complicated to achieve that on Windows, we advise you to do it on a Linux labs machine or use that to open a linux session. However if you feel that you can handle it you can do the following on your (personal) computer:
It is much simpler:
As mentioned before, if you are a Windows user and you want to work on linux, you can go to http://www.cs.ucl.ac.uk/cgi-bin/csrw and open a new gnome session using your CS login/passwd (Matlab seems to have problem with KDE). Let us know of any problem.
You can now upload your submissions on the moodle page.
The second part of this assigment is a competition. This is optional and the deadline is set to Tuesday, March 1st at 16:00.
The goal, as you might have understood, is to create the best, or most creative, restoration algorithm -- surprise us! The winner will have a chance to present their solution during one of the lectures.
For this task, you will have to correct the film contained in competition.zip.
Unlike the mandatory coursework, there is no restriction, and you can use external libraries mixed with your own code in order to get better results, or you may use other film sources to demonstrate your software. As with the marked coursework, please submit your output video together with your code and a description.
Note: contribution to the competition will not have any influence on the marked component. The competition is not marked.