Image super-resolution is a classical problem which has found application in areas such as video processing, light field imaging and image reconstruction. As applied to spectral imaging, super-resolution is a key task to improve the spatial resolution of imaging spectroscopy data. Moreover, despite the major advantages of new spectral imaging sensors with filters fully integrated into the CMOS or CCD detector, one of their main drawbacks is the low raw spatial resolution per wavelength-indexed band in the image. Thus, super-resolution is an important means to a much improved spatial resolution in these devices.
This challenge consists of two tracks. Both are motivated by the notion that, by using machine learning techniques, single image super-resolution systems can be trained so as to obtain reliable multispectral super-resolved images at testing.
Track 1 (spectral image super-resolution) focuses on to the problem of super-resolving the spatial resolution of spectral images given training pairs of low and high spatial resolution spectral images.
Track 2 (colour-guided spectral image super-resolution) aims to leverage the increased spatial resolution of colour (RGB) cameras and the link between spectral and trichromatic images of the scene. It will using both training pairs of low and high spatial resolution spectral images and also aligned colour images in low and high resolution.
At runtime it is assumed as input a low spatial resolution spectral image and its aligned colour image in high spatial resolution.
In this track, the aim is to obtain 3x spatially super-resolved spectral images making use of training imagery which was down-sampled with a bicubic kernel. For all the spectral images we compute the corresponding pseudocolour (RGB) image making use of the CIE 2-degree colour matching functions. We remit the interested reader to the paper of [Fairman et al.] for a motivation on the principles for the CIE colorimetry system used here.
In this track, the aim is to obtain 3x spatially super-resolved spectral images and pseudo-colour images making use of stereo pairs. The paired images are pixel-wise aligned and were acquired by a stereo setting formed from a spectral camera and a colour camera. The images were down-sampled with a bicubic kernel.
All submitted images will be evaluated with respect to two criteria. The first of these concerns the fidelity of the reconstruction of the spectra in the super-resolved spectral images. The second applies to the perceptual evaluation of the pseudocolour images at testing (the colour analogue of the spectral imagery).
For the perceptual evaluation, the mean opinion score (MOS) will be used. For the spectral image fidelity evaluation, the results will be evaluated using the Spectral Information Divergence (SID) and the mean of relative absolute error (MRAE). The SID is an information theoretic measure which assesses similarity making use of the probabilistic discrepancy between the spectra under consideration and has been widely used in the literature on hyperspectral image data processing. For each of these, the winner is then that who achieves the lower error or divergence. For both measures, the scores are obtained using the per-pixel average over the testing images.
All algorithms will be evaluated on the corresponding testing images. Note that, even though the testing ground truth is not provided with the datasets, participants can evaluate their results on the validation sets using the code found at (coming during the validation phase). For the validation of the perceptual results, we will provide participants with SSIM scripts.
Submitted results on the testing images will be displayed on the leaderboard. After registering, participants will receive submission instructions. During the validation phase (until July 17th), each team will be limited to 20 validation submissions in total. All submitted spectral images should be ENVI standard (for more info on ENVI files go to here), with resolution 480 by 240, in band sequential format and 16 bits per sample (data type 12). All the pseudoclour images are expected to be 960 by 480 pixels in png, 16 bit format.
Track 1: Spectral Image Super-Resolution: The dataset for this track consists of 240 different spectral images which have been subsampled by a factor of three. All the images are in band-sequential, 16 bit, ENVI standard format and were captured using a multispectral camera based on the IMEC 16-band snapshot sensor in the visible range. The 240 images have been split into 200 for training, 20 for validation and 20 for testing on self explicatively named directories. On the ZIP file, we also provide a Readme file and Matlab routines to read and write ENVI standard files to disk and compute pseudocolour images using the CIE 1931 2-degree colour matching functions.
Track 2: Colour-guided Spectral Image Super-resolution: The dataset contains 120 image stereo pairs, where one view is captured by an IMEC 16-band snapshot sensor and the other one by a colour camera. The images have been subsampled by a factor of three and split into 100 pairs for training, 10 for validation and 10 for testing on self explicatively named directories. For convenience, we have also provided a Readme PDF, a registered version of the colour image obtained using FlowNet 2.0 and Matlab routines to read and write ENVI standard files to disk and compute pseudocolour images using the CIE 1931 2-degree colour matching functions.
Final submission In addition to the test set results, each team must submit: (i) a test code/executable for reproducing the results, and (ii) a fact sheet describing the method (a format will be released).
Final results and ranking The final results will be announced at the PIRM Workshop (in conjunction with ECCV'18).
Paper submission (optional) Challenge participants are invited to submit papers for the ECCV 2018 workshop proceedings. Papers will be accepted based on: (i) academic quality, and (ii) challenge ranking. The length limit is 14 pages (excluding references) in ECCV format.
Date |
Event |
|---|---|
June 8th 2018 |
Dataset released |
July 9th 2018 |
Validation server online |
August 9th 2018 |
Test results and fact sheets submission deadline |
August 11th 2018 |
Code submission deadline |
August 14th 2018 |
Final test results released to the participants |
August 22nd 2018 |
Paper submission deadline |
September 5th 2018 |
Notification of accepted papers |
September 14th 2018 |
PIRM 2018 Workshop |
TBD |
Camera-ready deadline |
Those intending to participate in the challenge must register for Track 1 (Spectral Image Super-Resolution) and/or Track 2 (Colour-guided Spectral Image Super-resolution). Once registered, the participants will be allowed to download the data and submit results once the submission stage opens.
Please note that:
* Names and affiliations of participants will remain private during the challenge validation and test stages (only the group name will appear).
* The submitted results may be assessed, for comparison purposes, using other error metrics. These metrics will be used for comparison only.
* The challenge organizers, their close collaborators and the researchers affiliated with the challenge organizers' labs cannot participate in the challenge.
Rank |
Team |
SID |
RMAE |
MOS |
|---|---|---|---|---|
1 |
||||
2 |
||||
3 |
Rank |
Team |
SID |
RMAE |
MOS |
|---|---|---|---|---|
1 |
||||
2 |
||||
3 |
Rank |
Team |
SID |
RMAE |
MOS |
|---|---|---|---|---|
1 |
||||
2 |
||||
3 |
