Code for Cross-Modal Retrieval: A Pairwise Classification Approach, SDM 2015

The aim of this MATLAB code is to replicate the tables of results and figures from the paper Cross-Modal Retrieval: A Pairwise Classification Approach, appearing in SDM 2015.

The code comprises a main driver script, main.m, and several additional files organised into the following subfolders:

Example usage

To replicate the table of results from the paper, simply unzip the code, and run:

>> main;

The display window will then fill with the results of training each of the methods on each of the datasets, using previously saved optimal parameters. Sample output:

performing svd...done [1.7209 secs]

=== [ Random ] ===
MAP = 0.0652 0.0652

=== [ Linear regression ] ===
MAP = 0.1317 0.1069

=== [ CCA ] ===
MAP = 0.1681 0.1422

=== [ CMML ] ===
MAP = 0.1802 0.1431


Additionally, the script will produce PR curves for all three datasets.

In order to replicate the results of the case study, you must first download the full image and text of the Wikipedia dataset from here. (This file is ~1.4 GB.) Once this is done, unzip the contents in the libraries subfolder: this should create a new folder called wikipedia_dataset. At this stage, you may simply run:


This script performs 4 image-to-text and text-to-image queries. It displays figures showing the corresponding query and result images for both regimes, for both the cross-product and CCA methods. The filenames of the corresponding query and result texts are reported in the console.

Detailed descriptions

The provided code builds upon that provided by the project pages for the MMM '10 paper of Rasiwasia et al. (reference [15] in our paper), and the CVPR '12 paper of Costa Pereira and Vasconcelos (reference [13] in our paper). In detail, the following subfolders are taken from these sources: In addition, we rely on the following external libraries, copies of which are provided in the libraries/ subfolder:
As noted above, the script main.m used previously saved optimal parameters. Should you wish to search for these optimal parameters, simply change line 40 to read

TUNE = 1;