Effective Use of Frequent Itemset Mining for Image Classification

Basura Fernando*, Elisa Fromont** and Tinne Tuytelaars*

*KU Leuven ESAT-PSI IBBT Belgium

**UMR CNRS 5516 Laboratoire Hubert Curien University of Saint-Etienne France

Abstract

In this paper we propose a new and effective scheme for applying frequent itemset mining to image classification tasks. We refer to the new set of obtained patterns as Frequent Local Histograms or FLHs. During the construction of the FLHs, we pay special attention to keep all the local histogram information during the mining process and to select the most relevant reduced set of FLH patterns for classification. The careful choice of the visual primitives and some proposed extensions to exploit other visual cues such as colour or global spatial information allow us to build powerful bag-of-FLH-based image representations. We show that these bag-of-FLHs are more discriminative than traditional bag-of-words and yield state-of-the art results on various image classification benchmarks.

Fig :Histogram mining process

Example of local histogram mining

In order to avoid information loss and to make patterns more discriminative we directly mine the local histograms. An example of local histogram mining is shown below.

Table :Sample local histogram data. Transaction ID 1,2 and 3 are generated from image I_1 and transaction ID 4, 5 and 6 are generated from image I_2.

Table :Histogram patterns of length 1, 2 and 3 are shown respectively.

Using a support threshold of 2 we select set of closed patterns.

Table :Candidate frequent local histograms or FLHs.

Fig :The difference between BOW histogram and FLH histogram for image I_1 and image I_2. (Left BOW and right FLH)

FLH

There are three key elements for the success of FLH patterns.

• Smaller visual dictionaries
• Less discriminative SIFT features
• larger spatial neighbourhoods

Fig :Effect of neighbourhood size,dictionary size and SIFT feature size on Oxford-Flower-17 dataset.

Fig :Effect of neighbourhood size on several datasets.

Relevant pattern mining is used to select the most suitable set of patterns for classification.

Fig : Some relevant patterns on Oxford-Flower-17 dataset.