Log in / Register
Home arrow Computer Science arrow Virtual, Augmented and Mixed Reality
< Prev   CONTENTS   Next >

4 Visual Search Optimization with Saliency Based Metric

4.1 Overview

For mobile visual search (MVS) applications, most of existing methods use all keypoints detected from a given image, including those in unimportant regions such as small decoration or text areas. Different from state-of-the-art methods, our approach reduces the number of local features instead of reducing the size of each descriptor. Only keypoints with meaningful information are considered. As our method is independent of the choice of features, the combination of our idea with compact visual descriptors will give more efficiency.

Fig. 4. Our approach to detect a page in a printed lecture note or textbook

We propose the idea to utilize the saliency map of an image to quickly discard keypoints in unimportant or insensitive regions of a template image as well as a query image (c.f. Figure 4). The visual sensitivity of each region is evaluated to determine keypoints to be preserved and those to be removed. This helps to reduce computational cost in local feature extraction of an image. As keypoints in unimportant regions can be removed, the accuracy of visual object recognition can also be improved.

Figure 5 shows our proposed method with two main steps. First, an arbitrary image is decomposed into perceptually homogeneous elements. Then, saliency maps are derived based on the contrast of those elements. The proposed saliency detection algorithm is inspired by the works in object segmentation with image saliency [10]. In our approach, regions of interest can be discrete and there is no need of merging.

Fig. 5. Pre-processing phase

4.2 Image Abstraction

To simplify illustrations from color images, visual contents are abstracted by region based segmentation algorithms. A region grows by adding similar neighboring pixels according to certain homogeneity criteria, increasing size of region gradually. The proposed algorithm for this phase includes two steps: Over-Segmentation (c.f. Figure 5.B) and Region Growing (c.f. Figure 5.C).

Over-Segmentation: An image is over-segmentation by the watershed-like method. The regions are merged on the basis of a similarity color criterion afterwards:I1ci cj1I2:s;where ci and cj are pixels in the same region.

Region Growing: Neighboring segments are merged based on their sizes, which are the number of pixels of each region. If a region whose size is below a threshold, it is merged to its nearest region, in terms of average Lab color distance. To speed up, we use Prims algorithm [11] to optimize merging regions.

4.3 Visual Saliency Estimation

An image captured from a camera is intentionally focused on meaningful regions by human vision which reacts to regions with features such as unique colors, high contrast, or different orientation. Therefore, to estimate the attractiveness, the contrast metric is usually used to evaluate sensitivity of elements in image.

A region with high level of contrast with surrounding regions can attract human attention and is perceptually more important. Instead of evaluating the contrast difference between regions in an original image, the authors only calculate the contrast metric based on Lab color between regions in the corresponding segmented image. As the number of regions in the original image is much more than the number of regions in its corresponding segmented image, our approach not only simplifies the calculation cost but also exploits the meaningful regions in the captured image efficiently.

The contrast Ci of a region Ri is calculated as the difference between Lab color of Ri and its surrounding regions:

where cj and ci are Lab colors of regions Rj and Ri respectively, and Rj is the number of pixels in region Ri. Regions with more pixels contribute higher localcontrast weights than those containing only a few pixels. Finally, Ci is normalized to the range [0,1]. Figure 6 shows that our method can provide better results than existing saliency calculation techniques.

Fig. 6. Visual comparison between the proposed method and other state-of-the-art methods

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
Business & Finance
Computer Science
Language & Literature
Political science