Visual query expansion with or without geometry: refining local descriptors by feature aggregation

Giorgos Tolias (1) , Hervé Jégou (1)

1 TEXMEX - Multimedia content-based indexing

This paper proposes a query expansion technique for image search that is faster and more precise than the existing ones. An enriched representation of the query is obtained by exploiting the binary representation offered by the Hamming Embedding image matching approach: The initial local descriptors are refined by aggregating those of the database, while new descriptors are produced from the images that are deemed relevant. The technique has two computational advantages over other query expansion techniques. First, the size of the enriched representation is comparable to that of the initial query. Second, the technique is effective even without using any geometry, in which case searching a database comprising 105k images typically takes 80 ms on a desktop machine. Overall, our technique significantly outperforms the visual query expansion state of the art on popular benchmarks. It is also the first query expansion technique shown effective on the UKB benchmark, which has few relevant images per query.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Giorgos Tolias, Hervé Jégou. Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern Recognition, 2014. ⟨hal-00971267⟩