Image search in Flickr is a recommender system. This allows users to search for an image that are similar in features. Unlike the text or document search, Flickr images search uses advance technology or algorithm called “semantic content similarity based” as it requires deep understanding of image content for which it employs deep neural networks.
The neural network is train to map the raw pixels of a photo for example, into a set of relevant tags which later transform to vectors of number that are corresponding to the pixel intensities. It flaws is that it will be constrained by the task the network itself was trained to perform, i.e., scene recognition.
An alternative is the Locally Optimized Product Quantization, LOPQ for short. Instead of ranking all the vectors in the index, LOPQ will first filter a set of good candidates and only do expensive distance computation by using k-means algorithm, because this is very effective at allocating cluster centroid that target the distribution of data.