Modeling Dynamic Behaviors of Web Image Collections
People
Publication
Gunhee Kim, Eric P.Xing, and Antonio Torralba
Modeling and Analysis of Dynamic Behaviors of Web Image Collections European Conference on Computer Vision (ECCV 2010), Crete, Greece, September 5-11, 2010. (Acceptance = 322/1174 ~ 27.4%)
[Paper(PDF)] [Poster(PDF)] [Spotlight(PPT)]
Dataset
Images
We used images from Flickr only.
A script for image downloading.
You can google several different scripts to download Flickr images.
I recommend this script.
Description
This work starts from the fact that images on the Web evolve over time. For example, suppose that we download millions of apple images and their associated timestamps from the Flickr and distribute them on the timeline. Then, we can easily obtain a Google-trends-like picture (See Fig.1). The apple topic consists of several subtopics (eg. fruit, tree, laptop, and iPhone). Their popularities change over time. The fruit apple is relatively stationary along the timeline, but iPhone shows a lot of up-and-downs by new events (eg. the release of a new model or Steve Jobs’ presentations).
 Figure 1. The Google trends-like visualization of the subtopic evolution in the apple Flickr images (fruit: blue, logo: red, laptop: orange, tree: green, iPhone: purple). The fruit subtopic is stable along the timeline, whereas the iPhone subtopic is highly fluctuated. |
This paper aims at solving the following research questions about image dynamics.
Q1) How can we model the dynamics of Web images? Obviously, the first question should be about the method to model and analyze image dynamics from millions of Web images.
Q2) Does image dynamics solve existing problems better? Is the temporal context able to improve classification performance?
Q3) Does image dynamics solve novel problems?
Q4) Can images be more effective than texts for topic detection? One may argue that the topic evolution can be detected by associated texts as well. Then, why should we concern about images instead of texts?
Our answers to these questions are as follows.
A1) We propose a nonparametric method, which represents the millions of images with timestamps in the form of similarity networks by using Sequential Monte Carlo. Thus, diverse dynamics analysis can be performed by simple link analysis methods.
A2) We show that training using temporal association improves the classification performance.
A3) We carry out subtopic outbreak detection that points out when the evolution rapidly changes. In the example of Fig.2, we can estimate the degree of subtopic variations by a simple information-theoretic measure. At the highest peak, we can observe the abrupt changes of subtopic distributions in the image domain.
A4) We make sure that the images can be a more reliable and delicate source of information to detect topical evolution than the texts. See Fig. 3 for the example of grandcanyon, one of the most stationary and coherent topics. Unlike the images, the associated tag texts fail to capture this stationary dynamic behavior due to noisy human labeling.
 |
 Figure 3. The comparison between the topical analysis on the images and associated text tags. (a) The variation of KL divergences for the grandcanyon topic. The KL divergences of images are stationary along the timeline whereas those of texts are highly fluctuated. (b) The subtopic changes around the two highest peaks A (05-Nov-2007) and B (16-Aug-2009). Very little visual variation is observed between them. |
Funding
NSF IIS-0713379, DBI-0546594, Career Award, ONR N000140910758, DARPA NBCH 1080007 and Alfred P. Sloan Foundation awarded to Eric P. Xing.
NSF Career Award IIS 0747120 to Antonio Torralba.