Reconstruction and Applications of Web Photo Storylines
My PhD thesis was chosen as the winner of
2014 SIGKDD doctoral dissertation award. [CMU Link] 45-min talk + ceremony + plaque + $2,500 prize money will be awarded at KDD2014 (Aug 10, 2014).
Dissertation
Gunhee Kim
Reconstruction and Applications of Collective Storylines from Web Photos Collections
Ph.D. Thesis, CMU-CS-13-125, Computer Science Department, Carnegie Mellon University, September, 2013.
[Draft (PDF)] [BibTeX] [Department Link]
Abstract
Widespread access to photo-taking devices and high speed Internet has combined with rampant social networking to produce an explosion in picture sharing on Web platforms. In this environment, new challenges in image acquisition, processing, and sharing have emerged, creating exciting opportunities for research in computer vision and multimedia data mining. In this dissertation, we explore one of these interesting problems, the reconstruction of collective storylines as an efficient but comprehensive structural summary of ever-growing big image data shared online.
More specifically, the goal of this dissertation can be summarized as follows. Given large-scale online image collections and associated meta-data, we aim to create the collective storylines by jointly inferring the temporal trends and the overlapping contents of image collections. We also explore novel computer vision and data mining applications taking advantage of the reconstructed photo storylines.
In order to achieve the proposed research objective, we develop the required technologies from three research directions, which are (1) understanding of temporal trends of image collections, (2) discovery of overlapping contents across image collections, and (3) reconstruction and applications of collective photo storylines. The first direction of the work addresses the problems of understanding what topics are popular when by whom in the image collections, while the second line of the work studies the approaches for detecting salient and recurring contents across the image collections in the form of bounding boxes or pixel-wise segmentations. Finally, based upon the results of the work in the first two directions, we propose the reconstruction algorithms of branching storyline graphs, and explore their promising applications at the intersection of computer vision and multimedia data mining.
Funding
This work is supported by NSF IIS-1115313, NSF IIS-0713379, NSF DBI-0640543, AFOSR FA9550010247, ONR N000140910758, and Google to Eric P. Xing. Some parts of this research are also supported by MURI N00014- 07-1-0747 to Takeo Kanade.