Publications

2024

FedAvP: Augment Local Data via Shared Policy in Federated Learning

Minui Hong, Junhyeog Yun, Insu Jeon and Gunhee Kim
NeurIPS 2024 PDF Code

Sample Selection via Contrastive Fragmentation for Noisy Label Regression

Chris Dongjoo Kim*, Sangwoo Moon*, Jihwan Moon, Dongyeon Woo and Gunhee Kim (* equal contribution)
NeurIPS 2024 PDF Code

Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback

Fatemeh Pesaran, Juyeon Kim, Jin-Hwa Kim and Gunhee Kim
EMNLP 2024 (Oral) PDF Code

DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG

Jinyoung Kim, Dayoon Ko and Gunhee Kim
EMNLP 2024 PDF Code

When Meta-Learning Meets Online and Continual Learning: A Survey

Jaehyeon Son*, Soochan Lee* and Gunhee Kim (* equal contribution)
TPAMI 2024 PDF

Meta-Learning Approach for Joint Multimodal Signals with Multimodal Iterative Adaptation

Sehun Lee*, Wonkwang Lee* and Gunhee Kim (* equal contribution)
TMLR 2024 PDF Code

Spherical World-Locking for Audio-Visual Localization in Egocentric Videos

Heeseung Yun, Ruohan Gao, Ishwarya Ananthabhotla, Anurag Kumar, Jacob Donley, Chao Li, Vamsi Krishna Ithapu, Calvin Murdock and Gunhee Kim
ECCV 2024 PDF Code

Bi-directional Contextual Attention for 3D Dense Captioning

Minjung Kim, Hyungsuk Lim, Soonyoung Lee, Bumsoo Kim and Gunhee Kim
ECCV 2024 (Oral) PDF Code

Who Wrote this Code? Watermarking for Code Generation

Taehyun Lee*, Seokhee Hong*, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin and Gunhee Kim (* equal contribution)
ACL 2024 PDF Code

TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models

Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jinhwa Kim, Sangdoo Yun, Hwaran Lee and Gunhee Kim
ACL 2024 (Findings) PDF Code

See It All: Contextualized Late Aggregation for 3D Dense Captioning

Minjung Kim, Hyungsuk Lim, Seunghwan Kim, Soonyoung Lee, Bumsoo Kim and Gunhee Kim
ACL 2024 (Findings) PDF Code

GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge?

Dayoon Ko, Jinyoung Kim, Hahyeon Choi and Gunhee Kim
ACL 2024 PDF Code

Learning to Continually Learn with the Bayesian Principle

Soochan Lee, Hyeonseong Jeon, Jaehyeon Son and Gunhee Kim
ICML 2024 PDF Code

ESR-NeRF: Emissive Source Reconstruction Using LDR Multi-view Images

Jinseo Jeong, Junseo Koo, Qimeng Zhang and Gunhee Kim
CVPR 2024 PDF Code

Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning

Yeda Song*, Dongwook Lee* and Gunhee Kim (* equal contribution)
ICLR 2024 PDF Code

2023

mRedditSum: A Multimodal Abstractive Summarization Dataset of Reddit Threads with Images

Keighley Overbay, Jaewoo Ahn, Fatemeh Pesaran, Joonsuk Park and Gunhee Kim
EMNLP 2023 PDF Code

SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization

Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap and Yejin Choi
EMNLP 2023 PDF Code

FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions

Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi and Maarten Sap
EMNLP 2023 PDF Code

Can Language Models Laugh at YouTube Short-form Videos?

Dayoon Ko, Sangho Lee and Gunhee Kim
EMNLP 2023 PDF Code

Recasting Continual Learning as Sequence Modeling

Soochan Lee, Jaehyeon Son and Gunhee Kim
NeurIPS 2023 PDF Code

Federated Learning via Meta-Variational Dropout

Insu Jeon, Minui Hong, Junhyeog Yun and Gunhee Kim
NeurIPS 2023 PDF Code

EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large Scale Visual Localization

Minjung Kim, Junseo Koo and Gunhee Kim
ICCV 2023 PDF Code

Dense 2D-3D Indoor Prediction with Sound via Aligned Cross-Modal Distillation

Heeseung Yun*, Joonil Na* and Gunhee Kim (* equal contribution)
ICCV 2023 PDF Code

SQuARe: A Large Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human Machine Collaboration

Hwaran Lee*, Seokhee Hong*, Joonsuk Park, Takyuong Kim, Meeyoung Cha, Yejin Choi, Byoungpil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park and Jung-Woo Ha (* equal contribution)
ACL 2023 PDF Code

Recursion of Thought: A Divide-and-Conquer Approach to Multi Context Reasoning with Language Models

Soochan Lee and Gunhee Kim
ACL 2023(Short, Findings) PDF Code

MPCHAT: Towards Multimodal Persona-Grounded Conversation

Jaewoo Ahn, Yeda Song, Sangdoo Yun and Gunhee Kim
ACL 2023 PDF Code

KoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications

Hwaran Lee*, Seokhee Hong*, Joonsuk Park, Takyuong Kim, Gunhee Kim and Jung-Woo Ha (* equal contribution)
ACL 2023(Industry Track) PDF Code

Fusing Pre-trained Language Models with Multimodal Prompts through Reinforcement Learning

Youngjae Yu*, Jiwan Chung*, Heeseung Yun, Jack Hessel (AI2), Jaesung Park (UW), Ximing Lu (UW), Prithviraj Ammanabrolu (AI2), Rowan Zellers (AI2), Ronan Le Bras (AI2), Gunhee Kim and Yejin Choi (UW, AI2) (* equal contribution)
CVPR 2023 PDF Code

2022

Neural Variational Dropout Processes

Insu Jeon, Youngjin Park and Gunhee Kim
ICLR 2022 PDF Code

Lipschitz-constrained Unsupervised Skill Discovery

Seohong Park, Jongwook Choi, Jaekyeom Kim, Honglak Lee and Gunhee Kim
ICLR 2022 PDF Code

2021

Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

Seohong Park, Jaekyeom Kim and Gunhee Kim
Neurips 2021 PDF Code

Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes

Hyunwoo Kim, Byeongchang Kim and Gunhee Kim
EMNLP 2021 PDF Code

Viewpoint-Agnostic Change Captioning with Cycle Consistency

Hoeseong Kim, Jongseok Kim, Hyungseok Lee (Hyundai Motors), Hyunsung Park (Hyundai Motors) and Gunhee Kim
ICCV 2021 PDF

Pano-AVQA: Grounded Audio-Visual Question Answering on 360° Videos

Heeseung Yun, Youngjae Yu, Wonsuk Yang (Oxford University), Kangil Lee (Hyundai Motors) and Gunhee Kim
ICCV 2021 PDF Code

Continual Learning on Noisy Data Streams via Self-Purified Replay

Chris Dongjoo Kim*, Jinseo Jeong*, Sangwoo Moon and Gunhee Kim (* equal contribution)
ICCV 2021 PDF Code

ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning

Sangho Lee*, Jiwan Chung*, Youngjae Yu, Gunhee Kim, Thomas Breuel (NVIDIA), Gal Chechik (NVIDIA) and Yale Song (Microsoft) (* equal contribution)
ICCV 2021 PDF Code

Unsupervised Skill Discovery with Bottleneck Option Learning

Jaekyeom Kim*, Seohong Park* and Gunhee Kim (* equal contribution)
ICML 2021 PDF Code

Unsupervised Representation Learning via Neural Activation Coding

Yookoon Park, Sangho Lee, Gunhee Kim and David Blei
ICML 2021 (Oral) PDF Code

How Robust are Fact Checking Systems on Colloquial Claims?

Byeongchang Kim*, Hyunwoo Kim*, Seokhee Hong and Gunhee Kim (* equal contribution)
NAACL-HLT 2021 PDF Code

Transitional Adaptation of Pretrained Models for Visual Storytelling

Youngjae Yu*, Jiwan Chung*, Jongseok Kim, Heeseung Yun and Gunhee Kim (* equal contribution)
CVPR 2021 PDF Code

StyleMix: Separating Content and Style for Enhanced Data Augmentation

Minui Hong*, Jinwoo Choi* and Gunhee Kim (* equal contribution)
CVPR 2021 PDF Code

Self-Supervised Learning of Compressed Video Representations

Youngjae Yu*, Sangho Lee*, Yale Song and Gunhee Kim (* equal contribution)
ICLR 2021 PDF

SEDONA: Search for Decoupled Neural Networks Toward Greedy Block-wise Learning

Myeongjang Pyeon, Jihwan Moon, Taeyoung Hahn and Gunhee Kim
ICLR 2021 PDF Code

Parameter Efficient Multimodal Transformers for Video Representation Learning

Sangho Lee, Youngjae Yu, Gunhee Kim, Thomas Breuel, Jan Kautz and Yale Song
ICLR 2021 PDF Code

Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration

Jaekyeom Kim, Minjung Kim, Dongyeon Woo and Gunhee Kim
ICLR 2021 PDF Code

IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks

Insu Jeon, Wonkwang Lee, Myeongjang Pyeon and Gunhee Kim
AAAI 2021 PDF

Dual Compositional Learning in Interactive Image Retrieval

Jongseok Kim, Youngjae Yu, Hoeseong Kim and Gunhee Kim
AAAI 2021 PDF Code

2020

Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness

Hyunwoo Kim, Byeongchang Kim and Gunhee Kim
EMNLP 2020 PDF Code

Rethinking Class Activation Mapping for Weakly Supervised Object Localization

Junhyug Noh, Wonho Bae and Gunhee Kim
ECCV 2020 PDF Code

Model-Agnostic Boundary-Adversarial Sampling for Test-Time Generalization in Few-Shot learning

Jaekyeom Kim, Hyoungseok Harry Kim and Gunhee Kim
ECCV 2020 (Oral) PDF Code

Imbalanced Continual Learning with Partitioning Reservoir Sampling

Chris Dongjoo Kim, Jinseo Jeong and Gunhee Kim
ECCV 2020 PDF Code

Character Grounding and Re-Identification in Story of Videos and Text Descriptions

Youngjae Yu, Jongseok Kim, Heeseung Yun, Jiwan Chung and Gunhee Kim
ECCV 2020 (Spotlight) PDF Code

Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue

Byeongchang Kim, Jaewoo Ahn and Gunhee Kim
ICLR 2020 (Spotlight) PDF Code

A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning

Soochan Lee, Junsoo Ha, Dongsu Zhang and Gunhee Kim
ICLR 2020 PDF Code

2019

POL360: A Universal Mobile VR Motion Controller using Polarized Light

Hyouk Jang, Juheon Choi(SOCAR) and Gunhee Kim
ACM VRST 2019 (Best Paper Award - Runner-up) PDF Demo

Self-Routing Capsule Networks

Taeyoung Hahn, Myeongjang Pyeon and Gunhee Kim
NeurIPS 2019 PDF Code

Better to Follow, Follow to be Better: Towards Precise Supervision of Feature Super-Resolution for Small Object Detection

Junhyug Noh, Wonho Bae, Wonhee Lee, Jinhwan Seo and Gunhee Kim
ICCV 2019 PDF Code

Video Question Answering with Spatio-Temporal Reasoning

Yunseok Jang, Yale Song(Microsoft AI & Research), Chris Dongjoo Kim, Youngjae Yu, Youngjin Kim and Gunhee Kim
IJCV 2019 (Volume: 127, Issue: 10, pp: 1385-1412 18 June 2019) DOI: 10.1007/s11263-019-01189-x PDF

Curiosity-Bottleneck: Exploration by Distilling Task-Specific Novelty

Youngjin Kim, Hyunwoo Kim*, Wontae Nam*(KC ML2), Jihoon Kim(NAVER) and Gunhee Kim (* equal contribution)
ICML 2019 PDF Code

Variational Laplace Autoencoders

Yookoon Park, Chris Dongjoo Kim and Gunhee Kim
ICML 2019 PDF Code

Multi-task Self-Supervised Object Detection via Recycling of Bounding Box Annotations

Wonhee Lee, Joonil Na and Gunhee Kim
CVPR 2019 PDF Code

Abstractive Summarization of Reddit Posts with Multi-level Memory Networks

Byeongchang Kim, Hyunwoo Kim and Gunhee Kim
NAACL-HLT 2019 (Oral) PDF Code&Dataset

AudioCaps: Generating Captions for Audios in The Wild

Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee and Gunhee Kim
NAACL-HLT 2019 (Oral) PDF Code&Dataset

Harmonizing Maximum Likelihood with GANs for Multimodal Conditional Generation

Soochan Lee, Junsoo Ha and Gunhee Kim
ICLR 2019 PDF Project Code

Discovery of Natural Language Concepts in Individual Units of CNNs

Seil Na, Yo Joong Choe(KAKAO), Dong-Hyun Lee(KAKAO Brain) and Gunhee Kim
ICLR 2019 PDF Code

2018

A Joint Sequence Fusion Model for Video Question Answering and Retrieval

Youngjae Yu, Jongseok Kim and Gunhee Kim
ECCV 2018 PDF Code

Video Prediction with Appearance and Motion Conditions

Yunseok Jang, Gunhee Kim and Yale Song(Microsoft AI & Research)
ICML 2018 PDF Code

Towards Personalized Image Captioning via Multimodal Memory Networks

Cesc Chunseong Park, Byeongchang Kim and Gunhee Kim
IEEE TPAMI (Volume: 41, Issue: 4, 1 April 2019) DOI: 10.1109/TPAMI.2018.2824816 PDF Code&Dataset

Retrieval of Sentence Sequences for an Image Stream via Coherence Recurrent Convolutional Networks

Cesc Chunseong Park, Youngjin Kim and Gunhee Kim
IEEE TPAMI (Volume: 40, Issue: 4, 1 April 2018) DOI: 10.1109/TPAMI.2017.2700381 PDF Code

Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors

Junhyug Noh, Soochan Lee, Beomsu Kim and Gunhee Kim
CVPR 2018 PDF Code

A Memory Network Approach for Story-based Temporal Summarization of 360° Videos

Sangho Lee, Jinyoung Sung, Youngjae Yu and Gunhee Kim
CVPR 2018 PDF Code&Dataset

A Hierarchical Latent Structure for Variational Conversation Modeling

Yookoon Park, Jaemin Cho and Gunhee Kim
NAACL-HLT 2018 (Oral) PDF Code

Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks

Youngjin Kim, Minjung Kim and Gunhee Kim
ICLR 2018 PDF Code

A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360° Video

Youngjae Yu, Sangho Lee, Joonil Na, Jaeyun Kang and Gunhee Kim
AAAI 2018 (Spotlight) PDF

2017

A Read-Write Memory Network for Movie Story Understanding

Seil Na, Sangho Lee, Jisung Kim(SKT) and Gunhee Kim
ICCV 2017 PDF Project

SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization

Juyong Kim*, Yookoon Park*, Gunhee Kim and Sung Ju Hwang(UNIST) (* equal contribution)
ICML 2017 PDF Project

Attend to You: Personalized Image Captioning with Context Sequence Memory Networks

Cesc Chunseong Park, Byeongchang Kim and Gunhee Kim
CVPR 2017 (Spotlight) PDF Project

TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering

Yunseok Jang, Yale Song(Yahoo Research), Youngjae Yu, Youngjin Kim and Gunhee Kim
CVPR 2017 (Spotlight) PDF Project

Supervising Neural Attention Models for Video Captioning by Human Gaze Data

Youngjae Yu, Jongwook Choi, Sang-Hun Lee and Gunhee Kim
CVPR 2017 PDF

End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answering

Youngjae Yu, Hyungjin Ko, Jongwook Choi and Gunhee Kim
CVPR 2017 (Spotlight)
Winners of two tracks of LSMDC 2016(Annotation and Retrieval and Fill-in-the-Blank) Old paper, Presentation

Detection and Recognition of Text Embedded in Online Images via Neural Context Models

Chulmoo Kang, Gunhee Kim and Suk I. Yoo
AAAI 2017 PDF Project

2016

Taxonomy-Regularized Semantic Deep Convolutional Neural Networks

Wonjoon Goo, Juyong Kim, Gunhee Kim and Sung Ju Hwang(UNIST)
ECCV 2016 PDF Project

A Calibration Method for Optical See-through Head Mounted Displays with a Depth Camera

Hanseul Jun and Gunhee Kim
IEEE VR 2016 PDF Project

2015

Expressing an Image Stream with a Sequence of Natural Sentences

Cesc Chunseong Park and Gunhee Kim
NIPS 2015 PDF Project

Storyline Representation of Egocentric Videos and Its Applications to Story-based Search

Bo Xiong(UT Austin), Gunhee Kim and Leonid Sigal(Disney Research)
ICCV 2015 PDF

Dynamic Topic Modeling for Monitoring Market Competition from Online Text and Image Data

Hao Zhang(CMU), Gunhee Kim and Eric P. Xing(CMU)
KDD 2015 (Research) PDF Project

Discovering Collective Narratives of Theme Parks from Large Collections of Visitors’ Photo Streams

Gunhee Kim and Leonid Sigal(Disney Research)
KDD 2015 (Practice) PDF Project

Ranking and Retrieval of Image Sequences from Multiple Paragraph Queries

Gunhee Kim, Seungwhan Moon(CMU) and Leonid Sigal(Disney Research)
CVPR 2015 PDF Project

Joint Photo Stream and Blog Post Summarization and Exploration

Gunhee Kim, Seungwhan Moon(CMU) and Leonid Sigal(Disney Research)
CVPR 2015 PDF Project

2014

Reconstructing Storyline Graphs for Image Recommendation from Web Community Photos

Gunhee Kim and Eric P. Xing(CMU)
CVPR 2014 (Oral) PDF Project

Joint Summarization of Large-scale Collections of Web Images and Videos for Storyline Reconstruction

Gunhee Kim, Leonid Sigal(Disney Research) and Eric P. Xing(CMU)
CVPR 2014 PDF Project

Visualizing Brand Associations from Web Community Photos

Gunhee Kim and Eric P. Xing(CMU)
WSDM 2014 PDF Project

Discovering Pictorial Brand Associations from Large-Scale Online Image Data

Gunhee Kim and Eric P. Xing(CMU)
LSVisCom workshop with ICCV 2013 PDF Project

2013

Jointly Aligning and Segmenting Multiple Web Photo Streams for the Inference of Collective Photo Storylines

Gunhee Kim and Eric P. Xing(CMU)
CVPR 2013 (Oral) PDF Project

Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression

Gunhee Kim and Eric P. Xing(CMU)
WSDM 2013 PDF Project

2012

Web Image Prediction Using Multivariate Point Processes

Gunhee Kim, Li Fei-Fei(Stanford) and Eric P. Xing(CMU)
KDD 2012 PDF Project

On Multiple Foreground Cosegmentation

Gunhee Kim and Eric P. Xing(CMU)
CVPR 2012 (Oral) PDF Project

2011

Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion

Gunhee Kim, Eric P. Xing(CMU), Li Fei-Fei(Stanford) and Takeo Kanade(CMU)
ICCV 2011 (Oral) PDF Project

2010

Modeling and Analysis of Dynamic Behaviors of Web Image Collections

Gunhee Kim, Eric P. Xing(CMU) and Antonio Torralba(MIT)
ECCV 2010 PDF Project

QMAS: Querying, Mining And Summarization of Multi-modal Databases

Robson L. F. Cordeiro, Fan Guo, Donna S. Haverkamp, James H. Horne, Ellen K. Hughes, Gunhee Kim, Agma J. M. Traina, Caetano Traina Jr and Christos Faloutsos
ICDM 2010 PDF

2009

Unsupervised Detection of Regions of Interest Using Iterative Link Analysis

Gunhee Kim and Antonio Torralba(MIT)
NIPS 2009 PDF Project

Object Recognition with 3D Models

Bernd Heisele, Gunhee Kim and Andrew J. Meyer
BMVC 2009 PDF

2008

Unsupervised Modeling and Recognition of Object Categories with Combination of Visual Contents and Geometric Similarity Links

Gunhee Kim, Christos Faloutsos(CMU) and Martial Hebert(CMU)
ACM MIR 2008 (Oral) PDF Project

Unsupervised Modeling of Object Categories Using Link Analysis Techniques

Gunhee Kim, Christos Faloutsos(CMU) and Martial Hebert(CMU)
CVPR 2008 (Oral) PDF Project

Detection of Salient Regions using Imagery and 3-D Data

Gunhee Kim, Daniel Huber(CMU) and Martial Hebert(CMU)
WACV 2008 (Oral) PDF Project

Line Feature-based Object Recognition for Textureless Indoor Objects

Gunhee Kim and Martial Hebert(CMU)
ICAR 2008 Project