(Photo by Katie Lenhart / Dartmouth)

I am an Assistant Professor in the Department of Computer Science at Dartmouth College. Before joining Dartmouth, I was a postdoctoral associate at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT), working with Dr. Aude Oliva. Previously I earned my PhD in College of Information and Computer Sciences (CICS), University of Massachusetts, Amherst (UMass Amherst), where I researched on improving face clustering in videos under Dr. Erik Learned-Miller in the Computer Vision Lab.

My main research area is in Computer Vision, Machine Learning and Cognitive Science. My area of expertise is video understanding. As a computer vision researcher, one of my ultimate aspirations is to build an AI system that can understand and respond to the richness of human experience and emotion, much like Jarvis in the Iron Man movie. Here is my CV.

I will serve as an Area Chair at WACV 2025.

I am looking for highly motivated students! If you want to join our lab, please apply to the Dartmouth CS PhD Program and mention SouYoung Jin on your application. The deadline is December 15, 2024. You do not need to email me.
If you are a Dartmouth student and would like to chat, feel free to schedule an appointment with me via this appointment page or please send your CV to my email address.

Thank you for attending New England Computer Vision Workshop (NECV) 2023! See more stories at [Dartmouth News].

Publications

(* indicates equal contribution)
(† denotes equal senior author contribution)

Preprints

  • Wayner Barrios and SouYoung Jin. Multi-layer Learnable Attention Mask for Multimodal Tasks. Submitted. [arXiv]

  • Benjamin Lahner, Kshitij Dwivedi, Polina Iamshchinina, Monika Graumann, Alex Lascelles, Gemma Roig, Alessandro Thomas Gifford, Bowen Pan, SouYoung Jin, Ratan Murty, Kendrick Kay, Aude Oliva†, Radoslaw Cichy†. BOLD Moments: modeling short visual events through a video fMRI dataset and metadata. Submitted. [bioRxiv] [project page]

Peer-Reviewed

  • Bowen Pan, Rameswar Panda, SouYoung Jin, Rogerio Feris, Aude Oliva, Phillip Isola, and Yoon Kim. LangNav: Language as a Perceptual Representation for Navigation. Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) 2024 (Findings). [paper] [MIT News]

  • Howard Zhong, Samarth Mishra, Donghyun Kim, SouYoung Jin, Rameswar Panda, Hilde Kuehne, Leonid Karlinsky, Venkatesh Saligrama, Aude Oliva, and Rogerio Feris. Learning Human Action Recognition Representations Without Real Humans. Neural Information Processing Systems Datasets and Benchmarks Track 2023. [paper] [supp] [project page]

  • Camilo Luciano Fosco, SouYoung Jin, Emilie L Josephs, and Aude Oliva. Leveraging Temporal Context in Low Representational Power Regimes. Computer Vision and Pattern Recognition (CVPR), 2023. [paper] [supp] [project page]

  • Yo-whan Kim, Samarth Mishra, SouYoung Jin, Rameswar Panda, Hilde Kuehne, Leonid Karlinsky, Venkatesh Saligrama, Kate Saenko, Aude Oliva, and Rogerio Feris. How Transferable are Video Representations Based on Synthetic Data?. Neural Information Processing Systems Datasets and Benchmarks Track 2022. [paper] [supp]
    [project page] [MIT News]

  • Alexander H Liu, SouYoung Jin, Cheng-I Jeff Lai, Andrew Rouditchenko, Aude Oliva, and James Glass. Cross-Modal Discrete Representation Learning. Annual Meeting of the Association for Computational Linguistics (ACL) 2022.
    oral [paper] [MIT News]

  • Mathew Monfort*, SouYoung Jin*, Alexander Liu, David Harwath, Rogerio Feris, James Glass, and Aude Oliva. Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions. Computer Vision and Pattern Recognition (CVPR), 2021.
    [paper] [supp] [project page]

  • Ashish Singh*, Hang Su*, SouYoung Jin, Huaizu Jiang, Chetan Manjesh, Geng Luo, Ziwei He, Li Hong, Erik G. Learned-Miller, and Rosemary Cowell. Half&Half: New Tasks and Benchmarks for Studying Visual Common Sense. CVPR 2019 Workshop on Vision Meets Cognition, 2019. [paper]

  • Aruni RoyChowdhury, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, and Erik Learned-Miller. Automatic adaptation of object detectors to new domains using self-training. Computer Vision and Pattern Recognition (CVPR), 2019.
    [paper] [project page]

  • SouYoung Jin*, Aruni RoyChowdhury*, Huaizu Jiang, Ashish Singh, Aditya Prasad, Deep Chakraborty, and Erik Learned-Miller. Unsupervised hard example mining from videos for improved object detection. European Conference on Computer Vision (ECCV), 18 pages, 2018.
    [paper] [project page]

  • SouYoung Jin, Hang Su, Chris Stauffer, and Erik Learned-Miller. End-to-end face detection and cast grouping in movies using Erdos-Renyi clustering. International Conference on Computer Vision (ICCV), 10 pages, 2017.
    spotlight [paper] [supp] [code] [project page]

  • Sou-Young Jin, Ho-Jin Choi, and Yu-Wing Tai. A Randomized Algorithm for Natural Object Colorization. Computer Graphics Forum (Special Issue on Eurographics), vol. 33, no. 2, 2014. [paper] [supp]

  • Sou-Young Jin, Suwon Lee, Nur Aziza Azis, and Ho-Jin Choi. Jigsaw Puzzle Image Retrieval via Pairwise Compatibility Measurement. International Conference on Big Data and Smart Computing (BigComp), pages 123 – 127, Bangkok, Thailand, Jan. 15-17, 2014. [paper]

  • Sou-Young Jin and Ho-Jin Choi. Essential Body-Joint and Atomic Action Detection for Human Activity Recognition Using Longest Common Subsequence Algorithm. Lecture Notes in Computer Science, vol. 7729, pages 148-159, 2013. [paper]

  • Sou-Young Jin, Ho-Jin Choi, and Youssef Iraqi. Depth Consistency Evaluation for Error-Pose Detection. International Conference on Machine Vision (ICMV), 15-16 November 2013. [paper]

  • Sou-Young Jin, Ho-Jin Choi. Clustering Space-Time Interest Points for Action Representation. International Conference on Machine Vision (ICMV), 15-16 November 2013. [paper]

  • Sou-Young Jin, Young-Seob Jong, Chankyu Park, Kyo-Joong Oh, and Ho-Jin Choi. An Intelligent Multi-Sensor Surveillance System for Elderly Care. Smart Computing Review, vol. 2, no. 4, August 2012. [paper]

  • Sou-Young Jin, Jae-Kyung Won, Hojin Lee, and Ho-Jin Choi. Construction of Automated Screening System to Predict Breast Cancer Diagnosis and Prognosis. Basic and Applied Pathology, vol. 5, issue 1, pages 15-18, 16 Mar. [paper]

Teaching

        • COSC 74/274 Machine Learning and Statistical Data Analysis . W2023, W2024
        • COSC 78/278 Deep Learning . F2024
        • COSC 89.30/189 Video Understanding. S2023, W2024