David Chuan-En Lin

chuanenl[at]cs.cmu.edu

Hi there 👋. I am a 5^th Year Computer Science PhD student at Carnegie Mellon University. I work with Professor Nik Martelaro in the Augmented Design Capability Studio. I did internships at Adobe Research and Runway. My research is also supported by the Toyota Research Institute and Accenture Technology Labs. I am co-organizing the CMU Interactive AI group. I enjoy traveling and recording my experiences.

I am seeking Summer 2025 Research Internship opportunities!

Research Summary

I build tools for designers — augmented with AI.

I create interactive tools for designers to co-create with AI capabilities using intuitive high-level abstractions. For example, ✏️ sketching with AI-generated scaffolding, 🧩 assembling AI model puzzle pieces, and 🗺️ exploring latent space maps. I enjoy working on both interaction design and implementing machine learning pipelines to support these abstractions, coming from my mixed background in Human-Computer Interaction and Computer Vision.

I work on projects spanning various modalities, including 🎞 video, 🖼️ image, 🧱 3D, 🔤 text, 🔊 audio, and ✏️ sketching. In particular, a thread of my research has been on AI-augmented video creation tools, including tools for adding sound effects to visuals, detecting highlight moments, and organizing video footage.

My research impacts design practice by blending AI capabilities into natural creative workflows, improving creative exploration and productivity.

Papers

Inkspire: Sketching Product Designs with AI

David Chuan-En Lin, Hyeonsu B Kang, Nikolas Martelaro, Aniket Kittur, Yan-Ying Chen, Matthew K. Hong

UIST '24 and CHI '24 Workshop

We developed a tool that enables designers to sketch product designs with AI scaffolding beneath the canvas and analogy inspirations.

Short Paper Workshop Paper

Jigsaw: Supporting Designers to Prototype Multimodal Applications by Chaining AI Foundation Models

David Chuan-En Lin, Nikolas Martelaro

CHI '24

We developed a tool for combining AI models across different capabilities and modalities by combining them like puzzle pieces.

Webpage Paper Video Code Cite

VideoMap: Supporting Video Editing Exploration, Brainstorming, and Prototyping in the Latent Space

David Chuan-En Lin, Fabian Caba Heilbron, Joon-Young Lee, Oliver Wang, Nikolas Martelaro

C&C '24 and NeurIPS '22 Workshop

We developed a proof-of-concept video editing interface that operates on video frames projected onto a latent space.

Webpage Full Paper Short Paper Video Cite

Videogenic: Identifying Highlight Moments in Videos with Professional Photographs as a Prior

David Chuan-En Lin, Fabian Caba Heilbron, Joon-Young Lee, Oliver Wang, Nikolas Martelaro

C&C '24 and NeurIPS '22 Workshop

We developed a system for detecting highlight moments by leveraging photographs taken by photographers.

Webpage Full Paper Short Paper Video Cite

Soundify: Matching Sound Effects to Video

David Chuan-En Lin, Anastasis Germanidis, Cristóbal Valenzuela, Yining Shi, Nikolas Martelaro

UIST '23 and NeurIPS '21 Workshop

We developed a system to assist video editors in adding content-aware spatial sound effects to video.

Webpage Full Paper Short Paper Talk Demo Code Cite

Learning Personal Style from Few Examples

David Chuan-En Lin, Nikolas Martelaro

DIS '21

We developed a model for learning personal graphic design style from a handful of examples.

Webpage Paper Talk Cite

ARchitect: Building Interactive Virtual Experiences from Physical Affordances by Bringing Human-in-the-Loop

Chuan-En Lin*, Ta Ying Cheng*, Xiaojuan Ma(* = equal contribution)

CHI '20

We explored an asymmetric workflow of an AR builder and a VR player for creating VR experiences that incorporate real-world interaction affordances.

Webpage Paper Video Cite

SeqDynamics: Visual Analytics for Evaluating Online Problem-solving Dynamics

Meng Xia, Min Xu, Chuan-En Lin, Ta Ying Cheng, Huamin Qu, Xiaojuan Ma

EuroVis '20

We developed an interactive visual analytics system for instructors to evaluate problem-solving dynamics of student learners.

Paper Video Cite

Learning to Film from Professional Human Motion Videos

Chong Huang, Chuan-En Lin, Zhenyu Yang, Yan Kong, Peng Chen, Xin Yang, Kwang-Ting Cheng

CVPR '19

We developed an automatic drone cinematography system by learning from cinematic drone videos captured by professionals.

Paper Video Cite

Fun Projects That Aren't Papers

which frame

search a video semantically

semantic pinching

pinch to change text abstraction

media mash

interpolate multimedia concepts in latent space

pdf2preview— preview thumbnail for papers

foodnet— find complementary food

tiny papers— research paper summaries w/ tim

lane detector— lane detector tutorial

object detection— yolov3 tutorial

optical flow— optical flow tutorial

ib physics— highschool physics notes, 5m views...

ollie— virtual skateboarding with physical skateboard

learn— augmented reality cube

Resources I Found Helpful

grad school adviceby Jason Hong

awesome phd adviceby Paul Liang

cs hci phd opportunitiesby Andrew Kuznetsov

inventing on principleby Bret Victor

the sciences of the artificialby Herbert Simon

mother of all demosby Douglas Engelbart

sketching user experiencesby Bill Buxton

the design of everyday thingsby Don Norman

notes on writingby Fredo Durand

writing research papersby Aaron Hertzmann

research contributions in hciby Jacob Wobbrock

phd comicsby Jorge Cham

Last updated on Dec 2024