HCII PhD Thesis Proposal: Frederic Gmeiner
When
-
Description
Designing Interactions to Empower Thoughtful Human-AI Co-Creation
Frederic Gmeiner
PhD Thesis Proposal
CMU Human-Computer Interaction Institute
Date & Time: Wednesday, April 16th at 1:50pm ET
Location: Scaife Hall, 309B
Zoom: https://cmu.zoom.us/j/98860086129?pwd=dES9mexAS18VpTaaPV9aHis3uPTo0J.1
Meeting ID: 98860086129
Passcode: 329437
Committee:
Nikolas Martelaro (co-chair), Carnegie Mellon University
Ken Holstein (co-chair), Carnegie Mellon University
Aniket Kittur, Carnegie Mellon University
Christopher McComb, Carnegie Mellon University
Abstract:
Despite Generative AI (GenAI) systems' potential for supporting complex, open-ended creation tasks, such as designing or writing activities, professionals often struggle to integrate GenAI into their workflows effectively. Key challenges include misalignment between AI-generated content and user intentions, difficulty in prompt formulation, and reduced cognitive involvement due to cognitive offloading, leading to insufficient problem exploration and limited ability to evaluate generated outcomes. I argue that to utilize GenAI's potential more effectively in assisting people with complex and cognitively demanding tasks, it is essential to design interactions and systems that better align with and enhance human thought processes.
In this thesis, I explore interaction techniques and systems to assist people in better thinking through and specifying complex problems while enabling more effective human-AI co-creation workflows in the context of AI-assisted mechanical design and rich content creation tasks. My exploration follows a research-backed user-centered design approach, employing observational studies, interactive system development, and novel AI prototyping methods while focusing on two primary interface modalities (personal voice-based agents and adaptive graphical user interfaces). My work builds upon theories and principles from cognitive and learning sciences, such as metacognitive thinking and Socratic dialogue, and concepts of design and creation processes, such as reflection-in-action.
First, by studying how mechanical and architectural designers engage with AI-based CAD tools in multi-objective manufacturing tasks, I identified promising support strategies—particularly metacognitive approaches like reflective questioning and thought externalization—that improved designers' cognitive engagement with GenAI tools. Next, through a Wizard-of-Oz study with mechanical designers, I demonstrated how personal voice agents can enhance critical thinking and GenAI workflow effectiveness by facilitating metacognitive support. Ultimately, this led to the building of SocraBot, a voice-based multimodal interactive agent supporting designers in GenAI-based mechanical design tasks. Challenged with implementing SocraBot atop a pre-trained LLM, I developed novel user-centered prototyping approaches to refine the AI system's behavior through an iterative design process. In parallel, to extend metacognitive support beyond voice modalities, I developed Intent Tags–a graphical interaction technique for AI-assisted content creation that enables users to gradually articulate and refine their intent through graphical, granular, and adaptive UI elements and micro-prompting. An evaluation of IntentTagger, a system implementing this approach for GenAI slide creation, demonstrated that users felt more control and self-efficacy than with conventional chat-based GenAI interactions.
In the proposed work, I will synthesize my prior contributions by developing a new system that blends the complementary strengths of my previous interaction techniques and support strategies—such as reflective questioning, guided externalization, and graphical micro-prompting—into a unified, flexible interface. Finally, I will evaluate the system through user studies in the context of GenAI-assisted manufacturing design tasks.
Proposal Document: https://drive.google.com/file/d/1eI1VDHkhwQtUxSGKVTQb1l9qz3NjmE6E/view?usp=sharing
Best,
Frederic