HCII PhD Thesis Proposal: Alexandria K. Vail
When
-
Description
"Multimodal and Dyadic Modeling of Client-Therapist Interaction: An Interpretable and Causal Strategy"
Thesis Committee:
Dr. Louis-Philippe Morency (Chair, LTI)
Dr. Jeffrey F. Cohn (University of Pittsburgh)
Dr. Robert Kraut (HCII)
Dr. Adam Perer (HCII)
Abstract:
Productive interaction between client and therapist is central to successful therapy, but is often hindered by substantial challenges along the way. During each therapy session, the therapist is constantly assessing the client's symptoms through their behavior. These behaviors may be expressed through multiple channels: spoken language and "body language". Therefore, the first construct we focus on is the multimodal aspect of behavior. Another fundamental challenge during therapy is the development and maintenance of a collaborative relationship between the client and the therapist. This relationship develops over the course of several weeks, requiring longitudinal study within and across multiple sessions. Thus, the second construct we focus on is the social aspect of behavior. The ambition of this thesis is to address these challenges of long-term psychotherapeutic interaction by approaching behavioral analysis through the lens of both multimodal and social behavior dynamics.
We pursue the challenge of multimodal behavior dynamics through three perspectives: verbal behavior, nonverbal behavior, and cross-modal behavior. This work addresses the difficulty of evaluating client symptoms across multiple modalities. The verbal component of behavior conveys information not only through high-level message intent, but also through more detailed aspects of speech, such as word choice and sentence structure. We present a multifaceted analysis of the client's language use as it relates to their psychological health, including a detailed consideration of lexical, structural, and disfluency components of their speech. The nonverbal component of behavior includes behaviors such as facial expressions, gestures, or eye gaze patterns. In particular, we study the ever-prevalent nonverbal signal of gaze aversion patterns and how they provide considerable information about the severity of the client's symptoms. Building upon this work, we then propose the consideration of cross-modal behavior: we seek to identify what knowledge can be gained from multiple modalities in unison that we cannot gain from single modalities in isolation.
We pursue the challenge of social behavior dynamics in three dimensions: facilitative behavior, convergent behavior, and divergent behavior. This work investigates the growth and decline of the collaborative relationship between the client and therapist over the course of multiple dyadic interactions. Through facilitative behavior, interaction participants attempt to maintain the flow of conversation, such as through turn-taking patterns. We recount a detailed analysis of turn-taking behaviors and mirroring of head gestures as they signal the quality of the collaboration between client and therapist. Through convergent behavior, participants (consciously or subconsciously) coordinate their behavior, such as through linguistic entrainment. We present a modeling of stylistic and content entrainment over multiple sessions as it relates to the client-therapist relationship. The final remaining component that we propose to address is divergent behavior, which occurs with an increase in contrast between the behavior of the participants.
Document @ Google Drive: https://bit.ly/VailProposal