CMU at CHI 2022

April 25, 2022
CHI 2022 logo: a purple and gold jester mask since it is held in New Orleans, LA

The international ACM Conference on Human Factors in Computing Systems (also known as CHI) begins this week. CHI, the premier international conference on Human-Computer Interaction, will take place from April 30 to May 5, 2022, in New Orleans, LA.

"CHI is an important conference because researchers from so many areas of HCI come together. It's a place where ideas can cross-pollinate and researchers from different subfields can connect. With this year's hybrid conference, we're looking forward to the resumption of old relationships and the development of new collaborations," said Jessica Hammer, Thomas and Lydia Moran Associate Professor of Learning Science and Interim Associate Director of the HCI Institute.

More than one hundred researchers from Carnegie Mellon have registered to attend this year's hybrid CHI event.

We celebrate the following awards and research affiliated with Carnegie Mellon University authors at CHI 2022:

CHI Academy Award
Professor Niki Kittur was named to the 2022 CHI Academy, an honorary group of individuals who have made substantial contributions to the field of human-computer interaction (HCI).

Accepted Papers and Journals
Carnegie Mellon University authors contributed to more than 40 accepted papers this year, including three Best Paper and six Honorable Mention awards. A list of papers with CMU contributing authors is available below.

Accepted Papers

"'It's Kind of Like Code-Switching': Black Older Adults' Experiences with a Voice Assistant for Health Information Seeking" -- Best Paper Award


Christina Harrington, Radhika Garg, Amanda Woodward, Dimitri Williams


Black older adults from lower socioeconomic environments are often neglected in health technology interventions. Voice assistants have a potential to make healthcare more accessible to older adults, yet, little is known about their experiences with this type of health information seeking, especially Black older adults. Through a three-phase exploratory study, we explored health information seeking with 30 Black older adults in lower-income environments to understand how they ask health-related questions, and their perceptions of the Google Home being used for that purpose. Through our analysis, we identified the health information needs and common search topics, and discussed the communication breakdowns and types of repair performed. We contribute an understanding of cultural code-switching that has to be done by these older adults when interacting with voice assistants, and the importance of such phenomenon when designing for historically excluded groups.


"Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels" -- Best Paper Award


Jochen Görtler, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, Donghao Ren, Rahul Nair, Marc Kirchner, Kayur Patel


The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We conduct formative research with machine learning practitioners at Apple and find that conventional confusion matrices do not support more complex data-structures found in modern-day applications, such as hierarchical and multi-output labels. To express such variations of confusion matrices, we design an algebra that models confusion matrices as probability distributions. Based on this algebra, we develop Neo, a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications. Finally, we demonstrate Neo's utility with three model evaluation scenarios that help people better understand model performance and reveal hidden confusions.


"Mouth Haptics in VR using a Headset Ultrasound Phased Array" -- Best Paper Award


Vivian Shen, Craig Shultz, and Chris Harrison


Today’s consumer virtual reality (VR) systems offer limited haptic feedback via vibration motors in handheld controllers. Rendering haptics to other parts of the body is an open challenge, especially in a practical and consumer-friendly manner. The mouth is of particular interest, as it is a close second in tactile sensitivity to the fingertips, offering a unique opportunity to add fine-grained haptic effects. In this research, we developed a thin, compact, beamforming array of ultrasonic transducers, which can render haptic effects onto the mouth. Importantly, all components are integrated into the headset, meaning the user does not need to wear an additional accessory, or place any external infrastructure in their room. We explored several effects, including point impulses, swipes, and persistent vibrations. Our haptic sensations can be felt on the lips, teeth and tongue, which can be incorporated into new and interesting VR experiences.


"'All that You Touch, You Change': Expanding the Canon of Speculative Design Towards Black Futuring" -- Honorable Mention Award


Christina Harrington, Shamika Klassen, Yolanda Rankin


Traditional approaches to technology design have historically ignored Blackness in both who engages and conceptualizes future technologies. Design contributions of groups marginalized along race and class are often \textit{othered}, and rarely considered the design standard. While frameworks have emerged to encourage attention to gender and social justice in design, little work has acknowledged evidence of the Black imaginary in this process. The current canon of design defines futuring and speculation as stemming from a narrow view of science fiction, one which does not include Black futurist perspectives. In this essay, we expand the canon of design by arguing that frameworks such as \textit{Afrofuturism}, \textit{Afrofuturist feminism}, and \textit{Black feminism} be considered instrumental in design’s imagining of our future technological landscape. We contribute to the larger conversation of who gets to future in design, suggesting a dialogic relationship between those who conceptualize design and those who consider design’s societal impact.


"Anticipate and Adjust: Cultivating Access in Human-Centered Methods" -- Honorable Mention Award


Kelly Mack, Emma McDonnell, Venkatesh Potluri, Maggie Xu, Jailyn Zabala, Jennifer Mankoff, Jeffrey Bigham, Cynthia L. Bennett


Methods are fundamental to doing research and can directly impact who is included in scientific advances. Given accessibility research’s increasing popularity and pervasive barriers to conducting and participating in research experienced by people with disabilities, it is critical to ask how methods are made accessible. Yet papers rarely describe their methods in detail. This paper reports on 17 interviews with accessibility experts about how they include both facilitators and participants with disabilities in popular user research methods. Our findings offer strategies for anticipating access needs while remaining flexible and responsive to unexpected access barriers. We emphasize the importance of considering accessibility at all stages of the research process, and contextualize access work in recent disability and accessibility literature. We explore how technology or processes could reflect a norm of accessibility. Finally, we discuss how various needs intersect and conflict and offer a
practical structure for planning accessible research.


"Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support" -- Honorable Mention Award


Anna Kawakami, Venkat Sivaraman, Hao-Fei Cheng, Logan Stapleton, Yang Cheng, Diana Qing, Adam Perer, Steven Wu, Haiyi Zhu, and Kenneth Holstein


AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers’ experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers’ reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS’s capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.


"ReCompFig: Designing Dynamically Reconfigurable Kinematic Devices Using Compliant Mechanisms and Tensioning Cables"-- Honorable Mention Award


Humphrey Yang, Tate Johnson, Ke Zhong, Dinesh K Patel, Gina Olson, Carmel Majidi, Mohammad Islam, Lining Yao


From creating input devices to rendering tangible information, the field of HCI is interested in using kinematic mechanisms to create human-computer interfaces. Yet, due to fabrication and design challenges, it is often difficult to create kinematic devices that are compact and have multiple reconfigurable motional degrees of freedom (DOFs) depending on the interaction scenarios. In this work, we combine compliant mechanisms (CMs) with tensioning cables to create dynamically reconfigurable kinematic mechanisms. The devices’ kinematics (DOFs) is enabled and determined by the layout of bendable rods. The additional cables function as on-demand motion constraints that can dynamically lock or unlock the mechanism’s DOFs as they are tightened or loosened. We provide algorithms and a design tool prototype to help users design such kinematic devices. We also demonstrate various HCI use cases including a kinematic haptic display, a haptic proxy, and a multimodal input device.


"TriboTouch: Micro-Patterned Surfaces for Low Latency Touchscreens" -- Honorable Mention Award


Craig Shultz, Daehwa Kim, Karan Ahuja, Chris Harrison


Touchscreen tracking latency, often 80ms or more, creates a rubber-banding effect in everyday direct manipulation tasks such as dragging, scrolling, and drawing. This has been shown to decrease system preference, user performance, and overall realism of these interfaces. In this research, we demonstrate how the addition of a thin, 2D micro-patterned surface with 5 micron spaced features can be used to reduce motor-visual touchscreen latency. When a finger, stylus, or tangible is translated across this textured surface frictional forces induce acoustic vibrations which naturally encode sliding velocity. This acoustic signal is sampled at 192kHz using a conventional audio interface pipeline with an average latency of 28ms. When fused with conventional low-speed, but high-spatial-accuracy 2D touch position data, our machine learning model can make accurate predictions of real time touch location.


"Understanding Challenges for Developers to Create Accurate Privacy Nutrition Labels" - Honorable Mention Award


Tianshi Li, Kayla Reiman, Yuvraj Agarwal, Lorrie Cranor, Jason I. Hong


Apple announced the introduction of app privacy details to their App Store in December 2020, marking the first ever real-world, large-scale deployment of the privacy nutrition label concept, which had been introduced by researchers over a decade earlier. The Apple labels are created by app developers, who self-report their app's data practices. In this paper, we present the first study examining the usability and understandability of Apple's privacy nutrition label creation process from the developer's perspective. By observing and interviewing 12 iOS app developers about how they created the privacy label for a real-world app that they developed, we identified common challenges for correctly and efficiently creating privacy labels. We discuss design implications both for improving Apple's privacy label design and for future deployment of other standardized privacy notices.


“'A Second Voice': Investigating Opportunities and Challenges for Interactive Voice Assistants to Support Home Health Aides"


Vince Bartle, Janice Lyu, Freesoul El Shabazz-Thompson, Yunmin Oh, Angela Anqi Chen, Yu-Jan Chang, Kenneth Holstein, Nicola Dell


Home health aides are a vulnerable group of frontline caregivers who provide personal and medically-oriented care in patients' homes. Their work is difficult and unpredictable, involving a mix of physical and emotional labor as they adapt to patients’ changing needs. Our paper presents an exploratory, qualitative study with 32 participants, that investigates design opportunities for Interactive Voice Assistants (IVAs) to support aides’ essential care work. We explore challenges and opportunities for IVAs to (1) fill gaps in aides' access to information and care coordination, (2) assist with decision making and task completion, (3) advocate on behalf of aides, and (4) provide emotional support. We then discuss key implications of our work, including how materiality may impact perceived ownership and usage of IVAs, the need to carefully consider tensions around surveillance, accountability, data collection, and reporting, and the challenges of centering aides as essential workers in complex home health care contexts.

"'It Feels Like Taking a Gamble': Exploring Perceptions, Practices, and Challenges of Using Makeup and Cosmetics for People with Visual Impairments"


Franklin Mingzhe Li, Franchesca Spektor, Meng Xia, Mina Huh, Peter Cederberg, Yuqi Gong, Kristen Shinohara, Patrick Carrington


Makeup and cosmetics offer the potential for self-expression and the reshaping of social roles for visually impaired people. However, there exist barriers to conducting a beauty regime because of the reliance on visual information and color variances in makeup. We present a content analysis of 145 YouTube videos to demonstrate visually impaired individuals' unique practices before, during, and after doing makeup. Based on the makeup practices, we then conducted semi-structured interviews with 12 visually impaired people to discuss their perceptions of and challenges with the makeup process in more depth. Overall, through our findings and discussion, we present novel perceptions of makeup from visually impaired individuals (e.g., broader representations of blindness and beauty). The existing challenges provide opportunities for future research to address learning barriers, insufficient feedback, and physical and environmental barriers, making the experience of doing makeup more accessible to people with visual impairments.


"Alert Now or Never: Understanding and Predicting Notification Preferences of Smartphone Users" (TOCHI Paper)


Tianshi Li, Julia Haines, Miguel Flores Ruiz de Eguino, Jason I. Hong, Jeffrey Nichols


Notifications are an indispensable feature of mobile devices, but their delivery can interrupt and distract users. Prior work has examined interventions, such as deferring notification delivery to opportune moments, but has not systematically studied how users might prefer an intelligent system to manage their notifications. Hence, we directly probed Android smartphone users’ notification preferences via a one-week experience-sampling study (N = 35). We found that users prefer mitigating undesired interruptions by suppressing alerts over deferring them and referred to notification content factors more frequently than contextual factors for explaining their preferences. Then we demonstrated the challenges and potentials of leveraging user actions to help predict notification preferences. Specifically, we showed that a model personalized using user actions achieved a performance gain of 39% than a generic model. This improvement is similar to the 42% performance gain using labels solicited from the user while using observable user actions causes no extra disruption.


"ControllerPose: Inside-Out Body Capture with VR Controller Cameras"


Karan Ahuja, Vivian Shen, Cathy Mengying Fang, Nathan Riopelle, Andy Kong, and Chris Harrison


We present a new and practical method for capturing user body pose in virtual reality experiences: integrating cameras into handheld controllers, where batteries, computation and wireless communication already exist. By virtue of the hands operating in front of the user during many VR interactions, our controller-borne cameras can capture a superior view of the body for digitization. Our pipeline composites multiple camera views together, performs 3D body pose estimation, uses this data to control a rigged human model with inverse kinematics, and exposes the resulting user avatar to end user applications. We developed a series of demo applications illustrating the potential of our approach and more leg-centric interactions, such as balancing games and kicking soccer balls. We describe our proof-of-concept hardware and software, as well as results from our user study, which point to imminent feasibility.


"Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making"


Michael Xieyang Liu, Aniket Kittur, Brad A. Myers


Developers perform online sensemaking on a daily basis, such as researching and choosing libraries and APIs. Prior research has introduced tools that help developers capture information from various sources and organize it into structures useful for subsequent decision-making. However, it remains a laborious process for developers to manually identify and clip content, maintaining its provenance and synthesizing it with other content. In this work, we introduce a new system called Crystalline that attempts to automatically collect and organize information into tabular structures as the user searches and browses the web. It leverages natural language processing to automatically group similar criteria together to reduce clutter as well as passive behavioral signals such as mouse movement and dwell time to infer what information to collect and how to visualize and prioritize it. Our user study suggests that developers are able to create comparison tables about 20% faster with a 60% reduction in operational cost without sacrificing the quality of the tables.


"ElectriPop: Low-Cost Shape-Changing Displays with Electrostatically Inflated Mylar Sheets"


Cathy Mengying Fang, Jianzhe Gu, Lining Yao, and Chris Harrison


We describe how sheets of metalized mylar can be cut and then “inflated” into complex 3D forms with electrostatic charge for use in digitally-controlled, shape-changing displays. This is achieved by placing and nesting various cuts, slits and holes such that mylar elements repel from one another to reach an equilibrium state. Importantly, our technique is compatible with industrial and hobbyist cutting processes, from die and laser cutting to handheld exacto-knives and scissors. Given that mylar film costs <$1 per m^2, we can create self-actuating 3D objects for just a few cents, opening new uses in low-cost consumer goods. We describe a design vocabulary, interactive simulation tool, fabrication guide, and proof-of-concept electrostatic actuation hardware. We detail our technique's performance metrics along with qualitative feedback from a design study. We present numerous examples generated using our pipeline to illustrate the rich creative potential of our method.


"Enabling Hand Gesture Customization on Wrist-Worn Devices"


Xuhai Xu, Jun Gong, Carolina Brum, Lilian Liang, Bongsoo Suh, Shivam Gupta, Yash Agarwal, Laurence Lindsey, Runchang Kang, Behrooz Shahsavari, Tu Nguyen, Heriberto Nieto, Scott Hudson, Charlie Maalouf, Jax Mousavi, Gierad Laput


We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95.7% and a false-positive rate of 0.6 per hour when tested on everyday non-gesture data. Next, we design a few-shot learning framework which derives a lightweight model from our pre-trained model, enabling knowledge transfer without performance degradation. We validate our approach through a user study (N=20) examining on-device customization from 12 new gestures, resulting in an average accuracy of 55.3%, 83.1%, and 87.2% on using one, three, or five shots when adding a new gesture, while maintaining the same recognition accuracy and false-positive rate from the pre-existing gesture set. We further evaluate the usability of our real-time implementation with a user experience study (N=20). Our results highlight the effectiveness, learnability, and usability of our customization framework. Our approach paves the way for a future where users are no longer bound to pre-existing gestures, freeing them to creatively introduce new gestures tailored to their preferences and abilities.


"Examining Identity as a Variable of Health Technology Research for Older Adults: A Systematic Review"


Christina Harrington, Aqueasha Martin-Hammond, Kirsten Bray


Innovations in HCI research of health-related pervasive and ubiquitous technologies can potentially improve older adults’ access to healthcare resources and support long-term independence in the home. Despite efforts to include their voices in technology research and design, many older adults have yet to actualize these health benefits, with barriers of access and proficiency actually widening the gap of health inequities. We reviewed 174 HCI publications through a systematic review to examine who is engaged in the design of health technologies for older adults, methods used to engage them, and how different types of participation might impact design directions. Findings highlight that thus far, many identity dimensions have not been explored in HCI aging research. We identify research gaps and implications to promote expanding research engagement with these dimensions as a way to support the design of health technologies that see better adoption among marginalized populations.


"Exploring the Needs of Users for Supporting Privacy-protective Behavior in Smart Homes"


Haojian Jin, Boyuan Guo, Rituparna Roychoudhury, Yaxing Yao, Swarun Kumar, Yuvraj Agarwal, Jason Hong


In this paper, we studied people’s smart home privacy-protective behaviors (SH-PPBs), to gain a better understanding of their privacy management do’s and don’ts in this context. We first surveyed 159 participants and elicited 33 unique SH-PPB practices, revealing that users heavily rely on ad hoc approaches at the physical layer (e.g., physical blocking, manual powering off). We also characterized the types of privacy concerns users wanted to address through SH-PPBs, the reasons preventing users from doing SH-PPBs, and privacy features they wished they had to support SH-PPBs. We then storyboarded 11 privacy protection concepts to explore opportunities to better support users’ needs, and asked another 227 participants to criticize and rank these design concepts. Among the 11 concepts, Privacy Diagnostics, which is similar to security diagnostics in anti-virus software, was far preferred over the rest. We also witnessed rich evidence of four important factors in designing SH-PPB tools, as users prefer (1) simple, (2) proactive, (3) preventative solutions that can (4) offer more control.


"From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks"}">From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks"


Hyeonsu Kang, Rafal Kocielnik, Andrew Head, Jiangjiang Yang, Matt Latzke, Aniket Kittur, Daniel S Weld, Doug Downey, Jonathan Bragg"}">Hyeonsu Kang, Rafal Kocielnik, Andrew Head, Jiangjiang Yang, Matt Latzke, Aniket Kittur, Daniel S Weld, Doug Downey, Jonathan Bragg


The ever-increasing pace of scientific publication necessitates methods for quickly identifying relevant papers. While neural recommenders trained on user interests can help, they still result in long, monotonous lists of suggested papers. To improve the discovery experience we introduce multiple new methods for augmenting recommendations with textual relevance messages that highlight knowledge-graph connections between recommended papers and a user's publication and interaction history. We explore associations mediated by author entities and those using citations alone. In a large-scale, real-world study, we show how our approach significantly increases engagement---and future engagement when mediated by authors---without introducing bias towards highly-cited authors. To expand message coverage for users with less publication or interaction history, we develop a novel method that highlights connections with proxy authors of interest to users and evaluate it in a controlled lab study. Finally, we synthesize design implications for future graph-based messages.


"How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions"


Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, Haiyi Zhu


Machine learning tools have been deployed in various contexts to support human decision-making, in the hope that human-algorithm collaboration can improve decision quality. However, the question of whether such collaborations reduce or exacerbate biases in decision-making remains underexplored. In this work, we conducted a mixed-methods study, analyzing child welfare call screen workers' decision-making over a span of four years, and interviewing them on how they incorporate algorithmic predictions into their decision-making process. Our data analysis shows that, compared to the algorithm alone, call screen workers reduced the disparity in screen-in rate between Black and white children from 20% to 9%. Our qualitative data show that workers achieved this by making holistic risk assessments and complementing the algorithm's limitations. These results shed light on potential mechanisms for improving human-algorithm collaboration in high-risk decision-making contexts.

"How Experienced Designers of Enterprise Applications Engage AI as a Design Material"


Nur Yildirim, Alex Kass, Teresa Tung, Connor Upton, Donnacha Costello, Robert Giusti, Sinem Lacin, Sara Lovic, James O’Neill, Rudi O’Reilly Meehan, Eoin Ó Loideáin, Azzurra Pini, Medb Corcoran, Jeremiah Hayes, Diarmuid Cahalane, Gaurav Shivhare, Luigi Castoro, Giovanni Caruso, Changhoon Oh, James McCann, Jodi Forlizzi, John Zimmerman



HCI research has explored AI as a design material, suggesting that designers can envision AI's design opportunities to improve UX. Recent research claimed that enterprise applications offer an opportunity for AI innovation at the user experience level. We conducted design workshops to explore the practices of experienced designers who work on cross-functional AI teams in the enterprise. We discussed how designers successfully work with and struggle with AI. Our findings revealed that designers can innovate at the system and service levels. We also discovered that making a case for an AI feature's return on investment is a barrier for designers when they propose AI concepts and ideas. Our discussions produced novel insights on designers' role on AI teams, and the boundary objects they used for collaborating with data scientists. We discuss the implications of these findings as opportunities for future research aiming to empower designers in working with data and AI.


"Interactive Fiction Provotypes for Coping with Interpersonal Racism"


Alexandra To, Hillary Carey, Riya Shrivastava, Jessica Hammer, Geoff Kaufman


Reducing uncertainty around the nature of racist interactions is one of the key motivations driving individual behaviors for coping with those incidents. However, there are few appropriate technologies to support BIPOC (Black, Indigenous, People of Color) in engaging in social uncertainty reduction around this vulnerable, sensitive topic. This paper reports on an exploratory design study investigating how social technology might facilitate uncertainty reduction through three ``provotypes'' - provocative prototypes of user-generated speculative design concepts. U.S.-based participants engaged with the provotypes through an interactive fiction to explore their usefulness in the context of a racist microaggression. Results showed that engaging the provotypes through interactive fiction facilitated complex and productive interactions and critiques. This work contributes a novel method for conducting exploratory design, remote user studies using interactive fiction as well as priorities, tensions, and further information what role, if any, technology might play in managing racist interactions.


"Lean Privacy Review: Collecting Users’ Privacy Concerns of Data Practices at a Low Cost" (TOCHI Paper)


Haojian Jin, Hong Shen, Mayank Jain, Swarun Kumar, Jason Hong


Today, industry practitioners (e.g., data scientists, developers, product managers) rely on formal privacy reviews (a combination of user interviews, privacy risk assessments, etc.) in identifying potential customer acceptance issues with their organization’s data practices. However, this process is slow and expensive, and practitioners often have to make ad-hoc privacy-related decisions with little actual feedback from users. We introduce Lean Privacy Review (LPR), a fast, cheap, and easy-to-access method to help practitioners collect direct feedback from users through the proxy of crowd workers in the early stages of design. LPR takes a proposed data practice, quickly breaks it down into smaller parts, generates a set of questionnaire surveys, solicits users’ opinions, and summarizes those opinions in a compact form for practitioners to use. By doing so, LPR can help uncover the range and magnitude of different privacy concerns actual people have at a small fraction of the cost and wait-time for a formal review. We evaluated LPR using 12 real-world data practices with 240 crowd users and 24 data practitioners. Our results show that (1) the discovery of privacy concerns saturates as the number of evaluators exceeds 14 participants, which takes around 5.5 hours to complete (i.e., latency) and costs 3.7 hours of total crowd work (≈ $80 in our experiments); and (2) LPR finds 89% of privacy concerns identified by data practitioners as well as 139% additional privacy concerns that practitioners are not aware of, at a 6% estimated false alarm rate.


"Letters from the Future: Exploring Ethical Dilemmas in the Design of Social Agents"


Michal Luria, Stuart Candy


We present a case-study of using Ethnographic Experiential Futures (EXF) to surface underlying divergences, tensions and dilemmas implicit in views of the futures of "social agents" among professional researchers familiar with the state of the art. Based on expert interviews, we designed three "letters from the future," research probes that were mailed to 15 participants working in the field, to encounter and respond to. We lay out the elements and design choices that shaped these probes, present our remote and asynchronous study design, and discuss lessons learned about the use of EXF. We find that this form of hybrid design/futures intervention has the potential to provide professional communities with opportunities to grapple with potential ethical dilemmas early on. However, the knowledge and tools for doing so are still in the making. Our contribution is a step towards advancing the potential benefits of experiential futures for technology designers and researchers.


"Maptimizer: Using Optimization to Tailor Tactile Maps to Users Needs"


Megan Hofmann, Kelly Mack, Jessica Birchfield, Jerry Cao, Autumn Hughes, Shriya Kurpad, Kathryn Lum, Emily Warnock, Anat Caspi, Scott Hudson, Jennifer Mankoff


Tactile maps can help people who are blind or have low-vision navigate and familiarize themselves with unfamiliar locations. Ideally, tactile maps can be customized to an individual's unique needs and abilities because of their limited space for representation. We present Maptimizer, a tool that generates tactile maps based on users' preferences and requirements. Maptimizer uses a two stage optimization process to pair representations with geographic information and tune those representations to present that information more clearly. In a small user study, Maptimizer helped participants more successfully and efficiently identify locations of interest in unknown areas. These results demonstrate the utility of optimization techniques and generative design in complex accessibility domains.


"PneuMesh: Pneumatic-driven Truss-based Shape Changing System"


Jianzhe Gu, Yuyu Lin, Qiang Cui, Xiaoqian Li, Jiaji Li, Lingyun Sun, Fangtian Ying, Guanyun Wang, Lining Yao


From transoceanic bridges to large-scale installations, truss structures have been known for their structural stability and shape complexity. In addition to the advantages of static trusses, truss structures have a large degree of freedom to change shape when equipped with rotatable joints and retractable beams. However, it is difcult to design a complex motion and build a control system for large numbers of trusses. In this paper, we present PneuMesh, a novel truss-based shape-changing system that is easy to design and build but still able to achieve a range of tasks. PneuMesh accomplishes this by introducing an air channel connection strategy and reconfgurable constraint design that drastically decreases the number of control units without losing the complexity of shape-changing. We develop a design tool with real-time simulation to assist users in designing the shape and motion of truss-based shape- changing robots and devices. A design session with seven participants demonstrates that PneuMesh empowers users to design and build truss structures with a wide range of shapes and various functional motions.


"Prediction for Retrospection: Integrating Algorithmic Stress Prediction into Personal Informatics Systems for College Students' Mental Health"


Taewan Kim, Haesoo Kim, Ha Yeon Lee, Hwarang Goh, Shakhboz Abdigapporov, Mingon Jeong, Hyunsung Cho, Kyungsik Han, Youngtae Noh, Sung-Ju Lee, Hwajung Hong


Reflecting on stress-related data is critical in addressing one’s mental health. Personal Informatics (PI) systems augmented by algorithms and sensors have become popular ways to help users collect and reflect on data about stress. While prediction algorithms in the PI systems are mainly for diagnostic purposes, few studies examine how the explainability of algorithmic prediction can support user-driven self-insight. To this end, we developed MindScope, an algorithm-assisted stress management system that determines user stress levels and explains how the stress level was computed based on the user's everyday activities captured by a smartphone. In a 25-day field study conducted with 36 college students, the prediction and explanation supported self-reflection, a process to re-establish preconceptions about stress by identifying stress patterns and recalling past stress levels and patterns that led to coping planning. We discuss the implications of exploiting prediction algorithms that facilitate user-driven retrospection in PI systems.


"Radical Futures: Supporting Community-Led Design Engagements through an Afrofuturist Speculative Design Toolkit"


Kirsten Bray, Christina Harrington, Andrea Grimes Parker, N'Deye Dhiakte, Jennifer Roberts


When considering the democratic intentions of co-design, designers and design researchers must evaluate the impact of power imbalances embedded in common design and research dynamics. This holds particularly true in work with and for marginalized communities, who are frequently excluded in design processes. To address this issue, we examine how existing design tools and methods are used to support communities in processes of community building or reimagining, considering the influence of race and identity. This paper describes our findings from 27 interviews with community design practitioners conducted to evaluate the Building Utopia toolkit, which employs an Afrofuturist lens for speculative design processes. Our research findings support the importance of design tools that prompt conversations on race in design, and tensions between the desire for imaginative design practice and the immediacy of social issues, particularly when designing with Black and brown communities.


"Round Numbers Can Sharpen Cognition"


Huy A. Nguyen, Jake M. Hofman, and Daniel G. Goldstein


Scientists and journalists strive to report numbers with high precision to keep readers well-informed. Our work investigates whether this practice can backfire due to the cognitive costs of processing multi-digit precise numbers. In a pre-registered randomized experiment, we presented readers with several news stories containing numbers in either precise or round versions. We then measured their ability to approximately recall these numbers and make estimates based on what they read. Our results revealed a counter-intuitive effect where reading round numbers helped people better approximate the precise values, while seeing precise numbers made them worse. We also conducted two surveys to elicit individual preferences for the ideal degree of rounding for numbers spanning seven orders of magnitude in various contexts. From the surveys, we found that people tended to prefer more precision when the rounding options contained only digits (e.g., "2,500,000") than when they contained modifier terms (e.g., "2.5 million"). We conclude with a discussion of how these findings can be leveraged to enhance numeracy in digital content consumption.


"Scaling Creative Inspiration with Fine-Grained Functional Facets of Product Ideas"


Tom Hope, Ronen Tamari, Hyeonsu Kang, Daniel Hershcovich, Joel Chan, Aniket Kittur, Dafna Shahaf


Large repositories of products, patents and scientific papers offer an opportunity for building systems that scour millions of ideas and help users discover inspirations. However, idea descriptions are typically in the form of unstructured text, lacking key structure that is required for supporting creative innovation interactions. Prior work has explored idea representations that were either limited in expressivity, required significant manual effort from users, or dependent on curated knowledge bases with poor coverage. We explore a novel representation that automatically breaks up products into fine-grained functional aspects capturing the purposes and mechanisms of ideas, and use it to support important creative innovation interactions: functional search for ideas, and exploration of the design space around a focal problem by viewing related problem perspectives pooled from across many products. In user studies, our approach boosts the quality of creative search and inspirations, substantially outperforming strong baselines by 50-60%.


"Symphony: Composing Interactive Interfaces for Machine Learning"


Alex Bäuerle, Ángel Alexander Cabrera, Fred Hohman, Megan Maher, David Koski, Xavier Suau, Titus Barik, Dominik Moritz


Interfaces for machine learning (ML), information and visualizations about models or data, can help practitioners build robust and responsible ML systems. Despite their benefits, recent studies of ML teams and our interviews with practitioners (n=9) showed that ML interfaces have limited adoption in practice. While existing ML interfaces are effective for specific tasks, they are not designed to be reused, explored, and shared by multiple stakeholders in cross-functional teams. To enable analysis and communication between different ML practitioners, we designed and implemented Symphony, a framework for composing interactive ML interfaces with task-specific, data-driven components that can be used across platforms such as computational notebooks and web dashboards. We developed Symphony through participatory design sessions with 10 teams (n=31), and discuss our findings from deploying Symphony to 3 production ML projects at Apple. Symphony helped ML practitioners discover previously unknown issues like data duplicates and blind spots in models while enabling them to share insights with other stakeholders.


"Tech Help Desk: Support for Local Entrepreneurs Addressing the Long Tail of Computing Challenges"


Yasmine Kotturi, Herman T Johnson Jr, Michael Skirpan, Sarah E Fox, Jeffrey Bigham, Amy Pavel


Even entrepreneurs whose businesses are not technological (e.g., handmade goods) need to be able to use a wide range of computing technologies in order to achieve their business goals. In this paper, we follow a participatory action research approach and collaborate with various stakeholders at an entrepreneurial co-working space to design “Tech Help Desk”, an on-going technical service for entrepreneurs. Our model for technical assistance is strategic, in how it is designed to fit the context of local entrepreneurs, and responsive, in how it prioritizes emergent needs. From our engagements with 19 entrepreneurs and support personnel, we reflect on the challenges with existing technology support for non-technological entrepreneurs. Our work highlights the importance of ensuring technological support services can adapt based on entrepreneurs’ ever-evolving priorities, preferences and constraints. Furthermore, we find technological support services should maintain broad technical support for entrepreneurs’ long tail of computing challenges.


"Templates and Trust-o-meters: Towards a widely deployable indicator of trust in Wikipedia"


Andrew Kuznetsov, Margeigh Novotny, Jessica Klein, Diego Saez-Trumper, Aniket Kittur


The success of Wikipedia and other user-generated content communities has been driven by the openness of recruiting volunteers globally, but this openness has also led to a persistent lack of trust in its content. Despite several attempts at developing trust indicators to help readers more quickly and accurately assess the quality of content, challenges remain for practical deployment to general consumers. In this work we identify and address three key challenges: empirically determining which metrics from prior and existing community approaches most impact reader trust; 2) validating indicator placements and designs that are both compact yet noticed by readers; and 3) demonstrating that such indicators can not only lower trust but also increase perceived trust in the system when appropriate. By addressing these, we aim to provide a foundation for future tools that can practically increase trust in user generated content and the sociotechnical systems that generate and maintain them.


"To Self-Persuade or be Persuaded: Examining Interventions for Users' Privacy Setting Selection"


Isadora Krsek, Kimi Wenzel, Sauvik Das, Jason Hong, Laura Dabbish


User adoption of security and privacy (S&P) best practices remains low, despite sustained efforts by researchers and practitioners. Social influence is a proven method for guiding user S&P behavior, though most work has focused on studying peer influence, which is only possible with a known social graph. In a study of 104 Facebook users, we instead demonstrate that crowdsourced S&P suggestions are significantly influential. We also tested how reflective writing affected participants' S&P decisions, with and without suggestions. With reflective writing, participants were less likely to accept suggestions --- both social and Facebook default suggestions. Of particular note, when reflective writing participants were shown the Facebook default suggestion, they not only rejected it but also (unknowingly) configured their settings in accordance with expert recommendations. Our work suggests that both non-personal social influence and reflective writing can positively influence users' S&P decisions, but have negative interactions.


"Toward User-Driven Algorithm Auditing: Investigating users' strategies for uncovering harmful algorithmic behavior"


Alicia DeVos, Aditi Dhabalia, Hong Shen, Kenneth Holstein, Motahhare Eslami


Recent work in HCI suggests that users can be powerful in surfacing harmful algorithmic behaviors that formal auditing approaches fail to detect. However, it is not well understood how users are often able to be so effective, nor how we might support more effective user-driven auditing. To investigate, we conducted a series of think-aloud interviews, diary studies, and workshops, exploring how users find and make sense of harmful behaviors in algorithmic systems, both individually and collectively. Based on our findings, we present a process model capturing the dynamics of and influences on users' search and sensemaking behaviors. We find that 1) users' search strategies and interpretations are heavily guided by their personal experiences with and exposures to societal bias; and 2) collective sensemaking amongst multiple users is invaluable in user-driven algorithm audits. We offer directions for the design of future methods and tools that can better support user-driven auditing.


"Towards Understanding Diminished Reality"


Yi Fei Cheng, Hang Yin, Yukang Yan, Jan Gugenheimer, David Lindlbauer


Diminished reality (DR) refers to the concept of removing content from a user's visual environment. While its implementation is becoming feasible, it is still unclear how users perceive and interact in DR-enabled environments and what applications it benefits. To address this challenge, we first conduct a formative study to compare user perceptions of DR and mediated reality effects (e.g., changing the color or size of target elements) in four example scenarios. Participants preferred removing objects through opacity reduction (i.e., the standard DR implementation) and appreciated mechanisms for maintaining a contextual understanding of diminished items (e.g., outlining). In a second study, we explore the user experience of performing tasks within DR-enabled environments. Participants selected which objects to diminish and the magnitude of the effects when performing two separate tasks (video viewing, assembly). Participants were comfortable with decreased contextual understanding, particularly for less mobile tasks. Based on the results, we define guidelines for creating general DR-enabled environments.


"Travelogue: Representing Indoor Trajectories as Informative Art" (Late Breaking Work)


Yunzhi Li, Tingyu Cheng, Ashutosh Dhekne


In this work, we explore if informative art can represent a user’s indoor trajectory and promote user’s self-reflection, creating a new type of interactive space. Under the assumption that the simplicity of a digital picture frame can be an appealing way to represent indoor activities and further create a dyadic relationship between users and the space they occupy, we present Travelogue, a picture-frame like self-contained system which can sense human movement using wireless signal reflections in a device free manner. Breaking away from traditional dashboard-based visualization techniques, Travelogue only renders the high-level extent and location of users’ activities in different informative arts. Our preliminary user study with 12 participants shows most users found Travelogue intuitive, unobtrusive, and aesthetically pleasing, as well as a desired tool for self-reflection on indoor activity.


"Understanding AR Activism: An Interview Study with Creators of Augmented Reality Experiences for Social Change"


Rafael Machado de Lima Silva, Erica Principe Cruz, Daniela Rosner, Dayton Kelly, Andrés Monroy-Hernández, Fannie Liu


The rise of consumer augmented reality (AR) technology has opened up new possibilities for interventions intended to disrupt and subvert cultural conventions. From defacing corporate logos to erecting geofenced digital monuments, more and more people are creating AR experiences for social causes. We sought to understand this new form of activism, including why people use AR for these purposes, opportunities and challenges in using it, and how well it can support activist goals. We conducted semi-structured interviews with twenty people involved in projects that used AR for a social cause across six different countries. We found that AR can overcome physical world limitations of activism to convey immersive, multilayered narratives that aim to reveal invisible histories and perspectives. At the same time, people experienced challenges in creating, maintaining, and distributing their AR experiences to audiences. We discuss open questions and opportunities for creating AR tools and experiences for social change.


"Understanding How Programmers Can Use Annotations on Documentation"


Amber Horvath, Michael Xieyang Liu, River Hendriksen, Connor Shannon, Emma Paterson, Kazi Jawad, Andrew Macvean, Brad A. Myers


Modern software development requires developers to find and effectively utilize new APIs and their documentation, but documentation has many well-known issues. Despite this, developers eventually overcome these issues but have no way of sharing what they learned. We investigate sharing this documentation-specific information through annotations, which have advantages over developer forums as the information is contextualized, not disruptive, and is short, thus easy to author. Developers can also author annotations to support their own comprehension. In order to support the documentation usage behaviors we found, we built the Adamite annotation tool, which provides features such as multiple anchors, annotation types, and pinning. In our user study, we found that developers are able to create annotations that are useful to themselves and are able to utilize annotations created by other developers when learning a new API, with readers of the annotations completing 67% more of the task, on average, than the baseline.


"Understanding iOS Privacy Nutrition Labels: An Exploratory Large-Scale Analysis of App Store Data" (Late Breaking Work)


Yucheng Li, Deyuan Chen, Tianshi Li, Yuvraj Agarwal, Lorrie Faith Cranor, Jason I. Hong


Since December 2020, the Apple App Store has required all developers to create a privacy label when submitting new apps or app updates. However, there has not been a comprehensive study on how developers responded to this requirement. We present the first measurement study of Apple privacy nutrition labels to understand how apps on the U.S. App Store create and update privacy labels. We collected weekly snapshots of the privacy label and other metadata for all the 1.4 million apps on the U.S. App Store from April 2 to November 5, 2021. Our analysis showed that 51.6% of apps still do not have a privacy label as of November 5, 2021. Although 35.3% of old apps have created a privacy label, only 2.7% of old apps created a privacy label without app updates (i.e., voluntary adoption). Our findings suggest that inactive apps have little incentive to create privacy labels.


"VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users' Daily Lives"


Riku Arakawa, Hiromu Yakura, Sosuke Kobayashi


We demonstrate that recent natural language processing (NLP) techniques introduce a new paradigm of vocabulary learning that benefits from both micro and usage-based learning by generating and presenting the usages of foreign words based on the learner's context. Then, without allocating dedicated time for studying, the user can become familiarized with how the words are used by seeing the example usages during daily activities, such as Web browsing. To achieve this, we introduce VocabEncounter, a vocabulary-learning system that suitably encapsulates the given words into materials the user is reading in near real time by leveraging recent NLP techniques. After confirming the system's human-comparable quality of generating translated phrases by involving crowdworkers, we conducted a series of user studies, which demonstrated its effectiveness on learning vocabulary and its favorable experiences. Our work shows how NLP-based generation techniques can transform our daily activities into a field for vocabulary learning.


"Who owns the future of work?" (Alt.chi)


Alex Ahmed


This submission is intended to start a critical conversation about the "Future of Work." I expose the ideological commitments of major funders of research into this area, such as the NSF and private think tanks, because they shape narratives on work and worker organizing as well as the course of technology development more broadly. In the research itself, workers may be invited to participate in computing research, but they seldom determine the goals of projects nor do they own the results. Ultimately, while government agencies and nonprofit organizations claim to seek improved conditions for workers, the knowledge and technologies produced through this research serve primarily to undermine the power of working class people and their unions.