Organizational Implications of Autonomy-Mediated Interaction
Principal Investigator: Jonathan Gratch, University of Southern California
Co-Investigators: Nate Fast, University of Southern California
Years of Award: 2018-2010
Managing Service Agency: Air Force Office of Scientific Research
Project Description:
Irresistible pressures are driving the adoption of autonomy. Autonomy lowers costs, increases the agility and tempo of decisions, and augments the capabilities of human actors. Delegating tasks to autonomous machines and using them to mediate work interactions can offer many benefits, but at what cost? In human organizations, delegating tasks to others increases moral distance from the consequences of one's actions, but it is unclear whether this and related effects will carry over when delegating to machines. Will operating through an autonomous robot further undermine trust, increase risk-taking, reduce vigilance to threats and increase dehumanization of others? Or might it soften or even reverse these effects? What are the implications for power dynamics between humans when inserting autonomous machine representatives into the organizational chain of command? What methods of accountability are most effective when some decision-makers are machines? Social science research is needed to examine the psychological, organizational and cultural impact of these technological advances. With this grant, we are particularly interested in autonomy-mediated interactions. While organizations have increasingly operated through tools of automation (e.g. email; videoconferencing), new technologies allow "operators" (people deploying agents) to use automated systems to interact with others ("interactants"). For example, mission planners in the military are increasingly capable of deploying unmanned combat robots, and clinicians are able to use automated "nurses" to manage patient intake and discharge. These technologies present a psychologically, socially, and organizationally unique situation: how do people respond to machines representing humans? This proposal brings together a multi-disciplinary team of experts to investigate how this increasing reliance on autonomy shapes individual and organizational decisions.
Select Publications:
de Melo, Celso M., Stacy Marsella, and Jonathan Gratch. 2019. "Human Cooperation When Acting Through Autonomous Machines". Proceedings Of The National Academy Of Sciences 116 (9): 3482-3487.
Lee, Minha, Gale Lucas, Johnathan Mell, Emanuel Johnson, and Jonathan Gratch. 2019. "What's On Your Virtual Mind Perception In Human-Agent Negotiations?". 19th International Conference on Intelligent Virtual Agents. Paris, France, 2019.