Research Areas
Collective Intelligence of Human-AI Teams
How can AI collaborate with human teams to improve performance on cognitive tasks? In pioneering work published at AAAI’23 we develop a Bayesian theory of mind agent that can form ad-hoc mental models about its teammates, based exclusively on observations drawn from human communication. In the paper we pioneer a generative model approach to collective intelligence, which allows us to achieve two goals simultaneously: First, we can build agents for human-AI teams that can generate interventions to improve team performance and second, we can test theories about human cognition and use it to measure collective intelligence ability in real-time. This work provides a novel framework which expands collective intelligence theory from a static construct to a dynamical one that can vary according to situational factors, for example, due to changes in arousal, anxiety, and motivation with dynamically changing task requirements, time pressure, and recognition. Ongoing work in lab investigating communication in human-AI teams and how AI can help advance scientific discovery by helping design optimized experiments. Overall, we explore how AI-based automation and human-AI teaming affecting work, and workers, more broadly. Work by the lab has contributed to the ongoing policy debate about how labor laws need to change to accommodate the increasing role of AI and advance the discussion on how AI can and should be regulated.
Lab Members: Christoph Riedl
Publications, Grants, and Ongoing Projects:
How can AI collaborate with human teams to improve performance on cognitive tasks? In pioneering work published at AAAI’23 we develop a Bayesian theory of mind agent that can form ad-hoc mental models about its teammates, based exclusively on observations drawn from human communication. In the paper we pioneer a generative model approach to collective intelligence, which allows us to achieve two goals simultaneously: First, we can build agents for human-AI teams that can generate interventions to improve team performance and second, we can test theories about human cognition and use it to measure collective intelligence ability in real-time. This work provides a novel framework which expands collective intelligence theory from a static construct to a dynamical one that can vary according to situational factors, for example, due to changes in arousal, anxiety, and motivation with dynamically changing task requirements, time pressure, and recognition. Ongoing work in lab investigating communication in human-AI teams and how AI can help advance scientific discovery by helping design optimized experiments. Overall, we explore how AI-based automation and human-AI teaming affecting work, and workers, more broadly. Work by the lab has contributed to the ongoing policy debate about how labor laws need to change to accommodate the increasing role of AI and advance the discussion on how AI can and should be regulated.
Lab Members: Christoph Riedl
Publications, Grants, and Ongoing Projects:
- Micro and Meso Signatures of Success in Human-Autonomy Teams, $1.5m research grant from Army Research Lab.
- Large Language Models for Automatic Milestone Detection in Group Discussions, under review.
- How humans learn from AI (with Eric Bogert), working paper.
- Navigating Bias: Human Editing Behavior in Collaboration with Generative AI (with Shun-Yang Lee), working paper.
- The potential welfare benefits of AI in a differentiated product market (with Imke Reimers & Joel Waldfogel), working paper.
- Westby, S., Radke, R.J., Welles, B.F., Riedl, C. (2023). Building Better Human-Agent Teams: Tradeoffs in Helpfulness and Humanness in Voice, working paper.
- Westby, S., & Riedl, C. (2023). Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind Approach. AAAI'23. arXiv:2208.11660, Github.
- Kiron, D., Altman, E.J., Riedl, C. (2023). Workforce Ecosystems and AI, Brookings Institute.
- Balietti, S., Klein, B., & Riedl, C. (2021). Optimal design of experiments to identify latent behavioral types. Experimental Economics, 24, 772–799. Preprint at arXiv, replication data.
Collective Intelligence of Teams
Our work in this research area focuses on the emergence of collective intelligence in teams. Society and scientific research rely increasingly on teams, and more and more problems—from addressing climate change to curing diseases and developing complex technologies—can only be solved by the work of teams. Thus, the ability of teams to function at a high level is critically important for many aspects of our well- being and our collective capacity to conduct research. Increasingly, those teams collaborate remotely, posing new questions. Work by the lab has shown that the collective intelligence of remote teams depends more on how work is done—i.e., team processes—and not so much on where work is done. Our current and past work in this area evaluates bursty nature of team communication, collective attention, team hierarchy, team gender composition, human-agent teaming, virtual teams, and collective intelligence more broadly. Most recently we have focused on studies of nonverbal and verbal synchronization using the free-energy principle, the effect of team structure and incentives on problem-solving performance, and progressive performance of online teams via virtual meeting room environments.
Lab Members: Zach Fulker, Julian Gullett, Christoph Riedl
Key Publications:
Our work in this research area focuses on the emergence of collective intelligence in teams. Society and scientific research rely increasingly on teams, and more and more problems—from addressing climate change to curing diseases and developing complex technologies—can only be solved by the work of teams. Thus, the ability of teams to function at a high level is critically important for many aspects of our well- being and our collective capacity to conduct research. Increasingly, those teams collaborate remotely, posing new questions. Work by the lab has shown that the collective intelligence of remote teams depends more on how work is done—i.e., team processes—and not so much on where work is done. Our current and past work in this area evaluates bursty nature of team communication, collective attention, team hierarchy, team gender composition, human-agent teaming, virtual teams, and collective intelligence more broadly. Most recently we have focused on studies of nonverbal and verbal synchronization using the free-energy principle, the effect of team structure and incentives on problem-solving performance, and progressive performance of online teams via virtual meeting room environments.
Lab Members: Zach Fulker, Julian Gullett, Christoph Riedl
Key Publications:
- Woolley, A., Chow, R., Mayo, A., Riedl, C.,Chang, J. (2023). Collective Attention and Collective Intelligence: The Role of Hierarchy and Team Gender Composition. Organization Science. 34(3), 1315–1331. article, PDF.
- Riedl, C., Kim, Y., Gupta, P., Malone, T., and Woolley, A. (2021). Quantifying Collective Intelligence in Human Groups. PNAS. 118 (21). article, PDF.
- Riedl, C., & Woolley, A. (2020). Successful remote teams communicate in bursts. Harvard Business Review, article.
- Balietti, S., Klein, B., & Riedl, C. (2020). Optimal design of experiments to identify latent behavioral types. Experimental Economics, in press. Preprint at arXiv, replication data.
- Riedl, C. & Woolley, A. (2017). Teams vs. Crowds: A Field Test of the Relative Contribution of Incentives, Member Ability, and Collaboration to Crowd-Based Problem Solving Performance. Academy of Management Discoveries, 3(4), 382-403, article, open access PDF, code.
Collective Intelligence of Crowds
We seek to better understand the behaviors of crowds, and other large, decentralized groups of individuals working together to solve a problem. A key theme of our work on collective intelligence in crowds is that the individuals in these networked environments show signs of rich sociality. Work by the lab has shown that individuals learn from each other, form rivalries among each other, show signs of sophisticated strategic interactions, and exhibit high levels of cooperation. Thus, our research helps to better elaborate the strength and weaknesses of crowdsourcing and online labor markets as approaches to problem solving. This knowledge is used to help organizations better understand and leverage the potential benefits of crowdsourcing.
Lab Members: Zach Fulker, Christoph Riedl
Key Publications:
We seek to better understand the behaviors of crowds, and other large, decentralized groups of individuals working together to solve a problem. A key theme of our work on collective intelligence in crowds is that the individuals in these networked environments show signs of rich sociality. Work by the lab has shown that individuals learn from each other, form rivalries among each other, show signs of sophisticated strategic interactions, and exhibit high levels of cooperation. Thus, our research helps to better elaborate the strength and weaknesses of crowdsourcing and online labor markets as approaches to problem solving. This knowledge is used to help organizations better understand and leverage the potential benefits of crowdsourcing.
Lab Members: Zach Fulker, Christoph Riedl
Key Publications:
- Riedl, C., Grad, T., & Lettl, C. (2024). “Competition and Collaboration in Crowdsourcing Communities: What happens when peers evaluate each other?” Organization Science, in press.
- Riedl, C., Hutter, K., Füller, J., Tellis, G. (2024). “Cash or Non-Cash? Unveiling Ideators' Incentive Preferences in Crowdsourcing Contests,” Journal of Management Information Systems, in press.
- Fulker, Z., Forber, P., Smead, R., Riedl, C. (2024). Spontaneous emergence of groups and signaling diversity in dynamic networks. Physical Review E, 109, 014309, article.
- Fulker, Z. & Riedl, C. (2024). Cooperation in the Gig Economy: Insights from Upwork Freelancers. Proceedings of the ACM on Human Computer Interaction, 8(CSCW1), 37. arXiv.
- Seidel, V. & Riedl, C. (2023). How creative versus technical constraints affect individual learning in an online innovation community. Working paper. arXiv.
- Fulker, Z., Forber, P., Smead, R., Riedl, C. (2021). Spite is Contagious in Dynamic Networks. Nature Communications, 12(260), replication data, article, open access PDF.
- Foley, M., Forber, P., Smead, R., Riedl, C. (2021). Avoiding the Bullies: The Resilience of Cooperation among Unequals. PLOS Computational Biology, 17(4) replication data, article.
- Foley, M., Forber, P., Smead, R., Riedl, C. (2018). Conflict and Convention in Dynamic Networks. Journal of the Royal Society Interface, 15(140), 20170835, article, open access PDF. Companion website with interactive visualization to provide explorable explanations and replication code.
- Riedl, C., Seidel, V. (2018). Learning from Mixed Signals in Online Innovation Communities. Organization Science, 29(6), 1010-1032, article, open access PDF.