Thinking About Deciding
Behavioral Attenuation
Benjamin Enke et al.
NBER Working Paper, September 2024
Abstract:
We report a large-scale examination of behavioral attenuation: due to information-processing constraints, the elasticity of people’s decisions with respect to economic fundamentals is generally too small. We implement more than 30 experiments, 20 of which were crowd-sourced from leading experts. These experiments cover a broad range of economic decisions, from choice and valuation to belief formation, from strategic games to generic optimization problems, involving investment, savings, effort supply, product demand, taxes, environmental externalities, fairness, cooperation, beauty contests, information disclosure, search, policy evaluation, memory, forecasting and inference. In 93% of our experiments, the elasticity of decisions to fundamentals decreases in participants’ cognitive uncertainty, our measure of the severity of information-processing constraints. Moreover, in decision problems with objective solutions, we observe elasticities that are universally smaller than is optimal. Many widely-studied decision anomalies represent special cases of behavioral attenuation. We discuss both its limits and why it often gives rise to the classic phenomenon of diminishing sensitivity.
A multinational analysis of how emotions relate to economic decisions regarding time or risk
Samuel Pertl, Tara Srirangarajan & Oleg Urminsky
Nature Human Behaviour, forthcoming
Abstract:
Emotions have been theorized to be important drivers of economic choices, such as intertemporal or risky decisions. Our systematic review and meta-analysis of the previous literature (378 results and 50,972 participants) indicates that the empirical basis for these claims is mixed and the cross-cultural generalizability of these claims has yet to be systematically tested. We analysed a dataset with representative samples from 74 countries (n = 77,242), providing a multinational test of theoretical claims that individuals’ ongoing emotional states predict their economic preferences regarding time or risk. Overall, more positive self-reported emotions generally predicted a willingness to wait for delayed rewards or to take favourable risks, in line with some existing theories. Contrary to the assumption of a universal relationship between emotions and decision-making, we show that these relationships vary substantially and systematically across countries. Emotions were stronger predictors of economic decisions in more economically developed and individualistic countries.
Falling Vocal Intonation Signals Speaker Confidence and Conditionally Boosts Persuasion
Thomas Vaughan-Johnston et al.
Personality and Social Psychology Bulletin, forthcoming
Abstract:
People are often advised to project confidence with their bodies and voices to convince others. Prior research has focused on the high and low thinking processes through which vocal confidence signals (e.g., fast speed, falling intonation, low pitch) can influence attitude change. In contrast, this research examines how the vocal confidence of speakers operates under more moderate elaboration levels, revealing that falling intonation only benefits persuasion under certain circumstances. In three experiments, we show that falling (vs. rising) vocal intonation at the ends of sentences can signal speaker confidence. Under moderate elaboration conditions, falling (vs. rising) vocal intonation increased message processing, bolstering the benefit of strong over weak messages, increasing the proportion of message-relevant thoughts, and increasing thought-attitude correspondence. In sum, the present work examined an unstudied role of vocal confidence in guiding persuasion, revealing new processes by which vocal signals increase or fail to increase persuasion.
The Language of (Non)Replicable Social Science
Michal Herzenstein et al.
Psychological Science, September 2024, Pages 1048–1061
Abstract:
Using publicly available data from 299 preregistered replications from the social sciences, we found that the language used to describe a study can predict its replicability above and beyond a large set of controls related to the article characteristics, study design and results, author information, and replication effort. To understand why, we analyzed the textual differences between replicable and nonreplicable studies. Our findings suggest that the language in replicable studies is transparent and confident, written in a detailed and complex manner, and generally exhibits markers of truthful communication, possibly demonstrating the researchers’ confidence in the study. Nonreplicable studies, however, are vaguely written and have markers of persuasion techniques, such as the use of positivity and clout. Thus, our findings allude to the possibility that authors of nonreplicable studies are more likely to make an effort, through their writing, to persuade readers of their (possibly weaker) results.
Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
Chenglei Si, Diyi Yang & Tatsunori Hashimoto
Stanford Working Paper, September 2024
Abstract:
Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first head-to-head comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation. Finally, we acknowledge that human judgements of novelty can be difficult, even by experts, and propose an end-to-end study design which recruits researchers to execute these ideas into full projects, enabling us to study whether these novelty and feasibility judgements result in meaningful differences in research outcome.
An empirical investigation of the impact of ChatGPT on creativity
Byung Cheol Lee & Jaeyeon (Jae) Chung
Nature Human Behaviour, forthcoming
Abstract:
This paper investigates the potential of ChatGPT for helping humans tackle problems that require creativity. Across five experiments, we asked participants to use ChatGPT (GPT-3.5) to generate creative ideas for various everyday and innovation-related problems, including choosing a creative gift for a teenager, making a toy, repurposing unused items and designing an innovative dining table. We found that using ChatGPT increased the creativity of the generated ideas compared with not using any technology or using a conventional Web search (Google). This effect remained robust regardless of whether the problem required consideration of many (versus few) constraints and whether it was viewed as requiring empathetic concern. Furthermore, ChatGPT was most effective at generating incrementally (versus radically) new ideas. Process evidence suggests that the positive influence of ChatGPT can be attributed to its capability to combine remotely related concepts into a cohesive form, leading to a more articulate presentation of ideas.
The Effects of Real-Time Feedback on Effort and Performance: Evidence from a Natural Quasi-Experiment
Maximilian Margolin, Marko Reimer & Daniel Schaupp
Management Science, forthcoming
Abstract:
New digital technologies allow companies to provide employees with performance feedback in real time. We investigate the causal effects of such real-time feedback on individual effort and the quality of initial (prefeedback) and final (postfeedback) decisions by exploiting a natural quasi-experiment in the setting of professional soccer refereeing. Our results suggest that real-time feedback has differential effects for experienced compared with inexperienced agents. Specifically, we find some evidence that experienced agents decrease their effort under real-time feedback without harming the quality of their initial decisions. This results in better final decisions because real-time feedback allows agents to correct wrong initial decisions upon feedback. In contrast, inexperienced agents increase their effort, but make worse initial decisions. Upon feedback, they manage to compensate for their increased number of wrong initial decisions, but see no improvement in the quality of final decisions.
Genetically-diverse crowds are wiser
Meir Barneron et al.
Personality and Individual Differences, December 2024
Abstract:
A fundamental question in the social sciences is how collectives of individuals form intelligent judgments. This article tests the hypothesis that genetically-diverse groups make better collective judgments than genetically more homogenous groups. Two studies were conducted (a total of N = 602 participants) in which sets of twins (both monozygotic and dizygotic) were required to perform the task of making numerical judgments. The accuracy of the judgments made by pairs of participants -- who were either co-twins (i.e., genetically-related) or were not related -- was then compared. The results indicate that the judgments made by unrelated pairs were more accurate than those of the genetically-related twins. Critically, however, this superior performance was found only among monozygotic twins, evidencing the role of genetic relatedness in collective judgment. This research provides the first empirical demonstration of the benefit of genetic diversity for collective judgments, shedding light on the origins of the ‘wisdom of crowds’ phenomenon.
How Real Is Hypothetical?: A High-Stakes Test of the Allais Paradox
Uri Gneezy et al.
Harvard Working Paper, August 2024
Abstract:
Researchers in behavioral and experimental economics often argue that only incentive-compatible mechanisms can elicit effort and truthful responses from participants. Others argue that participants make less-biased decisions when the stakes are sufficiently high. Are these claims correct? We investigate the change in behavior as incentives are scaled up in the Allais paradox, and document an increase, not decrease, in deviations from expected utility with higher stakes. We also find that if one needs to approximate participants’ behavior in real high-stakes Allais (which are often too expensive to conduct), it is better to use hypothetically high stakes than real low stakes, as is typically the practice today.
People Place Larger Bets When Risky Choices Provide a Postbet Option to Cash Out
Daniel Bennett et al.
Psychological Science, forthcoming
Abstract:
After a risky choice, decision makers must frequently wait out a delay period before the outcome of their choice becomes known. In contemporary sports-betting apps, decision makers can “cash out” of their bet during this delay period by accepting a discounted immediate payout. An important open question is how availability of a postchoice cash-out option alters choice. We investigated this question using a novel gambling task that incorporated a cash-out option during the delay between bet and outcome. Across two experiments (N = 240 adults, recruited via Prolific), cash-out availability increased participants’ bet amounts by up to 35%. Participants who were more likely to cash out when odds deteriorated were less likely to cash out when odds improved. Furthermore, the effect of cash-out availability on bet amounts was positively correlated with individual differences in cash-out propensity for bets with deteriorating odds only. These results suggest that cash-out availability may promote larger bets by allowing bettors to avoid losing their entire stake.