Stating of Knowledge
The effect of seeing scientists as intellectually humble on trust in scientists and their research
Jonah Koetke et al.
Nature Human Behaviour, forthcoming
Abstract:
Public trust in scientists is critical to our ability to face societal threats. Here, across five pre-registered studies (N = 2,034), we assessed whether perceptions of scientists’ intellectual humility affect perceived trustworthiness of scientists and their research. In study 1, we found that seeing scientists as higher in intellectual humility was associated with greater perceived trustworthiness of scientists and support for science-based beliefs. We then demonstrated that describing a scientist as high (versus low) in intellectual humility increased perceived trustworthiness of the scientist (studies 2–4), belief in their research (studies 2–4), intentions to follow their research-based recommendations (study 3) and information-seeking behaviour (study 4). We further demonstrated that these effects were not moderated by the scientist’s gender (study 3) or race/ethnicity (study 4). In study 5, we experimentally tested communication approaches that scientists can use to convey intellectual humility. These studies reveal the benefits of seeing scientists as intellectually humble across medical, psychological and climate science topics.
Does counting change what counts? Quantification fixation biases decision-making
Linda Chang et al.
Proceedings of the National Academy of Sciences, 12 November 2024
Abstract:
People often rely on numeric metrics to make decisions and form judgments. Numbers can be difficult to process, leading to their underutilization, but they are also uniquely suited to making comparisons. Do people decide differently when some dimensions of a choice are quantified and others are not? We explore this question across 21 preregistered experiments (8 in the main text, N = 9,303; 13 in supplement, N = 13,936) involving managerial, policy, and consumer decisions. Participants face choices that involve tradeoffs (e.g., choosing between employees, one of whom has a higher likelihood of advancement but lower likelihood of retention), and we randomize which dimension of each tradeoff is presented numerically and which is presented qualitatively (using verbal estimates, discrete visualizations, or continuous visualizations). We show that people systematically shift their preferences toward options that dominate on tradeoff dimensions conveyed numerically -- a pattern we dub “quantification fixation.” Further, we show that quantification fixation has financial consequences -- it emerges in incentive-compatible hiring tasks and in charitable donation decisions. We identify one key mechanism that underlies quantification fixation and moderates its strength: When making comparative judgments, which are essential to tradeoff decisions, numeric information is more fluent than non-numeric information. Our findings suggest that when we count, we change what counts.
AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably
Brian Porter & Edouard Machery
Scientific Reports, November 2024
Abstract:
As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
Struggles and Symphonies: Does Money Affect Creativity in the History of Western Classical Music?
Karol Borowiecki, Yichu Wang & Marc Law
University of Vermont Working Paper, December 2024
Abstract:
How do financial constraints affect individual innovation and creativity? To investigate this, we focus on Western classical composers, a unique group of innovators whose lives offer a rich historical case study. Drawing on biographical data from a large sample of composers who lived between 1750 and 2005, we conduct the first systematic empirical exploration of how composers’ annual incomes correlate with measures of the popularity (as viewed from posterity), significance, and stylistic originality of their music. A key contribution is the development of novel measures of composers’ financial circumstances, derived from their entries within Grove Music Online, a widely used music encyclopedia. We find that financial insecurity is associated with reduced creativity: relative to the sample mean, in low income years, composers’ output is 15.7 percent lower, 50 percent less popular (based on Spotify’s index), and generates 13.9 percent fewer Google search results. These correlations are robust to controlling for factors influencing both income and creativity, with no evidence of pre-trends in creativity prior to low-income years, suggesting that reverse causality is unlikely. Case studies of Mozart, Beethoven, and Liszt show that low income periods coincide with declines in stylistic originality. Notably, the negative impact of low income is concentrated among composers from less privileged backgrounds, implying that financial support is crucial for fostering creativity and innovation. This paper sheds new light on how finance affects innovation and creativity, at the (previously under-explored) individual level.
Large language models surpass human experts in predicting neuroscience results
Xiaoliang Luo et al.
Nature Human Behaviour, forthcoming
Abstract:
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries. Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.
Densing Law of LLMs
Chaojun Xiao et al.
Tsinghua University Working Paper, December 2024
Abstract:
Large Language Models (LLMs) have emerged as a milestone in artificial intelligence, and their performance can improve as the model size increases. However, this scaling brings great challenges to training and inference efficiency, particularly for deploying LLMs in resource-constrained environments, and the scaling trend is becoming increasingly unsustainable. This paper introduces the concept of "capacity density" as a new metric to evaluate the quality of the LLMs across different scales and describes the trend of LLMs in terms of both effectiveness and efficiency. To calculate the capacity density of a given target LLM, we first introduce a set of reference models and develop a scaling law to predict the downstream performance of these reference models based on their parameter sizes. We then define the effective parameter size of the target LLM as the parameter size required by a reference model to achieve equivalent performance, and formalize the capacity density as the ratio of the effective parameter size to the actual parameter size of the target LLM. Capacity density provides a unified framework for assessing both model effectiveness and efficiency. Our further analysis of recent open-source base LLMs reveals an empirical law (the Densing Law) that the capacity density of LLMs grows exponentially over time. More specifically, using some widely used benchmarks for evaluation, the capacity density of LLMs doubles approximately every three months. The law provides new perspectives to guide future LLM development, emphasizing the importance of improving capacity density to achieve optimal results with minimal computational overhead.
Human Learning about AI
Bnaya Dreyfuss & Raphaël Raux
Harvard Working Paper, November 2024
Abstract:
We study how humans form expectations about the performance of artificial intelligence (AI) and consequences for AI adoption. Our main hypothesis is that people project human-relevant problem features onto AI. People then over-infer from AI failures on human-easy tasks, and from AI successes on human-difficult tasks. Lab experiments provide strong evidence for projection of human difficulty onto AI, predictably distorting subjects’ expectations. Resulting adoption can be sub-optimal, as failing human-easy tasks need not imply poor overall performance in the case of AI. A field experiment with an AI giving parenting advice shows evidence for projection of human textual similarity. Users strongly infer from answers that are equally uninformative but less humanly-similar to expected answers, significantly reducing trust and engagement. Results suggest AI “anthropomorphism” can backfire by increasing projection and de-aligning human expectations and AI performance.
Unlicensed Corrections Violate the Gricean Maxims of Communication: Evidence for a Cognitive Mechanism Underlying Misinformation Backfire Effects
Jacob Thomas & Kevin Autry
Applied Cognitive Psychology, November/December 2024
Abstract:
Successful correction of misinformation is complicated by the possibility of backfire effects where corrections may unintentionally increase false beliefs. Due to the conflicting evidence for the existence of backfire effects in the current literature, the present study investigated the influence of pragmatic licensing (i.e., contextual justification for communicating corrections) on the occurrence of backfire effects. Using text messages to manipulate the presence of misinformation and corrections about the meanings of novel words, we found evidence of a backfire effect occurring as a result of unlicensed negated corrections. Misinformation use was significantly greater when a correction was provided without licensing than when no information was provided at all. We suggest that the backfire effect observed in this study may be the result of a violation of the Gricean maxims of communication, and that this mechanism may help to explain the contradictory findings about the existence of backfire effects when correcting misinformation.