Managing Expectations
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality
Fabrizio Dell'Acqua et al.
Harvard Working Paper, September 2023
Abstract:
The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety of tasks. In our study conducted with Boston Consulting Group, a global management consulting firm, we examine the performance implications of AI on realistic, complex, and knowledge-intensive tasks. The pre-registered experiment involved 758 consultants comprising about 7% of the individual contributor-level consultants at the company. After establishing a performance baseline on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. We suggest that the capabilities of AI create a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI. For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities, consultants using AI were significantly more productive (they completed 12.2% more tasks on average, and completed task 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group). Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores. For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. Further, our analysis shows the emergence of two distinctive patterns of successful AI use by humans along a spectrum of human-AI integration. One set of consultants acted as “Centaurs,” like the mythical half-horse/half-human creature, dividing and delegating their solution-creation activities to the AI or to themselves. Another set of consultants acted more like “Cyborgs,” completely integrating their task flow with the AI and continually interacting with the technology.
Go West Young Firm: The Impact of Startup Migration on the Performance of Migrants
Jorge Guzman
Management Science, forthcoming
Abstract:
This paper studies how regional migration to tech clusters impacts the performance of startups within the United States. Startups that move to Silicon Valley experience a significant improvement in performance. This improvement is higher than migrations to other regions in the United States, many of which report null treatment effects. The startups that benefit the most from migration are those leaving low performing entrepreneurial ecosystems and moving to high performing ecosystems, consistent with an agglomeration mechanism. Within different measures of the ecosystem, the level of local patenting predicts startup improvements more than venture capital or the quality-adjusted number of startups, suggesting the local innovation environment is more important to migrant performance than financing or the presence of other startup peers.
The validity of general cognitive ability predicting job-specific performance is stable across different levels of job experience
David Hambrick, Alexander Burgoyne & Frederick Oswald
Journal of Applied Psychology, forthcoming
Abstract:
Decades of research in industrial–organizational psychology have established that measures of general cognitive ability (g) consistently and positively predict job-specific performance to a statistically and practically significant degree across jobs. But is the validity of g stable across different levels of job experience? The present study addresses this question using historical large-scale data across 31 diverse military occupations from the Joint-Service Job Performance Measurement/Enlistment Standards Project (N = 10,088). Across all jobs, results of our meta-analysis find near-zero interactions between Armed Forces Qualification Test score (a composite of math and verbal scores) and time in service when predicting job-specific performance. This finding supports the validity of g for predicting job-specific performance even with increasing job experience and provides no evidence for diminishing validity of g. We discuss the theoretical and practical implications of these findings, along with directions for personnel selection research and practice.
Organizational Ghosts: How “Ghostly Encounters” Enable Former Leaders to Influence Current Organizational Members
Jeffrey Bednar & Jacob Brown
Academy of Management Journal, forthcoming
Abstract:
Leadership research assumes that leader group prototypicality enhances leadership influence during a leader’s tenure. Other leadership research also assumes that leaders, especially founders, can transform their leadership influence into a legacy that will survive death and departure from the organization and continue to impact followers. However, to understand the historically embedded nature of leadership influence, it is critical to unpack how specific leaders can become group prototypes and influence followers after their departure. We address this opportunity by examining organizational ghosts, former organizational members who become the ideal prototype of an organization’s values and identity. Drawing on qualitative data from an organizational merger, our analysis revealed that admired and adored leaders with broad exposure became embodied organizational prototypes. After organizational exit, they became organizational ghosts, institutionalized by associative learning, perpetuated practices, and physical memory work. When activated -- either intentionally or organically -- these ghosts had “ghostly encounters” (remembered and imagined) with remaining members that safeguarded the organization, comforted organizational members, and devalued alternative value systems. We introduce the concept of organizational ghosts, explain how leaders can become ghosts, and specify remembered and imagined encounters as mechanisms through which former leaders can have enduring influence in organizations.
Conformity and Group Performance
Taher Abofol, Ido Erev & Raanan Sulitzeanu-Kenan
Human Nature, September 2023, Pages 381–399
Abstract:
This research provides evidence regarding the causal effect of group conformity on task performance in stable and variable environments. Drawing on studies in cultural evolution, social learning, and social psychology, we experimentally tested the hypotheses that conformity improves group performance in a stable environment (H1) and decreases performance (by hindering adaptability) in a temporally variable environment (H2). We compare the performance of individuals, low conformity groups, and high conformity groups in a four-arm randomized lab experiment (N = 240). High conformity was manipulated by rewarding agreement with the group’s majority and imposing a cost on disagreement. The monetary implications of conformity impaired performance in a variable environment but did not have a significant effect on performance in the stable environment. Intragroup individual-level analyses provide insights into the mechanisms that account for the group-level results by showing that lower conformity in groups facilitates efficient adaptability in the use of social information.
Too naïve to lead: When leaders fall for flattery
Benjamin Rogers, Ovul Sezer & Nadav Klein
Journal of Personality and Social Psychology, forthcoming
Abstract:
Flattery is one of the oldest and most commonly used impression–management tactics in everyday life. Though it often brings benefits to the flatterer, less is known about how it affects the target. In the present research, we explore when and why being flattered can be costly for leaders -- common targets of flattery -- depending on how they respond to it. We suggest that leaders who are observed rewarding flatterers risk appearing naïve to others. Across seven studies and six supplementary studies (N = 4,612), we find evidence that leaders who grant favors to flatterers are often perceived to have naively “fallen for flattery,” which shapes observers’ impressions of the leaders and the organizations they represent. A first set of studies (Studies 1–4) detail the variety of factors that lead observers to conclude their leader has fallen for flattery and the resulting impacts to the leaders’ reputation and their organization (e.g., competence, warmth, commitment to the leader, organizational fairness). The second set of studies look at the contextual factors that impact what costs leaders pay for being perceived to have fallen for flattery, including the type of flattery (Study 5), who is harmed by the favor (Study 6), and the leader’s apparent awareness of the motives underlying flattery (Study 7). Whereas previous research highlights positive consequences of flattery for the flatterer, we find that flattery comes with costs for leaders and their organizations. We discuss theoretical and practical implications for leaders who are frequently flattered.
Zooming out on bargaining tables: Exploring which conversation dynamics predict negotiation outcomes
Matteo Di Stasi, Emma Templeton & Jordi Quoidbach
Journal of Applied Psychology, forthcoming
Abstract:
How much should you talk, pause, or interrupt your counterpart in negotiations? The present research zooms out on the macrostructure of negotiation conversations to examine how systematic differences in conversation dynamics -- the structural and temporal patterns that arise from the presence or absence of speech between interlocutors -- relate to objective and relational outcomes at the bargaining table. We examined 38,564 speech turns from 239 online negotiation recordings and derived, for each negotiator (N = 380), 16 measures pertaining to seven dimensions of conversation dynamics: speaking time, turn length, pauses, speech rate, interruptions, backchannels, and response time. Network analyses reveal that many of these measures are interconnected, with clusters of variables suggesting broad differences in negotiators’ propensity to “talk vs. listen” and to mimic their counterparts. Regression and Least Absolute Shrinkage and Selection Operator (LASSO) analyses further show that several measures uniquely predict objective and relational outcomes in videoconference negotiations. At the objective level, negotiators who speak more, faster, and with fewer pauses tend to get better deals. At the relational level, negotiators who refrain from interrupting and display more dynamic turn length (i.e., low similarity over successive turns) are better liked. Taken together, the results suggest that conversation dynamics could make or break deals.
Sports injuries and game stakes: Concussions in the National Football League
Pascal Courty & Jeffrey Cisyk
Economic Inquiry, forthcoming
Abstract:
The National Football League's regular-season games are not of equal importance: some games loom larger than others for determining a team's chance to qualify for the playoffs. We develop an incentive-based measure of the impact of winning a game on a team's qualification probability to study the relationship between stakes and injuries. We find teams are 24 percentage points more likely to suffer concussions in games where a win secures one team a playoff berth. This is the first evidence to support the risk-escalation hypothesis that injuries increase with a competition's stakes. We then discuss implications for sports injury prevention.
Efficient automatic design of robots
David Matthews et al.
Proceedings of the National Academy of Sciences, 10 October 2023
Abstract:
Robots are notoriously difficult to design because of complex interdependencies between their physical structure, sensory and motor layouts, and behavior. Despite this, almost every detail of every robot built to date has been manually determined by a human designer after several months or years of iterative ideation, prototyping, and testing. Inspired by evolutionary design in nature, the automated design of robots using evolutionary algorithms has been attempted for two decades, but it too remains inefficient: days of supercomputing are required to design robots in simulation that, when manufactured, exhibit desired behavior. Here we show de novo optimization of a robot’s structure to exhibit a desired behavior, within seconds on a single consumer-grade computer, and the manufactured robot’s retention of that behavior. Unlike other gradient-based robot design methods, this algorithm does not presuppose any particular anatomical form; starting instead from a randomly-generated apodous body plan, it consistently discovers legged locomotion, the most efficient known form of terrestrial movement. If combined with automated fabrication and scaled up to more challenging tasks, this advance promises near-instantaneous design, manufacture, and deployment of unique and useful machines for medical, environmental, vehicular, and space-based tasks.