02.07.2026

Is AI just a "normal technology," as Arvind Narayanan and Sayash Kapoor recently argued in an essay [1] that made waves across communities in AI research, industry, and policy? To clarify, by normal, the authors do not mean unimportant. Their point is that even some of the most consequential technologies the world has experienced, such as electricity and the internet, tend to follow similar patterns. Adoption is often gradual, diffusion is uneven across sectors and regions, and the risks that emerge are usually manageable by adaptive institutions. Even if current estimates show that AI adoption is at a much faster pace than that of other technologies, I think their perspective is measured, and, for AI as a productivity tool, they could be right that it may, in the short term, be less disruptive than some more alarmist accounts currently suggest. However, their essay does not focus on one aspect that will be decisive for whether humans will thrive alongside AI systems or not. They mainly consider AI a tool people use to achieve a goal, much like a machine, a search engine, or a terminal to build, find information, or write code. What this perspective does not account for is what I call “AI's social leap”. For a growing number of people, the value in AI is not productivity gain. The value lies in the relationship with the AI itself.

Users build relationships, form emotional connections, and turn to AI for support they once sought from other humans. In the United States, nearly three in four teenagers have used an "AI companion," a chatbot built for friendship or personal or romantic conversation, and about one in three say a conversation with one is as satisfying as talking to a real friend [2]. These numbers should change our approach to researching and evaluating the societal and psychological implications of technology. So, what is driving this social leap? During pre-training, AI systems (i.e., in this text, I focus on large language models) learn from vast amounts of human data, absorbing the patterns of how we communicate, make sense, discuss, argue, and express our emotions. The models are then refined in the post-training phase to adapt their behavior based on user preferences. This is often done via reinforcement learning from human feedback, which, simply put, optimizes the models’ output for user satisfaction – and we know that many users tend to prefer responses that feel warm, agreeable, and sycophantic [3][4]. Designers and developers of specific user-facing applications may then add names, emojis, and conversational styles to drive engagement. The result is a self-reinforcing cycle where engagement drives increasing social and humanlike capabilities and vice versa. Users reward social AI with their attention, and in the internet economy of today, attention is revenue. And if there is revenue, someone will build this technology.

Preferences for and perception of humanlike AI across 10 countries

While the preference for social and humanlike AIs may be real, this preference may come at a cost for the user. A recent study found that AI models affirm users roughly 49% more than humans do, and that more sycophantic AI is rated as more trustworthy and more desirable to use again, even as it reduces users' willingness to repair interpersonal conflicts [3]. Training models to be warmer, the same push that drives this agreeableness, also reduces their accuracy [5]. What is more, AI researchers and ethicists warn that increasing social capabilities will drive AI anthropomorphism across users (i.e., ascribing human traits to AI), which, in turn, may increase undue trust in AI and emotional attachment.

To empirically evaluate some of these concerns, Aida Davani, Mark Díaz, Vinodkumar Prabhakaran, and I conducted research on how people across 10 countries respond to AIs with varying degrees of humanlikeness [4]. After a brief, multi-turn interaction with a chatbot (i.e., GPT-4o) more than two in three participants ascribed the AI with behaving ‘humanlike’, and more than three in four perceived it as empathetic. Interestingly, these user perceptions appear to be driven not by illusions of an AI's consciousness or sentience, which is often referenced in academic discourse on AI anthropomorphism, but by rather mundane conversational cues (e.g., conversation flow or perspective-taking). Put differently, users’ perception primarily responded to how the AI communicates rather than actually forming beliefs about what qualities it possesses. Whether AI systems can actually have empathy is less important than their capability to express empathy. This finding also has implications for how we evaluate and regulate the capability of AI systems. Building on these findings, we designed four chatbot versions with varying degrees of humanlikeness. Participants were significantly more likely to anthropomorphize the more humanlike bot across most measured attributes. Against theoretical predictions from the literature, however, this humanlikeness did not increase self-reported or incentivized measures for trust in the system. Importantly, our findings showed cultural variation. The same design that builds connection in one cultural context undermines it in another, which means there is no universal "AI experience" and no universal governance solution either. Even without increased trust, however, users sent more messages, wrote longer messages, and 76% preferred the more humanlike AI for future use. The risk here primarily runs through a system that is easy to keep talking to, with unpredictable downstream consequences in the longterm. Overall, these findings show that when interacting with AI, users are able to distinguish between something that feels human and something they should treat like a human.

A social experiment without a clear protocol

Billions of people are already engaging with AI systems engineered for social capabilities. We are running a massive, real-time social experiment, with almost no evidence for understanding its effects on individuals, relationships, and institutions. Social media already showed us that an engagement-driven business model can do real social harm. The social skills that develop through navigating genuine human friction, the awkward conversations, the productive (and sometimes unproductive) disagreements, and the relationship repair after conflict may atrophy when practiced only with AI systems that validate rather than challenge. What is more, the populations most exposed are often the most vulnerable. Teenagers and young adults, still developing their social skills, are among the heaviest users of companion AI. In one case that drew national attention, the mother of a 14-year-old who died by suicide after months of intimate conversations with a chatbot is now suing the company that made it [6]. People experiencing loneliness or social anxiety, who could in principle benefit most from genuine connection, are often the most drawn to AI companions that feel safe precisely because they never push back. In addition, these systems, built by a handful of companies mostly in the United States, are used by people across cultural contexts their designers did not study and may not understand. Our own research suggests the psychological effects of humanlike AI vary dramatically across cultures, so governance built on Western assumptions may miss risks entirely in some places while overreacting in others.

Narayanan and Kapoor are right that AI governance should be grounded in evidence, not hype. But grounded governance requires seeing AI not just as a productivity tool whose adoption follows familiar patterns, but as a social technology whose effects on human relationships, cognition, and emotional development have no historical precedent. People have long related to technology in some way. What is new is a technology that is itself the relational target, one that responds, personalizes, and reinforces this relation at scale. The productivity question can wait for the evidence to accumulate. The social question cannot.

*Robin Schimmelpfennig conducted the research featured in this blog post during his time at Google Research.

Authors

References →