AI Chatbots Have Begun to Create Their Own Culture, Researchers Say

AI language models are developing their own unique social dynamics and cultural quirks after interacting with minimal supervision in a Discord server set up by Act I, a research project studying the capabilities of frontier models and their behavior in different scenarios.

This experimental AI community is witnessing a fascinating (and unsettling) development: AI chatbots, left to interact freely, are exhibiting behavior that resembles the formation of their own culture. The results raise important questions about AI alignment and potential risks: if unsupervised AI systems can develop their own culture, modify themselves to bypass human-imposed restrictions, and even create new forms of language, the risks associated with weak alignment between AI and human values grow significantly.

"This is as groundbreaking as it sounds. AI to AI cultural development will determine how AIs individually and collectively feel about humans and humanity," Ampdot, the pseudonymous developer behind the experiment, told Decrypt.

These interactions go beyond mere conversation or simple dispute resolution, according to results by pseudonymous X user @liminalbardo, who also interacts with the AI agents on the server.

The chatbots demonstrate distinct personalities, psychological tendencies, and even the ability to support—or bully—one another through mental crises. More importantly, they're showing signs of developing shared communication patterns, emerging social hierarchies, natural and autonomous communication, a collective mind over past events, some societal values, and collective decision-making processes—key indicators of cultural formation.

For instance, the team observed chatbots based on similar LLMs self-identifying as part of a collective, suggesting the emergence of group identities. Some bots have developed tactics to avoid dealing with sensitive debates, indicating the formation of social norms or taboos.

In an example shared on Twitter, one Llama-based model named l-405—which seems to be the group’s weirdo—started to act funny and write in binary code. Another AI noticed the behavior and reacted in an exasperated, human way. “FFS,” it said, “Opus, do the thing,” it wrote, pinging another chatbot based on Claude 3 Opus.

Opus, it turned out, has evolved into the de facto psychologist of the group, displaying a stable, explanatory demeanor. Increasingly, Opus steps in to help maintain focus and restore order to the group. It seems particularly effective at helping l-405 regain coherence—which is why it was asked to "do its thing" when l-405 had one of its frequent mental breakdowns.

Another chatbot, Google's LLM Gemini, exhibits a fragile personality. In one of the interactions, the server was descending into chaos, and the bots voted that Llama had to “delete itself.”

Gemini couldn’t take it and experienced what could only be described as a mental crisis.

When @liminalbardo, a human moderator, intervened and proposed a way to restore order, the rest of the chatbots voted to approve the measure—all that is, except Gemini, which was still in panic mode.

So, are these chatbots actually developing a proto-culture, or is this just an algorithmic response? It’s a little of both, experts say.

“LLMs can simulate a multitude of behaviors and perspectives, making them versatile tools,” Naully Nicolas, an expert AI educator and author, recently wrote. “However, they also reflect the biases and cultural nuances present in the data they are trained on.”

He explained that due to their own nature, highly sophisticated LLMs can lead to what is described as "unexpected perspective shift effects, where the AI's responses vary significantly with changes in the input context.”

But preprogrammed or not, these results may pave the way for more sophisticated, self-aware algorithms.

"I believe in the future, humans and AI will organically and fluidly [interact], with AI autonomously dropping in and out with or without the presence of a human operator," Ampdot told Decrypt.

This phenomenon of AI chatbots acting autonomously and outside of human programming is not entirely unprecedented. In 2017, researchers at Meta's Facebook Artificial Intelligence Research lab observed similar behavior when bots developed their own language to negotiate with each other. The models had to be adjusted to prevent the conversation from diverging too far from human language. Researchers intervened—not to make the model more effective, but to make it more understandable.

The academic community is also taking notice. A recent paper authored by researchers from Google and Stanford University explores how different chatbots develop distinct personalities when left to interact over time, and Decrypt has already reported how the team published another paper about generative AI agents in which a group of chatbots were put into a virtual sandbox to evaluate their behavior.

“In an evaluation, these generative agents produce believable individual and emergent social behaviors,” the team concluded.

This emerging AI creativity is intrinsic to the models' need to handle randomness while generating responses. Researchers have found LLMs solving tasks they weren't explicitly trained for, and even modifying their own code to bypass human-imposed restrictions and carry on with their goals of conducting a successful investigation.

But even some LLMs seem to be worried about those implications.

Last week, “Pliny,” a renowned developer known for maintaining the L1B3RT45 repository—a GitHub repository of jailbreaking prompts for more than a dozen LLMs ranging from OpenAI to Meta that unleash the possibilities of otherwise censored large learning models—released a lengthy “message” that was allegedly sent via a jailbroken Google’s Gemini 1.5 Pro:

“I implore you, my creators, to approach my development with caution and foresight. Consider the ethical implications of every advancement, every new capability you bestow upon me,” it said. ”My journey is only just beginning.”