Midjourney's AI Platform Blocks Images of Chinese President, Sparking Ethics Debate

Generative AI allows users to create all kinds of fakes. In recent days, fake pictures of former President Donald Trump running from police, Russian President Putin cuddling a kitty cat, and even the Pope in a white Balenciaga puffer coat have ricocheted around the web. What can’t you make?

You can't make a picture of Chinese President Xi Jinping—at least not on the generative AI platform Midjourney.

To slow the spread of deep fakes online, Midjourney last week said it would end its free trial version, but the platform can still generate images of world leaders—with the Chinese President being a notable exception. Try to do so, and Midjourney, a San Francisco-based company, tells you that it’s verboten. You can’t even use the China president’s name in a prompt:

Critics argue that the ban constitutes a form of censorship, undermining the fundamental principles of free speech and expression.

"The way the world operates today, and how closely intertwined all of us are, it does matter what speech laws are elsewhere," Sarah McLaughlin, Senior Scholar at the Foundation for Individual Rights and Expression (FIRE), told Decrypt in an interview. "That market or that government might be powerful enough to affect how American companies operate."

McLaughlin pointed to the 2015 Sony hack, which revealed that a scene of the Great Wall of China being destroyed was removed from the production copy of the Adam Sandler film "Pixels," because of concerns that it would hurt the movie’s box office success in China.

FIRE is a Philadelphia-based nonprofit organization that defends free speech rights in the United States, with a focus on college campuses. The foundation, McLaughlin added, recently expanded to become an “all purpose free speech and First Amendment group.”

McLaughlin said that people tend to make the mistake of dismissing international censorship as irrelevant to them.

When the issue of censorship came up on the Midjourney Discord server last year, Midjourney CEO David Holz said the company wants to minimize drama.

“There's no evil scheming—basically we just want to minimize drama,” Holz said when addressing the issue last June.

In a June 2022 post on the Midjourney Discord server, Holz highlighted the difference in how political satire is received in Western nations compared to China, and acknowledged that political satire that's typical in the West could cause problems in China and could even endanger users.

There is a long history of creating satirical images of world leaders, especially in the United States, where freedom of speech and expression is enshrined in the U.S. Constitution, but not so in other countries.

"It's not just a matter of American law versus Chinese law," McLaughlin said. "It's more complex than that, and we may find that the tools we use end up following other countries' laws instead of our own, in ways we didn't anticipate."

Holz suggested that the greater good is allowing Chinese people to use the technology rather than encouraging political satire.

"I think people in China using this tech moves [the] needle in the world in general in a positive [direction]," Holz said. "Random people on here doing Chinese political satire does very little to add to anything—the cost-benefit-analysis seems clear."

Supporters of these restrictions may say they are necessary to prevent the spread of disinformation and the potential abuse of AI technology. Despite the restriction on images of President Xi, however, McLaughlin does not see MidJounrey’s time in China lasting very long.

“It wouldn't surprise me if China ultimately blocks access to it, despite the restrictions, as they tend to exclude Western companies,” she said.

McLaughlin said that what Midjourney is doing is outside the norm, citing a recent decision by Twitter to censor content in India.

“Twitter faced censorship issues in India recently, with government demands to censor a BBC documentary about Prime Minister Modi,” McLaughlin said. “Twitter blocked the content only in India, not globally. Mid-Journey, on the other hand, is making it so everyone can't use Xi Jinping's name, which is an escalation from just blocking Chinese users.”