Greg Rutkowski Was Removed From Stable Diffusion, But AI Artists Brought Him Back

By Jose Antonio Lanz

Digital artist Greg Rutkowski wants nothing to do with art created using artificial intelligence, but AI art creators won't leave him alone.

Known for his vivid, surreal style, Rukowski is immensely popular among enthusiasts of AI-generated art. In fact, his name became the most invoked keyword used by AI artists looking to mimic a specific artistic style. But he wanted nothing to do with the technology.

After vocally opposing the AI art trend, Stability AI—creators of the popular AI image generator Stable Diffusion—responded by removing his work from their dataset. But the community has now created a tool to mimick Rutkowski's style against his wishes. And since Stable Diffusion is open source, there's nothing he or Stability AI can do about it.

Rutkowski rose to prominence within the AI art scene largely due to the rampant use of his name in art prompts. Keywords like "Greg Rutkowski," "Alphonse Mucha," or "trending on ArtStation" became synonymous with the easy generation of high-quality AI art generation when Stable Diffusion v1.4 and 1.5 were gaining steam.

His style has been requested over 400,000 times, even surpassing legends like Picasso and Da Vinci—all without his consent. The result? A digital quagmire where Rutkowski himself struggled to differentiate between AI-generated pieces and his genuine works.

Despite being the hottest name in the game, Rutkowski became one of AI art's fiercest critics. 

In response to feedback from him and other digital artists, a major change was introduced with the release of Stable Diffusion 2.0. Stability AI chose to remove the ability of emulating the style of specific artists, causing some discontent amongst users. The update was declared "nerfed," as it no longer allowed generation of images in Rutkowski's unique style. It also had problems reproducing human anatomy, and it required a whole new and more difficult technique for prompting.

The move made the updated model highly unpopular, and cemented Stable Diffusion v1.5 as the king of Open Source Text to Image Generators.

Enter the LoRa, a small, very specific model trained to emulate specific things like art styles, faces, stances or color gradings. With Stability AI omitting Rutkowski's style in SD 2.0, the art community saw a void and sought to fill it. The result? A Lora trained to recreate Rutkowski's style, now freely available on Civitai.

Lykon, the genius behind the first customized SDXL model, justified the move.

"Since this artist name was one of the most used in 1.5 prompts and now it's gone, I thought it was a good idea to train a LoRA," he wrote on Civitai.

The move wasn't without controversy. There was some debate in Civitai and Reddit with users arguing for both sides of the coin. "It's kinda messed up to make a LoRA of someone that specifically asked people not to train on their art," one wrote. "Legal doesn't always mean ethical."

Others sided with Lykon, alluding to the fact that Rutkowski's art has already been in circulation in SD1.5 for years. Others simply responded to the whole thing with humor.

While Lykon maintains he'll readily concede to Rutkowski, he said he believes he is acting on behalf of the greater good.

“If he contacts me asking for removal, I'll remove this.” Lykon said. “At the moment I believe that having an accurate immortal depiction of his style is in everyone's best interest.”

As the world of AI and art continues to evolve, the line between innovation and infringement becomes blurrier—especially when words like “styles,” “decentralization,” and open source come into play. The only certainty? The art community isn't shy about shaking up the digital canvas.

In the dynamic interplay between artists and AI, sometimes life does imitate art—and vice versa. If Rutkowski can't beat them, maybe he could just... paint them?

Recommended News