It might be just enough to make you question the nature of your reality.
Meta CEO Mark Zuckerberg and AI researcher Lex Fridman today showed off what Meta’s metaverse tech can really do, using ultra-realistic "codec avatars" developed by the company’s Reality Labs.
The duo recorded a podcast using digital doppelgangers that capture subtle facial expressions and body language, inducing an "incredible" sense of presence and intimacy, according to Fridman. "This is really the most incredible thing I've ever seen. It felt like we were talking in person," Fridman raved about the podcast. The in-depth conversation also provided insight into Meta's ambitious vision for blending AI with immersive virtual and augmented reality to create the "metaverse"—a fancy name for a digital alternate reality powered by computers.
The ultra-realistic avatars are in a whole different league compared to the legless, Nintendo Wii-like characters Meta unveiled last year. As Zuckerberg mentioned on the podcast, Meta went past the uncanny valley and achieved a level of naturality in the avatars that lets people feel familiar and comfortable with the experience as it becomes more and more immersive.
Meta is betting big that deep integration of AI into software and hardware will enable more immersive social experiences through extended reality platforms. "AI is shaping the way we build the metaverse," said Meta CTO Andrew Bosworth during the company's Meta Connect conference yesterday.
As Decrypt previously reported, Meta announced major strides in multimodal AI combining vision, language, speech, and other capabilities at Meta Connect. This includes AI companions like the personal assistant MetaAI which pairs with the company’s upcoming smart glasses. Users can interact conversationally with MetaAI as it identifies real-world objects and accomplishes tasks hands-free.
Here's my conversation with Mark Zuckerberg, his 3rd time on the podcast, but this time we talked in the Metaverse as photorealistic avatars. This was one of the most incredible experiences of my life. It really felt like we were talking in-person, but we were miles apart 🤯 It's… pic.twitter.com/Nu8a3iYWm0
— Lex Fridman (@lexfridman) September 28, 2023
During the two-hour podcast, Zuckerberg talked about the role of AI in Meta’s products and services, such as Facebook, Instagram, WhatsApp, and Oculus. He said that AI is essential for understanding the content and context of the metaverse, as well as for enhancing the quality and performance of the platform. He also said that Meta is committed to advancing state of the art AI research and development, and to sharing its work with the open-source community.
Zuckerberg also expanded on Meta's plan to embed AI into its vision of a society that embraces the metaverse as a way to interact.
"I could see a world where people stick to the photorealistic," said Zuckerberg on the podcast. "But I can also see a world that once people get used to the photo, the photorealistic avatars, and they get used to these experiences, that I actually think that there could be a world where people actually prefer being able to express themselves in ways that aren't so tied to their physical reality.”
Zuckerberg envisions people using these photorealistic avatars for remote-work meetings, gaming, and social interactions to feel a greater sense of presence and connection. The ability to "teleport" into virtual spaces with realistic avatars of loved ones could profoundly change human communication and civilization, according to Fridman.
Fridman asked Zuckerberg about some of the challenges and controversies that Meta faces, such as censorship, privacy, security, mental health, and competition. Zuckerberg acknowledged that Meta has a lot of responsibility and accountability for its impact on society and individuals, and that it is constantly trying to improve its policies and practices. He also said that he welcomes constructive criticism and feedback from users, regulators, and experts.
Meta is plowing full speed ahead into intelligent AI personas and immersive, extended reality hardware. But significant technological and social challenges remain on the road to realizing the ambitious vision of the metaverse and ubiquitous AI. The initial demos of the metaverse were glitchy and underwhelming. And the public does not seem fully on board yet with strapping on VR headsets or mingling with digital avatars.
But Zuckerberg believes AI-driven authenticity and realism will win over skeptics in due time. Meta's grand vision of living, working, and playing in a hybrid digital/physical world hinges on AI’s continued evolution. For now, it seems Meta is laser-focused on building the technological foundations and bringing this sci-fi vision to life one nimble step at a time.