Highlights –

  • Zuckerberg disclosed the project while speaking at the company’s live-streamed ‘Inside the Lab’ event.
  • The CEO also talked about how Meta is working on AI research to let people communicate naturally with voice assistants, taking a step forward on how people will connect with AI in the metaverse.

Meta (formally known as Facebook) is currently working on Artificial Intelligence (AI) to generate worlds through speech, improve how people chat to voice assistants, and translate between languages. Facebook CEO, MarkZuckerberg revealed about the project while speaking at the company’s live-streamed event “Inside the Lab” held on Wednesday. He pointed out the key steps to build the metaverse, a futuristic idea of virtual environments where users can work, socialize and play. The technology is touted to be the successor to the mobile internet.

Zuckerberg said, “The key to unlocking a lot of these advances is AI.” He explained that Meta was working on a new class of generative AI models that will enable people to describe a world and generate the surroundings accordingly. He also demonstrated an AI concept called Builder Bot in a prerecorded demo. In this video, Mark appeared as a legless 3D avatar on an island, where he gave commands to create a beach and then added other creatives like clouds, trees, and even a picnic blanket.

“As we advance this technology further, you’ll be able to create nuanced worlds to explore and share experiences with others, with just your voice,” said Zuckerberg. However, he did not set a timeline for these advancements or explain how the Builder Bot works.

The CEO explained how Meta is working on AI research to let people communicate naturally with voice assistants, which is a step ahead of how people will communicate with AI in the metaverse. He also mentioned that the company’s Project CAIRaoke was “a fully end-to-end neural model for building on-device assistants.”

Meta said that currently, it is using the model within its video-calling Portal device, and plans to integrate it into Augmented Reality (AR) and Virtual Reality (VR) devices. In an interview, AI Jérôme Pesenti, AI Vice President at Meta, said it was tightly restricting the responses of its new CAIRaoke-based assistant until it could ensure that the system did not generate offensive language.

“These language models are very powerful … so we are making a lot of effort to be able to control them,” said Pesenti.

Meta is also looking forward to working on a universal speech translator to provide instant speech-to-speech translation for all languages. The company previously set a target for its AI system for translating all written languages.

After a dismal earnings report where the social media giant recently lost a third of its market value, it has invested heavily in its new focus on building the metaverse and changed its name to reflect this ambition. This month, the company reported a 2021 net loss of USD 10.2 billion from its Reality Labs, its augmented and virtual reality business.

Different beast

The company’s AI head, Pesenti revealed that Meta is exploring how AI can help moderate content and activities in the metaverse.

“We use a lot of AI for moderation on our main platforms … the metaverse is a bit of a different beast, it’s a lot more real-time,” said Pesenti, who said this was “evolving work” and that Meta was still figuring out the policies for metaverse activity.

At the AI event, Zukerberg also revealed how Meta is preparing on how AI could interpret and predict the types of interactions occurring in the metaverse with the help of self-supervised learning where AI is provided with raw data instead of being trained on lots of labeled data.

The CEO also mentioned that Meta is working on egocentric data, which involves seeing worlds from a first-person perspective. He said that this has brought together a global consortium of 13 universities and labs to work on the largest ever egocentric dataset, called Ego4D.

Meta also plans to expand free education initiatives to bring more racial minorities into tech, which researchers say is essential for creating AI systems free of bias. According to a recent survey across the United States and Europe by recruiter Heidrick and amp; Struggles, about 80% of data analytics and AI executives identify as men and 65% as white.

In another event session, Pesenti said, for maintaining transparency, Meta plans to make open source the recommendations library TorchRec used to personalize products like Facebook’s news feed, said Pesenti in another event session. He said that the company would also publish a feed ranking prototype to show how its algorithms prioritize which content it displays to users on Instagram.