Meet the Omnivore: in3D turns selfies into avatars with Omniverse

Editor’s Note: This article is part of our Meet the omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Imagine taking a selfie and using it to get a moving, talking, customizable 3D avatar of yourself in seconds.

A new extension for NVIDIA Omniverse, a global simulation and design collaboration platform, does just that.

Created by the developers of software startup in3D, the extension allows users to instantly import 3D avatars of themselves into virtual environments using their smartphones. Omniverse extensions are the building blocks that allow anyone to build and extend the functions of Omniverse applications.

The in3D app can now bring people, in their digital forms, into Omniverse. It helps creators create engaging virtual worlds and use these avatars as heroes, actors, or viewers in their stories. The app works on any phone with a camera, recreating a user’s full geometry and texture based on a video selfie.

Avatars can even be added in 3D worlds with animations and a customizable wardrobe.

In3D is a member of NVIDIA Inception, a free global program that supports cutting-edge startups.

Simple and scalable avatar creation

Creating a photorealistic 3D avatar traditionally took several months, with costs running into the tens of thousands of dollars. Photogrammetry, a standard approach to creating 3D references of humans from images, is extremely expensive, requires a digital studio, and lacks scalability.

With in3D, the process of creating 3D avatars is simple and scalable. The app understands a person’s geometry, texture, depth and various vectors via mobile scan – and uses this information to reproduce realistic details and create predictive animations for avatars.

Dmitry Ulyanov, CEO of in3D, based in Tel Aviv, Israel, said the app captures even small details with centimeter precision and automatically corrects lighting. This allows for accurate head geometry from a single selfie, as well as an estimation of a user’s exact body shape.

For creators creating 3D worlds, in3D software can save countless hours, increase productivity and lead to substantial cost savings, Ulyanov said.

“Manually creating an avatar can take up to months,” he added. “With in3D’s scanning app and SDK, a user can scan and download 21,000 people with a single GPU and a single mobile phone in the same amount of time.”

Login to Omniverse

Ulyanov said that using in3D’s extension with NVIDIA Omniverse Avatar Cloud Engine (ACE) opens up many possibilities for avatar creation, as users can easily customize avatars imported from in3D to engage. and interact with their virtual worlds – in real time and at scale.

In3D uses Universal Scene Description (USD), an extensible open source file format, to seamlessly integrate its high-fidelity avatars into Omniverse. All avatar data is contained in a USD file, eliminating the need for complex shaders or embeddings. And bringing avatars into Omniverse is just a simple drag and drop.

Once imported into Omniverse via USD, avatars can be used in apps like Omniverse Create and Audio2Face. Users have a full toolset in Omniverse to support holistic content creation, whether it’s animating avatars’ bodies with the retargeting tool or creating their facial expressions with Audio2Face .

To create the Omniverse extension, in3D used the Omniverse Kit and followed the development flow using the computer program VSCode. Being able to place a breakpoint anywhere in the code made VSCode an easy-to-use, convenient, and out-of-the-box solution for connecting in3D to Omniverse, Ulyanov said.

“The ability to centralize our SDK alongside other software for 3D developers is a game-changer,” he said. “With our Omniverse expansion now available, we are looking to expand the base of developers who use our avatars.”

“Having the ability to download our SDK and connect it to all the tools used by 3D developers has made in3D a tangible solution to deploy in all 3D development environments,” said Sergei Sherman, Marketing Director at in3D. “It’s something we couldn’t have achieved on our own in such a short time.”

Participate in the creation

Creators and developers around the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Learn how to connect and create virtual worlds with Omniverse at NVIDIA GTC, the Design and Simulation Conference for the Age of AI and Metaverse, taking place online September 19-22. Registration is free and gives access to dozens of sessions and special events.

Developers can use Omniverse Code to create their own Omniverse extension for the inaugural #ExtendOmniverse contest by Friday, September 9 at 5:00 PM PT, for a chance to win an NVIDIA RTX GPU. Winners will be announced in the NVIDIA Omniverse User Group at GTC.

Find additional documentation and tutorials in the Omniverse Resource Center, which detail how developers like Ulyanov can build custom USD-based apps and extensions for the platform.

Follow NVIDIA Omniverse on instagram, Medium, Twitter and Youtube for additional resources and inspiration. Discover the Omniverse forumsand join our Discord Server and Tic channel to chat with the community.

Source link