Do everything and be everywhere all at once: why metahumans are the future of production

In the pursuit of enhancing the production workflows of my team, I’ve tried it all, and it is only sporadically that a new tool comes along with the potential to transform our way of working. 

Metahumans are the newest addition to that list. 

For those who are not familiar with them, metahumans serve as digital doubles of real people. Think virtual influencers, which you’ve probably seen already. Like their real-life counterparts, they establish partnerships, form communities and engage with millions of followers daily. Of course, they are fictional characters, but they are doing wonders for the brands they collaborate with—and in fact, they are easier to work with because they operate under fewer constraints than real people. 

Similarly, metahumans can be everywhere at once. Still, they combine the benefits of a virtual being with the appeal of a natural person—which makes them so advantageous for those of us in the advertising world. For influencers, celebrities, politicians, sports superstars and beyond, having a virtual double enables them to make more digital appearances and take on commercial gigs they otherwise wouldn’t be available for. And on the brand’s side, they can collaborate with these personalities without dealing with their schedules and potentially, for a fraction of the usual cost. 

Metahumans offer new possibilities for localisation

As a chief creative officer, I work alongside a team of designers, developers, 3D artists, animators and creative minds. Together, our job is to deliver relevant creative at speed and scale, using the latest technologies to our advantage. For a long time, we’ve been leveraging Epic Games’ Unreal Engine to automate tabletop production and iterate in real-time. It’s been an absolute game changer since the beginning, so we didn’t hesitate to experiment with Epic Games’ cloud-based app for developing digital humans, MetaHuman Creator—nor with the Mesh to MetaHuman feature—as soon as it was released. 

In case you haven’t heard of it, Mesh to MetaHuman offers a new way to create a metahuman from a 3D character mesh using Unreal Engine. Allowing developers to import scans of real people makes it incredibly easier to create a virtual double of yourself (or anyone else) almost immediately. 

In other words, this plugin unlocks endless possibilities in content creation, customer service, film and entertainment. For those in the industry who often have to localise and transcreate content, working with metahumans is much more time and cost-efficient than dealing with real people. 

Think about it: if a commercial features someone who is a celebrity in a specific region, for example, this technology makes it easy for the brand to replace them with someone who is better known in a different market without having to return to set. 

When combined with AI, metahumans become a powerhouse

For public figures, lending likeness rights to their digital doubles is the perfect trick to save time and energy while still earning income from smaller advertising tasks they usually wouldn’t have worked on. But it’s not all about celebrities. I firmly believe that metahumans are poised to transform other industries, too, especially if we combine them with AI. 

In education, for example, digital humans could teach personalised courses and engage in meaningful conversations with their students worldwide, all at the same time. Moreover, students can access their system on demand and ask their tutor questions just like they would in a one-on-one session—and at a much lower price.

Similarly, we could bring metahumans into customer service, where they could guide millions in real time. 

In building metahumans, context is key

When the Mesh to MetaHuman plugin was released, my team and I set out to develop our prototype and test its functionality and practical applications. First, we specifically scanned our APAC Chief Executive Officer, Michel de Rijk, using photogrammetry with Epic Games’ Reality Capture. 

This technique works with multiple photographs from different angles, lighting conditions and vantage points to truly capture the depth of each subject and build the base for a realistic metahuman mode. Then, once the scanning was completed, we imported the geometry into MetaHuman Creator, which our 3D designers refined using the platform’s editing tools. 

Naturally, certain parts of the face, such as the mouth, were more challenging than others. So if you want the metahuman to look genuinely realistic, refining it’s essential, especially if it’s your boss you’re cloning. 

Once we got our prototype as close to perfection as possible, we used FaceWare’s facial motion capture technology to unlock real-time facial animations. For this particular project, FaceWare became our tool of choice due to the multiple customisation options. However, there are limitations, and manual tweaking is still a necessary part of the process. That said, different alternatives are available depending on the budget, timeline and part of the body you want to animate. 

Unreal’s LiveLink, for example, offers a free version that allows you to use your phone and is easy to implement in real-time and pre-recorded applications but focuses on facial animations only. On the other hand, mocap suits with external cameras allow for full-body motion capture but with mid-fidelity. What’s more, the environment we intend the metahuman to inhabit is worth considering, as the clothes, hair, body type and facial structure must all fit accordingly. 

All in all, metahumans have everything in their favour to save us time, money and resources in our content creation endeavours. Of course, this technology is still nascent, but there’s no better time to start sharpening your skills and experimenting. In the meantime, we’ll continue to push forward in our pursuit of realism by focusing on new features, skin tones and textures. 

---

John Paite is the chief creative officer of MediaMonks India