NVIDIA unveils new RTX-powered streaming tech

It’s no surprise that NVIDIA had a few aces up their sleeve for Twitchcon, and it’s equally as unsurprising that it’s powered by their RTX line of videocards. This morning they announced the NVIDIA RTX Broadcast Engine, a trio of SDKs that will allow streaming software like OBS the ability to do some pretty amazing things. Let’s dig in and check it out:

• RTX Greenscreen, to deliver real-time background removal of a webcam feed, so only your face and body show up on the livestream. The RTX Greenscreen AI model understands which part of an image is human and which is background, so gamers get the benefits of a greenscreen without needing to buy one.

Ugh, I *just* purchased a greenscreen! Oh well. It’s still cool.

• RTX AR, which can detect faces, track facial features such as eyes and mouth, and even model the surface of a face, enabling real-time augmented reality effects using a standard web camera. Developers can use it to create fun, engaging AR effects, such as overlaying 3D content on a face or allowing a person to control 3D characters with their face.

I’m eager to see realtime face-tracking come to streaming.

• RTX Style Filters, which use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image. With the press of a hotkey, you can style your video feed with your favorite painting or game art.

Starry night? Well, I mean…sure that’s a thing you can do.

The thing I really like (despite owning a physical greenscreen myself) is that all of these AI-powered SDKs require no special cameras or physical devices.

In addition to these new capabilities, NVIDIA also announced further integration with XSplit, Huya, Douya, and Streamlabs for some additional capabilities:

• Twitch Studio, a new, easy-to-use application for new livestreamers currently in beta, has integrated the Video Codec SDK to enable high-quality livestreaming.
• Discord, the world’s leading gaming chat application, just released a new group broadcasting feature called “Go Live,” which uses NVIDIA GPUs and the Video Codec SDK to accelerate
  broadcasting games in Discord.
• Elgato is one of the world’s leading manufacturers of video capture cards for gaming. It recently integrated the Video Codec SDK into the software of its new 4K60 Pro MK.2 capture card for
  recording 4K at 60fps video in High Dynamic Range.

While you can certainly stream with nearly any card, NVIDIA showcased just how the Turing chip in their RTX line of cards is capable of delivering high quality at near-realtime speed:

With the rise of 4K, there was no doubt that we’d be seeing a lot more push towards that level of streaming, and it seems like NVIDIA is leading the way.   We’ll keep you posted as these new SDKs get rolled into our favorite streaming software, but I’m sure we’ll also be talking about it on our streams, so stay tuned to our Twitch channel.

An early integration of RTX Greenscreen, along with a host of other demos, will be on display in the OBS Booth (#1823) at TwitchCon this weekend.