Nvidia Broadcast App
nvidia.com
nvidia.com
So everyone can see how it looks with a makeshift lighting setup (kitchen lights, overhead lights, and a 3 bulb lamp).
Background Removal: https://i.imgur.com/e3y4kwX.png
Background Replacement: https://i.imgur.com/AajgJIf.png
Background Blur - Low Setting: https://i.imgur.com/CeXmneX.png
Background Blur - Max Setting: https://i.imgur.com/OoJ76ow.png
Frame Tracking - https://i.imgur.com/wQ8IOq1.mp4
Full Album: https://imgur.com/a/RWECpd4
EDIT: As a followup to a question my friend asked - memory wise it is sitting idly at 105MB; running OBS it went to 670MB.
EDIT2: The blur effect holds even if you move to the background as well without sharpening the objects in the background. It did look a little "off" but that's mostly because a sharply rendered body interacting with a blurred environment looks off.
Thanks for posting.
2450 views on Imgur
245 = 2450 * 0.1
259 likes on this post
147 comments
That's a strange way to say "yeah we admit that we were just blocking older GPUs in the installer and you guys noticed so decided to stop."
You know it just a matter of installing one, but they basically don't want to spend time supporting something they have no interest in.
I work for John Deere, it is well known that ECU code can change the horse power of our tractors. What isn't as well known is the tractors with higher horsepower ECUs have more warranty work, and that is a case where we engineer the system for the change (there is more than an ECU changed in getting higher horsepower if you buy it that way from us because we don't want warranty claims)
If you know what the real limits of your system are you will sometimes discover the stated limits are less than the real limits. However other times the stated limits are there for a reason.
Better than coming out of it looking like the evil dudes.
EDIT being most well known outside your sector for suing your customers is generally a bad look
- Well, we stated that you shouldn't do what you did...
- It doeesnnn't work !!!"
Ensue time, energy and money lost to defend yourself against idiots who don't listen.
Edit : I don't know anything about trucks or John Deere. But you can see this type of "artificial" limitation with simple unlock pretty often. For example the cpu overclocking community : the manufacturer doesn't officialy support overclocking, but they give hints and some advice if you want to do it anyway. You fry your cpu, you're on your own.
I'll also point out that you probably were mad at VW for making the types of modifications that you claim anyone should be able to make.
It's kind of funny, because parts DRM could be seen as the manufacturer breaking the device when a replacement could have succeeded.
If the tractor is rated for 50 whatever and you chip it to run at 60 whatever and sell it without saying that you'd ever done so, then if the buyer finds damage caused by overheating or who knows what from running at 60 whatever on a 50-rated-and-softlocked engine, then you absolutely could be held liable for failing to indicate the non-factory modifications you made to the engine, even if you later 'removed' them.
(Usual disclaimer: I am not your lawyer, I will not compile citations for your review, please seek legal counsel before taking action based on my comments.)
Honestly, I'm a fan of price discrimination through DRM. It leads to better economic efficiency. But John Deere DRM must really be something else. I'm not even in the field and I have a negative reaction to the brand. Like they're going to nickel and dime me and then give me the run-around on fixes.
I don't really see too much of a difference between buying a tractor from John Deere and buying a fancy SAN from Oracle. In either case you're paying a premium for the name and the associated service contract for a time-sensitive mission-critical locked-down (and to some extent black-box) function which is presumably crucial to your business's ability to generate revenue. JD have competitors, and I'd imagine that if this episode hasn't tanked their business it's due to risk-aversion from buyers (à la IBM), vendor lock-in, or customers being set in their ways.
It's also a bit like the Creative sector when Adobe moved to a subscription model; people balked and held out for a time, but ultimately they voted with their wallets and bought [subscriptions to] the magic product that could be provided by only one company on Earth, and without which they couldn't possibly continue to make art-work [commercially, in a productive manner, and in keeping with their prior experience].
I'm just cynical about how often people will knowingly make themselves reliant on a single company without reading through the small print or thinking through the implications of what is essentially entering into a long-term strategic partnership with a vastly unequal partner without a having a hedging strategy. If you can't imagine not using their products after the next refresh, you've put yourself in a very poor negotiating position.
Right-to-repair is all about people being able to repair their own products, or at the very least use certified 3rd-party mechanics, instead of being forced to go to the manufacturer for every possible problem.
Besides, if you have ever used a warranty recently then it’s a good bet that it was legislated to act that way. Right to repair is similar in focus to help consumers.
All that minimum warranty legislation does is to remove the freedom that people previously had to buy products with shorter warranties, or without one. It means that they can't even buy the same product together with an optional warranty from the manufacturer, thereby ending up with a situation that's functionally identical. Warranties for practically every type of durable product existed before these changes were introduced, and people have always tended to weigh up the projected lifetime (which they can infer from factors such as the availability of manufacturer or third-party warranties or insurance), manufacturer reputation and projected total costs of ownership whenever they have bought something sufficiently expensive.
Agriculture is of course an important industry, but I don't think there's any wider threat to agriculture in this particular case, and if these practices were to create one in the future, the market would respond by punishing a company like JD (and no doubt the state would as well in light of the strategic implications). It just seems like a slow and painful realisation for parts of the industry that things have changed. Hopefully we'll now see plenty of disruption and innovation (and modding ;)).
I think it’s rather that it was introduced in a domain where users historically can (and often need) to be able to do more or less anything as many operations are extremely time-sensitive and they generally live in the ass-end of nowhere, CA. Any measure which can (and does) hamper the ability to do the thing right here right now without huge benefits to compensate is an active threat.
Me too, because it means I can buy the cheapest version and unlock it for free.
We have a vendor that sells equipment with a built-in engine, and we use some of the power from that engine to run other equipment. Under Tier-4 emissions requirements the vendor moved to all-digital controls, which have a much tighter control on power output. Our customers are annoyed that they can't get the same amount of power out of the new equipment. Our vendor tells us, unofficially, you could tweak the ECU and get more power out, but we're not going to do that to equipment we're selling, and we're not going to tell our customers to do that, because it would violate the legal restrictions on The operation of that equipment.
I 100% guarantee you there are people who will call Nvidia support (yes, they will literally dial +1-800-797-6530 on a phone like it's 1972) to complain about broadcast. Some people are unbelievably entitled, and think that no matter what it says on the box if they paid good money for it then it should by God do whatever stupid ass thing they want it to do.
Non-RTX cards are technically capable of DXR as well, but the lack of specialized silicon makes it a nonstarter. I've used RTX Voice for months now, and coworkers have noticed issues and artifacts on occasion, I imagine they occur more often on non-RTX cards.
Seems to pretty heavily imply that it was an entirely artificial requirement.
Also it's NVidia, all of the home grade cards are artificially limited by drivers.
In some ways yes, but in the way this is usually meant (DP flops) this hasn't been true for many generations. GeForce chips simply only have one DP ALU for every 32 SP ALUs, while the HPC accelerators have one DP ALU for every two SP ALUs.
This also impacted only a tiny subset of CAD/professional imaging products which were out of reach of consumers anyhow often due to the massive PITA and cost that was certifying your hardware and drivers for those products.
Quadro wasn't running faster in Blender, 3DSM not even AutoDesk (in fact they were often slower due to lower frequencies), the apps were it made a difference when those drivers were finally released for Geforce and Titan cards were only the likes of Catia and Siemens NX hardly consumer/prosumer software.
I'm sorry but if you are a 7 figure a year license holder for Siemens NX spending $2000-3000 extra per on each GPU every 3 year isn't going to be an issue, not to mention that you aren't going to be using Geforce GPUs anyhow as the drivers are still not certified (which is required for the industry) and even more likely you'll be buying certified workstations from the likes of HP and Dell.
The Titan / Titan Black had 1/3 FP64 just like the Quadro K6000 which used the same die.
The normal ALU's aka CUDA Cores and the Tensor cores on RTX cards can run concurrently without blocking, without require context switching and for the most part outsides of the register file / cache in that SM and GPC they aren't competing for resources.
Sure no model runs only on the Tensor Cores and probably requires some generic ALU usage but optimized models only require a fraction of their execution time to involve those standard cores.
Optimizing their model to work on a multiple generation of GPUs with different compute capabilities (if you look at CUDA compute you have native/multiple instruction operations constantly being switched in and out) without having a severe impact on performance that would lead to "OMG RTX VOICE MAKES YOUR GAME LAG!!!!!" posts on social media and streamers trashing the software and the brand isn't easy.
So you launch it first on hardware where you can guarantee that it would run well without adverse impact then release it to a broader audience with a huge disclaimer often to showcase just how better your new hardware is like in the case of DXR ray tracing which now runs on essentially all DX12 GPUs from NVIDIA.
People in general are clueless to how GPUs actually work, and they tend to sensationalize everything (especially when it comes to Team Green vs Team Red) just look at the nonsense that came out after Crytek demoed their "ray tracing" on VEGA people were basically saying RTX is a scam.
The reality is quite different, Crytek used a hybrid model, they weren't running GI via ray tracing they were using SVOGI and have implemented very rudimentary ray tracing for reflections with heavy limits on how many BVH instances you can have at the same time and at what range objects can fall into the BVH.
So yes it can run for example on the Xbox One X but at <30 fps, with low quality SVOGI and upto 5 objects being able to be reflected at any time via RT.
NVIDIA isn't limiting homegrade cards artificially, FP64 units aren't there in the smaller dies, AMD used to do it during the days of GCN as their big dies still had full FP64 ALUs.
The only feature that NVIDIA has been currently limiting on their GPUs is SR-IOV support which what would you know is also disabled on AMD consumer cards ever since SR-IOV support was introduced with VEGA.
Even the "it's-definitely-not-the-founder-edition" VEGA FE which was quickly re-marketed at launch as a prosumer card despite the fact that it's driver support was essentially killed within 6 months which was sold at a huge markup (compared to Vega 56/64 which came later) didn't enable it.
Posts like this is why we can't get good stuff, because people will always complain why something doesn't work and then go into conspiracy theories when something does work only on newer hardware.
Yes, I agree that initially choosing not to support GTX cards was a money-based decision, but sometimes there are additional (somewhat more reasonable) factors that contribute to that money-based decision, which people often seem to leave out.
You know that all products with any electronics in them do this, right? It's not an Nvidia thing by any stretch of the imagination. Your phone and your computers all have software feature toggles for features meant for future hardware. Your OS has features disabled by design. Unless you're driving a 30 year old car, it's probably true that your car has features turned off that are meant for other models. Since you're on HN, there's a reasonable chance that the company you work for releases a product with "artificially limited" features. (Of course I have no idea what you do or who you work for, just making the point that hidden features are so common, the odds of me guessing right are quite good.)
There are good and legitimate reasons why products turn features off, especially when the features were designed for specific hardware, with specific specs in mind, and work best with specific hardware support. Since the fallback could be a drain on performance, battery, heat, noise, safety, and last but not least user experience, it's pretty understandable why some products have features disabled, especially when the features were developed after the product was released, right?
I've used the "tweaked" RTX Voice on my 980 and it mostly works, but while testing it in some social Zoom meetings I also found it was causing artifacts unless I applied the typical modest overclock profile I use when playing games.
I will confess that one of the reasons I look forward to getting a 3000-series is that I want to use RTX Voice and Broadcast during all of the web conferences I do nowadays. Hardware accelerated fake-chromakey and effective noise cancellation will both be welcome additions.
I might try to run some GPU profiler at some point, my best guest is that the model is time sensitive so it requires a very fast execution and one that might not be parallelized that well.
To me the sound issues seem to be the same as certain temporal based effect shader issues you might encounter at low framerate or frame drops when streaming video basically since the model and the app are built for real time voice buffering is limited so there is an expectation of specific timing constraints in which the computation must be completed when your GPUs cant meet those constraints you get audio artifacts on the output.
Today it's basically nagging developers and technical users, who don't want to run proprietary software unsandboxed (but want CUDA...) and have a hassle-free experience into buying quadros. I can't think of any business-case for that, because cracking down at the shady ML-cloud-vendors works out okish right now for them and at unis people won't notice at large...
Developers without Quadro cards just updated their drivers one day and weren't able to use it anymore.
Same goes for their hypervisor checks. Just silently added in a driver update, with the reps only saying why after people started asking why it broke.
Do you think that these Quadro features materialized out of thin air, or did they require engineering effort and additional silicon area?
If the latter, is it reasonable for Nvidia to expect to be compensated for that additional effort (which is born by, say, 2 orders of magnitude lower volume) ?
Additionally, would you be happier if Nvidia simply removed those features so that no home users would complain anymore (at the expense of Quadro users who would lose the ability to use real features.)
And, using my standard to-go-to argument, are you similarly upset that software companies are charging different amounts of money for different tiers of a software product (e.g. Microsoft Office Home vs Professional), even though these different tiers are only enabled by a license key? If not, why?
Going to the effort of making different hardware for the lower end SKUs that completely lacks these features would end up raising prices for consumers because it makes engineering and manufacturing more costly.
The other alternative is they just don't offer lower price SKUs and make everyone buy top-end parts.
You didn’t buy a GPU, you bought a consumer class GPU promised you the advertised consumer class features.
Why do you expect it to give you professional class features from a higher pricing tier?
Is there anyone who isn't upset by that?
By lowering the price for an entry version, customers who otherwise couldn’t afford the full version still get the option to use a feature reduced version.
(See how I framed that?)
The former Microsoft Excel Program Manager explains it well:
https://www.joelonsoftware.com/2004/12/15/camels-and-rubber-...
> this business about segmenting? It pisses the heck off of people.
> People want to feel they’re paying a fair price. They don’t want to think they’re paying extra just because they’re not clever enough to find the magic coupon code.
> God help you if an A-list blogger finds out that your premium printer is identical to the cheap printer, with the speed inhibitor turned off.
> So, while segmenting can be a useful tool to “capture consumer surplus,” it can have significant negative implications for the long term image of your product.
> Even assuming you’re willing to deal with a long-term erosion of customer goodwill caused by blatant price discrimination, segmentation is just not that easy to pull off. First of all, as soon as your customers find out you’re doing it, they’ll lie about who they are
> If your customers talk amongst themselves, they’re going to find out about the price you’re offering the other people, and you’ll find yourself forced to match the lowest prices for everyone.
So, no, I don't think there are a lot of people upset that there exists an Office product they can afford.
https://www.joelonsoftware.com/2004/12/15/camels-and-rubber-...
> this business about segmenting? It pisses the heck off of people.
> People want to feel they’re paying a fair price. They don’t want to think they’re paying extra just because they’re not clever enough to find the magic coupon code.
I don't know a single normal person who's ever bought Office anyway. Their real customers are corporations, since they have money and can actually be sued for copyright infringement if they use Office without licensing it. Microsoft should simply give it away to normal people and charge corporations for it.
Mind sharing the development & support cost numbers, and the sale numbers for the Office suite that you're basing that statement on? If not, maybe a Fermi calculation to show that it's a reasonable statement?
Empirically, pricing is - always and everywhere - determined by the intersection of supply and demand, nothing more and nothing less. The labor theory of value and its relatives are empirically false.
They're only going to get that, and thus set the price, if there is also a buyer who agrees to pay that price.
In technical terms, I was using "set the price" to mean "discover the the market-clearing price that allows a transaction to happen."
The labor theory of value certainly has issues but at the very least I would expect a critique to provide an alternative narrative of observable pricing pressures, such as scarcity and market controls. This type of reductive thinking doesn't have a clear end to my eyes.
So there must be an agreement between the seller and the buyer on what the item is worth, and "worth" includes what alternatives both of them have available as well.
I hadn't noticed the space between my head and headphones prior to this comment, great catch. I'll definitely be using it during work meetings now, I'm sold.
Noise Removal: remove background noise from your microphone feed – ...
Virtual Background: remove the background of your webcam feed and replace it with game footage, a replacement image, or even a subtle blur.
Auto Frame: zooms in on you and uses AI to track your head movements, keeping you at the center of the action even as you shift from side to side...
These features can be used beyond game broadcasting as well — from video conferencing at home with Zoom, to gaming with friends on Discord.
Hardware: Sony A6000 Mirrorless camera; 2x Elgato Key Lights for illumination; GameCapture HD60 S; RTX 2080 Super; OBS for the video feed and recording.
Google Meets also has similar tech https://support.google.com/meet/answer/9919960?hl=en
Audatic was really cool but they got acquired by a hearing aid company.
Whisper.ai has very similar processing but they are also focused on hearing aids.
Are there any more examples of this?
I wonder if Nvidia will try to move upmarket to own the best UI powered services (e.g. if you want a camera with a virtual background, it has to be nvidia or something similar)
They basically have to move towards vertical software stacks requiring their HW. GPUs are becoming commoditized, with integrated GPUs from AMD and Intel becoming good enough to play modern games now. 8k(haha) may keep things interesting for a while, but eventually AMD/Intel will compete there as well.
I don't know about user-facing apps though. So far, most of what they've released(i.e. Gaugan) seem more like demos.
They have hardware optical flow[1] which doesn't seem widely known, but would absolutely help with parts of this - especially the reframing.
Has anyone else gotten this to work, and what build of the NVIDIA drivers are you using on Windows?
How's that going for you? In my tests (directly on Windows host), tensorflow-directml was about 6x slower than tensorflow-gpu (e.g. with CUDA/cuDNN).
1. As in video on background removal , why was the chair not blurred and because generally in background removal only the person is visible and all rest are blurred, how is it working?
2. How Background Noise removal works, i mean how it is so smoothly removing background noise.How it is able to distinguish which is the primary and secondary,i know it's AI but still want to know the logic and basic understanding.Thanks
There don't appear to be beta drivers.
It works pretty well.
=(
I've had pretty much nothing but good experiences running Nvidia on Linux the past 4-5 years, barring one strange issue with multiple monitors, a DisplayLink adapter, and trying to rotate a screen sideways.
I can see fiscally how it doesn't make a lot of sense for them -- but why write a platform-dependent implementation and not use code that's generic?
Is the problem the GUI? Build it with Qt or something
Lovely!
I mean, isn't the whole point of the tech industry to iterate on old tech and make life better with each improvement?
2, How do you think Skype is doing it...? Here's a hint: the team has published a number of papers on convolutional neural networks.
I'd been trying to get rid of the noise in my mic forever- and RTX Voice just worked.
(That being said, I also had to patch the installer, but like who cares? Someone else did the hard part of figuring it out already, and at that point in time, it was 'beta' software, as well.)
This field is going to change everything. Look what I've been able to do with some of the recent ML innovations: https://vo.codes
Mimicking people's ideolects is a hobby of mine, and I typed in phrases that Dr Phil would say, and the voice itself is good but the delivery is off enough that I know it's definitely not him.
Im sure it all get there one day.