Even relatively small RGB -> depth models are pretty good. Which kind of implies depth is well encoded in the RGB, and adding depth would not really reduce entropy, while costing bandwidth.
4,776 karma · joined October 21, 2014
If the author used GPU culling it would likely be faster on modern hardware, they just said they can't because of platform restrictions. But that's what AAA games do.
The modern non-nanite techniques here are basically to regularize to grids, cull the grids on the GPU, and then cull more with a HZB. That's your "mipped occlusion boxes", except it's actually very cheap to do this because you're reusing depth you already had from previous frames, the test for each object just a few texture samples, and it all stays on the GPU. As a bonus, you can do your LODs on the GPU at the same time, saving even more CPU work and memory bandwidth.
Also, depth rejection is not going to help with the problems that culling solves; draw, vertex processing, raster, and then pixel tests is much more expensive than a cull test before doing any of this. And if you're forward rendering w/expensive fragment shader, the overdraw of relying on the Z buffer to do your "culling" can kill you.
It's a public bet, not a valuation based on product economics. Billion dollar bets follow different rules (and I'm not saying it's sane).
I'm not sure how much things like Omniverse are used, but NV has put out some really interesting stuff in the space (like for example ARTY for a somewhat recent example).
Overall it seems like a financial vote of confidence for "world models" (whatever that means, I still don't know). But it could also mean the death of the interesting things WL was doing with Marble + Atlas. It's hard to read PR.
If this gets widely deployed, it wouldn't be hard to spin a narrative that "our latest model is so dangerous you need to have this mystery meat DRM chip lockdown". It also wouldn't be hard to block competing/open source models running on the hardware, for "security".
Imagine how much money this kind of control is worth; why wouldn't they do this? Who would stop them? Seems the signatory companies are already onboard with this.
> It looks kinda cool till it gets overused and then you start hating it.
> This website is vibe coded; it looks cute tho.
This sounds like more of a no true scotsman/elitist argument rather than an indictment of AI use in UI. The author doesn't seem to have a problem with AI UI, other than whatever they decided doesn't vibe with their somewhat contrarian taste.
You could front this with an image -> text model but that would be much lower quality vs latency, and the whole point of doing it with a decision model is remove the latency.
Games are a really interesting testing ground for robotics; if we can solve game playing (incl 3d) we could embody "system one" intelligence into robots that have something emulating general reflexes without needing to fine tune.
When someone gets this right I do think we're all gonna ditch the 2d screens and phones.
It doesn't even matter if it's Meta that instigates this, if money is smelled there will be more consumer friendly alternatives popping up overnight (e.g. see how quickly the industry copied the iPhone).
The bigger issue is that the use cases and harnesses for models is infinite, which is hard to compress into benchmark numbers that actually apply to you.
Everyone is benchmaxxing, desperate to sell, and almost nobody except the labs is doing actual science on the results, so harnesses tend to be chosen on voodoo and hunches, like which company made it. There isn't necessarily a good alternative though, bearing the cost of being a harness researcher is probably not many people's goal.
> This is our first experiment in building a persistent Social World Engine — a world that continues to exist, evolve, and remember even when nobody is watching.
The audience is meant to direct the show. There's a crypto token involved in paying for it. Make of that what you will.
Personally I think it's an invitation to a lawsuit. Back in the day people were running AI Spongebob on Twitch and that got killed for copyright infringement. And this isn't a cartoon, it's Jim Carrey.
The model is quite impressive, though obviously it depends a lot on what you're doing. But you can literally just type whatever you want and it will reasonably happen a few seconds later. Logical continuity is not included; attempting to maintain continuity is probably the bulk of the work for this experiment.
If you want to try your own version, you can vibe code an interface on top of the fal API in a few minutes, it is coding agent friendly.
Keep in mind it's super expensive; per list pricing (at least on fal), it costs $5/min to run, and the minimum is 1 minute.
- in Pokemon gen 1 you can wrap your stats back to zero by buffing them too much
- In FF7 you can overflow the damage calculation to kill Ruby in one hit
- Mario 64 has a mountain of usable glitches; "parallel universes" let you skip collisions at high speeds
- GTA 3/VC/SA have crazy mission/wrong warp corruptions involving parallel game simulations
- NES Mario can be corrupted to execute arbitrary code in TAS
- OOT can be heap-corrupted so badly you can write a loader for an entire DLC via controller ("triforce%", my choice for most insane controller based hack of a game)The post is conflating hype and money with technical innovation, they are not really correlated. Kurzweil is known for saying most innovations succeed based not on technology but on timing. Today, who talks about it might matter even more than timing.
Superior research often gets overlooked in favor of someone raising millions, sometimes people who have produced literally nothing manage to sell it. Not saying that's happening here, but I've seen this pattern a lot over my career.
Someone riding (or manufacturing) a hype wave is playing a completely different game from a researcher. If you're a researcher you can't really feel dejected when someone is making a business on the back of what seems like your research; legal protections are decades out of date, even ignoring vibe coding. If you want to make money/hype/whatever off of your work, do that. But realize that it's a path that's often orthogonal to research.
It's literally stolen IP.
One minus epsilon of the corpus did not give informed consent, or get compensated. The fact that it's laundered doesn't make it any less literally stolen.
For example, the Hadamard activation transform used here feels a lot like multiplying Fourier basis ala DFT; strong parallels to how image codecs work to make the residuals more compressible (especially discrete block codecs like are used in GPU compressed textures).
I thought I was being clever suggesting that you could even abuse texture decode units to efficiently sample compressed LLMs with hardware; turns out Apple foundation models are already doing this [1].
This is 100% about money. The only way to pace the frontier is to make everyone (read: investors) lose all their money (read: no longer expect returns). Then nobody will pay the GPU bill or pay celebrities 10 million dollar salaries to stay at the hot lab. Suddenly the development is paced, almost like magic.
Conversely, it is hard to see how pacing development makes the current bubble justified, i.e. how development could be significantly paced without investors losing their faith in hot returns at current valuations. Faith in a bubble literally equals money, exactly in the way that loans created by a bank literally equals money. You can't have one without the other.
In fact if the whole industry goes bankrupt and investors are burned bigtime (many trillions wiped), this would spread transnationally, fixing the "if we don't do it, China will" loophole.
OpenAI had it right originally, the idea to be a NON profit and vow to never participate in an arms race. It's too bad that was tossed out the window now that there's money.
When you can't easily quantify and point to the benefits, that's the default and it works.
Not to say the industries mentioned don't have legitimate purposes, but yes it's pretty obvious a large part of the AI industry has joined its peers in selling fear, though they would never admit that.
Also, RSI is obviously guided. If only guided by "it's not giving results so we'll try something else". To require that RSI happens in a black box for it to count would be arbitrary, and also not how anyone is going to do it.
Obviously it's not the intent of the bill to jail calculator makers. So the bill should be written to make it clear exactly what it's criminalizing.
GPU driver engineering has received a tiny fraction of the resources of CPU engineering, while being significantly more complex. And GPU users will not pay for performance hits that better the architecture, they will just buy the other guy's GPU/use their driver. So it's a race to the top with performance and race to the bottom with architecture and stability.
Inspiration and creativity used to be something society rewarded you for, since you could capture the value and build on top of it. That was the moat.
Now most people are punished for publishing because your work will usually get taken over by whoever has the biggest mouthpiece or compute, and there is no protection against that. I expect people to realize this sooner or later and take their work underground. It's already happening.
The biggest problem is the Web has a dearth of good Flashy examples of people pouring their life into an experience, ever since the death of Flash. So web agents have no good training set for thinking creatively in this space.
Creativity is an entity's propensity to decrease entropy in a system by creating more order in structures (not necessarily "new").
Humans have a hard time understanding either; evolution of both genes and memes is so many orders beyond what a human brain can process. It's easier to invent religions than to grok reality.
Maybe the future of the web is bifurcation to headless for agents and fancy headful UI for humans.