I agree, low-level highly specialized technology roles within game companies do pay well.
64 karma · joined July 4, 2023
Meaning who decides if an outcome was yes or no? Answers to things like "Who will win the next Best Picture Oscar?" are fairly obvious and binary.
Can we make bets whose answers are not binary yes/no?
What about "Will celebraty X and Y break up?"? Does Polymarket go to X and Y to confirm if they broke up or something :D
Robotics folks probably want speed and accuracy. I'm from the video game industry so I generally look for speed and stability.
Note: This is a loose analogy and recent techniques are already blurring the lines between these axis.
* You could instruct your LLM to interact with a simulator to run experiments and infer behaviour
* You could edit the transformer model and inject spatially relevant data rather than text as is done in above paper
* You could change the architecture to be more condusive for representating a world state. I.e., LeCun's JEPA world model.
* You could further enhance some of the above by using a differentiable physics engine (eg. NVIDIA Newton) to calculate losses directly.
But at the end of the day if a model has any hope to always produce realistic physics, it HAS to learn the laws of nature in some form or other. It looks to me that the next big leap could be achieved by combining the last two approaches.
P.S.: I like discussing such topics. If anyone knows a forum or discord with like-minded people, please let me know :)
As such I always use this prompt as a test: "A video of a jenga brick tower falling over as a brick is removed. The physics of each brick must be realistic."
It gave me a video of where bricks suddenly disapper or morph into others[1]. The linked video is after 2-3 iterations of me insisting on realistic physics. If you are just glancing at this, you would believe it is realistic.
That said this is still very impressive and one more step towards .. IDK what. But I am a bit reasurred that at least my job won't be fully replaced with AI :)
As such you could try to specialize in this area (collision detection, ray queries, rigid body simulation, constraints, solvers, softbody sim, fluid sim etc.). Of course this isn't for everyone as it requires skills and interest in: low level concurrent programming, maths/linear algebra and physical behavior intuition. If you do find these topics fascinating and can demonstrate some ability in them, your skills will certainly be in demand.
My personal theory is that this is because you can make every sound you hear in English using the Devnagari script, but not the other way around.
Speculation as well: Looking at the trailer for Bitcraft, it looks like they have articulated characters and cloth physics. This raises more questions for me.
"What engine does BitCraft use? The BitCraft client is developed using the Unity game engine. For the server, we have developed a sophisticated distributed system called SpacetimeDB, which hosts the entirety of BitCraft’s server architecture. "
Correct me if wrong, but this suggests the physics engine is Unity's, and not something implemented in SpacetimeDB.
If that is the case I would really like to see some internals of how the physics engine has been implemented in their pattern. I'm not asking for the rapier port source code, but it is hard to think in terms of a new advertised programming paradgim when there's no working examples.
For example I am very curious to see how a constraint solve is implemented in a SQL-like fashion. You would need various math operations and an efficient matrix representation off the top of my head, and I can't think of how that maps to a SQL-like interface.
Game physics involves a lot of things (solvers, collision detection, numerical stability, etc.). I am skeptical of their claims of being able to run physics in what I understand are stored procedures for their database.
I looked at their docs for physics https://spacetimedb.com/docs/unity/part-4 where they demonstrate the simplest form of collision check (sphere overlap). I fail to see how that is an improvement or speedup over existing methods. Some quotes: "This may not be the most efficient way to do collision checking (building a quad tree or doing spatial hashing might be better), but SpacetimeDB is very fast so for this number of entities it'll be a breeze for SpacetimeDB." >> Nothing is quantified with numbers.
"For every circle, we look at all other entities." >> This is the most inefficient N^2 way you could do collision detection.
And not to mention networked physics is a whole additional layer of complexity, where you have to use some form of prediction techniques and very likely end up changing your core physics code to accommodate this.
All of this suggests to me much thought has not been put into the claim "you can also do physics with it!" and its implications. Perhaps it is enough for extremely simple physics, as demonstrated with their demo game. If the author is reading this, I suggest spending some time understanding what this claim implies and qualifying it better.
However I must mention that I applaud their courage to try something so outlandish. If you truly believe your claims are possible, I encourage you to keep working on it.
But I'll be convinced when extraordinary evidence backs extraordinary claims.
Source: I work on a commercial game physics engine and related netcode.
If anyone else is interested in a graduate degree in this field, I recommend doing something similar - I simply made a few reasonable PRs to fix some issues that I found when following along the books.