Vision transformer trained on 300M human images with state of the art results on a bunch of human tasks (keypoints, segmentation, depth, normals).
Disclaimer: Co-author here.
Disclaimer: Co-author here.
I wonder if there is something here that requires conda and not a simple requirements.txt or something like that. Every time I try conda is seems to mess up my entire environment (usually I just use pyenv w/ virtualenv). But trying with conda now, keeping my fingers crossed...
EDIT: yep, as usual, conda failed me. (fresh install of miniconda). "./conda.sh" finished with 0 exit code and said "Installation done!". Yet, now I have no new conda environment (I think I saw some warnings and errors deep in the logging output).
I see now how this has various requirements.txt for the different sub-projects - looks like I'll try to create a pyenv-virtualenv and do things manually to try to get an example working...
In case you're not aware, because I was only recently: https://github.com/pyenv/pyenv-virtualenv
The Apache parts seem to be dependencies.
[0]: https://github.com/facebookresearch/sapiens/blob/main/LICENS...
“Everything is useless unless it personally, financially benefits me.”
Why would you spend any of your finite attention here? It's a signal to researchers and would-be upstarts that Meta already lurks here.
Because it is research. And it's an open research, unlike OpenAI or (for the most part) modern Google DeepMind or xAI.
It's completely fair game for non-researchers to ignore that, but even non-researchers benefit from a higher pace of developing understanding how to do this kind of magic.
Thank you people at Meta Research for this release!
Which to me is something important to know when thinking about what can be built and the amount of effort it represents.
For better or worse, that ultimately shapes what can reasonably be done with it.
I’ve always wanted to scan whole books by just feeding Pictures of their pages into an AI. Prefer preferably with minimal labeling requirements. I also see this as a way to generate more training data for language models from old cheap books. Do you think your model could help with that?
Um, this looks really, really good.
Yo @yoknapthawa, can this be finetuned on an M3 chip? How much RAM is needed? What are the current low hanging fruit-type tasks you think the community could go at? What's latency like? I didn't see anything on the page / in the paper / github about speeds.
I'm also curious about the classes you use for the segmentation task -- do you have a list of them somewhere?
Finally, your generalization results are all on photorealistic images, did you do any looking at paintings / animation / other? I'm curious how broadly the generalization goes.
As always, thank you for opening the weights.