Foundation for Human Vision Models
github.com
github.com
Disclaimer: Co-author here.
I wonder if there is something here that requires conda and not a simple requirements.txt or something like that. Every time I try conda is seems to mess up my entire environment (usually I just use pyenv w/ virtualenv). But trying with conda now, keeping my fingers crossed...
EDIT: yep, as usual, conda failed me. (fresh install of miniconda). "./conda.sh" finished with 0 exit code and said "Installation done!". Yet, now I have no new conda environment (I think I saw some warnings and errors deep in the logging output).
I see now how this has various requirements.txt for the different sub-projects - looks like I'll try to create a pyenv-virtualenv and do things manually to try to get an example working...
In case you're not aware, because I was only recently: https://github.com/pyenv/pyenv-virtualenv
The Apache parts seem to be dependencies.
[0]: https://github.com/facebookresearch/sapiens/blob/main/LICENS...
“Everything is useless unless it personally, financially benefits me.”
Why would you spend any of your finite attention here? It's a signal to researchers and would-be upstarts that Meta already lurks here.
Because it is research. And it's an open research, unlike OpenAI or (for the most part) modern Google DeepMind or xAI.
It's completely fair game for non-researchers to ignore that, but even non-researchers benefit from a higher pace of developing understanding how to do this kind of magic.
Thank you people at Meta Research for this release!
Which to me is something important to know when thinking about what can be built and the amount of effort it represents.
For better or worse, that ultimately shapes what can reasonably be done with it.
I’ve always wanted to scan whole books by just feeding Pictures of their pages into an AI. Prefer preferably with minimal labeling requirements. I also see this as a way to generate more training data for language models from old cheap books. Do you think your model could help with that?
Um, this looks really, really good.
Yo @yoknapthawa, can this be finetuned on an M3 chip? How much RAM is needed? What are the current low hanging fruit-type tasks you think the community could go at? What's latency like? I didn't see anything on the page / in the paper / github about speeds.
I'm also curious about the classes you use for the segmentation task -- do you have a list of them somewhere?
Finally, your generalization results are all on photorealistic images, did you do any looking at paintings / animation / other? I'm curious how broadly the generalization goes.
As always, thank you for opening the weights.
Serious question: who are the people in the illustrations they used in the paper?[1] Are they Facebook/Instagram users? Did the authors ask permission to use their photos for an arXiv publication? Including their kids? Meta researchers really should be answering questions like this before they are asked - but these authors didn't even include an impact statement!
Yo @yoknapthawa, can this be finetuned on an M3 chip? How much RAM is needed? What are the current low hanging fruit-type tasks you think the community could go at? What's latency like? I didn't see anything on the page / in the paper / github about speeds.
I'm also curious about the classes you use for the segmentation task -- do you have a list of them somewhere?
Finally, your generalization results are all on photorealistic images, did you do any looking at paintings / animation / other? I'm curious how broadly the generalization goes.
As always, thank you for opening the weights.