How are you planning to train a model based on annotated actions done on a page? Wouldn’t it be more practical to have a model like PIX2ACT that understands the action to be performed and does it?
Or are you planning to compose future actions from decomposing flows?
I'm shocked that PIX2ACT isn't more common
I don’t want to be rude as I assume English is not the first language of the people behind this project but certainly something like ChatGPT could clean up this text.
Not that I love text fully generated my AI but it’s got to be better than what’s they have currently.