For example, has anyone ever attempted image -> html/css model? Seems like it be great if I can draw something on a piece of paper and have it generate a website view for me.
My hope now is that someone will figure out a way to separate intelligence from knowledge - i.e. train a model that knows how to interpret the wights of other models - so that training new intelligent models wouldn't require training them on a petabyte of data every run.
I had a discussion with a friend about doing this, but for CNC code. The answer was that a model trained on a narrow data set underperforms one trained on a large data set and then fine tuned with the narrow one.
(May work poorly of course, and the sample I think I saw a year ago may well be cherry picked)
Have you tried upload the image to a LLM with vision capabilities like GPT-4o or Claude 3.5 Sonnet?
There are already companies selling services where they generate entire frontend applications from vague natural language inputs.