given the focus on transformers in recent years, modernizing and adapting CNNs to hardware advances seems like an under-researched area. it's interesting and something we're exploring.
FAIR did great work with this paper.
FAIR did great work with this paper.
FAIR definitely did great work with ConvNext, and I do hope to see more. There always needs to be people pushing unpopular paradigms.
[0] https://github.com/huggingface/pytorch-image-models
breakthroughs by definition upend conventional wisdom.
transformers currently represent conventional wisdom.
maybe transformers are indeed better than CNNs, but tech history is full of cases where primitive technologies take magical leaps when married to powerful hardware and intelligent system design.
low probability at this stage, especially given the impressive long-range dependencies and generalizability of transformers, but let's see what happens.
besides wightmann, who else do you recommend following for CNNs?
Hard to say tbh. Saining Xie was last on both convnext papers. I do know Trever Darrel always has interesting things going on in his lab. But I can't say anyone consistently though. I've done a few architecture papers but my focus is on generative modeling.
re your comment on chatgpt and citations, my sincere belief is that one of the more enduring benefits of LLMs, once researchers can eliminate/reduce hallucinations, will be to expose baseless claims and flimsy logic.
this won't eliminate all misinformation and misleading arguments, of course, but it will increase friction for bad actors and help good actors identify specious reasoning.
an automated BS detector if you will -- not unlike a noisy car alarm that deters theft.
what do you think?
if unified modeling isn't necessary, or even hurtful, a key advantage of transformers goes away.