In my opinion, there’s nothing to keep up with. The future of AI is calling an api. Learning the math or theory is worthless. Better to just understand and leverage the capabilities
People who don't understand the basic 'theory/math', won't even understand things like parameter count, or the tradeoff between intelligence and response-time/running costs, a very important consideration for any real project.
From everything we've seen in AI thus far, it has become more complicated, not less. Prompt engineering has not gotten easier, with the explosion of use cases, it takes real skill and imagination to understand how to optimize prompts. Looking at stable diffusion, new techniques like controlnet are incredibly powerful but beyond the average person to use. LORAs make fine-tuning accessible, but also expected of any professional in the future.
Every white collar company beyond a hundred people, in the future, will probably want their own fine tuned models/plugins/model grounding. Hiring people to deliver even 10% extra accuracy is worth it because AI's are so useful. People need to understand how they work under the hood.
The usage of AI will spike 100x going forward, from a mere curiosity inside the company which only affects management decisions (Say sentiment analysis), to extremely intensive direct interactions with customers. This will drastically increase the need for reliability and tractability of the results. NLP researchers simply have to accept 50% of what they did is useless, and reorient around the other 50%, its the risk of researching in an open field.
I am not sure anyone needs to know the parameter count if you are using GPT4 or similar powerful LLM. Most people also would never need to know "the tradeoff between intelligence and response-time/running costs".
> People need to understand how they work under the hood.
Most people don't understand how computers work, and yet derive immense value from them (Non engineers, even most programmers I have met have no clue how the computer works mostly)
Besides, making your own AI is not easy. Unless you're a big shot company, you'll probably just use an API of an existing one.
Used to be that only large established cabinetmaking shops could produce quality cabinets at scale, now any joe schmoe can rig up a CNC machine, buy a 600$ dewalt planer, and a few other budget tools and can do it in their garage.
It used to be only hollywood studios could produce movies, scenery, and editing at professional levels, now film nerds with programming chops are creating Mandolorian like experiences with Unreal engine.
It used to be only large recording studios could produce the latest beats and set trends in various music genres. Now nerds like Zedd (well not anymore but he started out that way) do it on their personal computers and do it better than most producers.