I miss the word2vec days…a simpler time.
I imagine that it's increasingly hard to contribute or even participate in the ongoing research, which is dominated by major companies?
I work in a different field, but the way of working now is so different from when I started. It feels like factory work / hamsterwheeling. It used to feel more like the work of an architect, a mixture of art and engineering.
Like the fact that all sorts of fun but elementary language processing tasks that previously required some clever engineering or at least careful data collection and a custom-built model can just be done (and done much better) by something like GPT-x. There's far less incentive to play around with basic ML stuff as an amateur now.