Hacker News Reader

Comment by boredemployee | Hacker News Reader

>> Should I in fact be keeping up with the latest RAG techniques (which seem to change every time a new foundational model gets released)? >> Should I just stay out of the whole thing and double down on classical ML applications in my domain?

Thank you. Both questions are what I'm wondering right now.

View on HN

If you want my personal opinion, I think the hype around RAG techniques is mostly a fad. I don't see how it becomes a super broad high value technique that your average DS or ML Engineer is getting paid big bucks to know. It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks). Not something worth spending a ton of time on unless you're basically already knee deep in that kind of problem space. I think the only reason it's trendy right now is because LLMs are hammers looking for nails and semantic Q&A is one of the few identified nails.

I'm pursuing of a strategy of:

* Doubling down on domain knowledge. Even if coding becomes completely automated, there will always be value in knowing some particular corner of an industry very well. For me, that's insurance. This includes knowing how classical ML techniques can be applied in my industry (LLMs are not replacing xgboost any time soon).

* Keeping tabs on bleeding edge LLM capabilities, just to have a sense of what's possible. However, staying out of highly technical LLM implementation details like CUDA, inference, batching, etc. Perhaps fine tuning will be valuable but probably I would focus on understanding "fine tuning as a service" offerings rather than rolling my own.

* Investing in knowledge of infrastructure/orchestration of LLM services in a way that doesn't tie me to any particular problem space. It's hard to predict right now what kinds of new products LLMs might enable. But I think there will be value in being able to translate business problems into a high level architecture for an LLM-enabled app, even if that just means wiring together existing service offerings.

Yes. I agree with everything you said, and "infrastructure/orchestration" in the LLM space, seems to be what some ppl are already calling as "LLMops".

Thanks for your input

> It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks)

Maybe, but for every haystack that an LLM can search, RAG could search an even larger one. I don't think RAG is a killer app but I think tools that help LLMs find the right context will continue to be useful.

> Doubling down on domain knowledge

I'm a dev, but this is my main strategy right now. I am digging deep into my corner of fintech and hoping that will stave off some of the disruption I see coming for tech. Personally, I don't see why an AI couldn't also become a domain expert in my niche, but it should buy me some time and potentially align me with some powerful stakeholders (portfolio managers just need to convince the big money that a human touch is good and I will still be business).

Excellent advice.

Question: in what ways does insurance domain knowledge set you apart?

I understand certain industries benefit from Engineers with domain experience (e.g. Healthtech companies needing HIPAA knowledge).

I'm curious as to what particular skills insurance companies need from Engineers?

For pure SWE I don't think I have a great answer for you, but on the ML/DE side there are a lot of industry-specific problems, especially on the P&C side, it helps to have context and experience with:

* Rating (a deep, classical, tabular ML problem) is still catching up to SOTA techniques in many insurance companies + uneven regulatory environment makes it tricky to navigate.

* Marketing is another classical ML problem, the goal is to optimize ad/lead broker spend/conversion funnel.

* CS of course has huge opportunity with LLMs.

* Automated claim fraud detection is a very difficult problem amenable to LLMs/MM DL models. Lots of "pivot to AI" going on here.

And there are others of course. The main thing is that insurance companies have limits on product innovation - policy terms are in many cases mandated by law, so the main differentiator is operational efficiency and exellency. Solutions that move the needle on big revenue/cost levers are very valuable and can be applied widely.

Reply on news.ycombinator.com