Why not replace the backend of your site with the LLM? Let it do the recommendations in structured inputs and outputs so the front end isn’t a chat interface but leverages the fuzzy nature of LLM’s.
I'd guess it's because of costs. Either API costs or hardware costs. Hosting a simple webapp is very different from having an LLM backend in terms of marginal costs.
True. I don’t think it takes much of a model thankfully. Luna or something…