Hacker News Reader

manilion How much does CoT Reasoning typically cost for frameworks like LlamaIndex?

Fair enough, it's "nonsense". Curious to know about YOUR experience. Sounds "nonsense" yet?

manilion How much does CoT Reasoning typically cost for frameworks like LlamaIndex?

The issue is that I can give it a try on a set of toy problems, but finding a real-world-class problem is totally a different story IMHO. That is exactly the reason I asked about people's "experiences".

Anyway thanks for the response.

manilion Ask HN: Privacy-Aware Inferencing for LLMs

Thanks @koutetsu,

I know about FHE and TFHE, but as you said, they need a lot of computational resources.

Ignoring the training process and just looking at inference, what are the "technical" drawbacks of this idea? If the first and last layers of the network run on the local machine, how could a malicious cloud reverse engineer the inputs and discover the raw data?

manilion Ask HN: Versal HBM Series vs. Nvidia A100 for LLM Training?

Thanks a lot @brucethemoose2 Are there any evidences and studies (e.g. papers, case study, etc...) regarding possibility/impossibility of training or running the architecture on top of the infrastructure?