Hacker News Reader

Comment by oceanplexian | Hacker News Reader

> I think the adage about "a solution needs to be 10x other solutions to make someone switch" applies here.

It's already superior to OpenAI because it doesn't require an API. You can run the model on your own hardware, in your own datacenter, and your data is guaranteed to remain confidential. Creating a one-off fine-tune is a different story than permanently joining your company at the hip to OpenAI.

I know in our bubble, in the era of Cloud, it's easy to send confidential company data to some random API on the Internet and not worry about it, but that's absolutely not the case for anyone in Healthcare, Government, or even normal companies that are security conscious. For them, OpenAI was never a valid consideration in the first place.

View on HN

> It's already superior to OpenAI because it doesn't require an API.

But the quality is not superior to OpenAI however. I run Mistral 7B on LM Studio, and I can't get far before it starts giving me wrong answers.

ChatGPT-4 on the other hand is correct most of the time (and knows to trigger Python code evaluation or RAG to answer questions). This makes it useful.

what is the most prominent use case for private LLMs, doctor notes?

Anything related to the business or medium and large enterprises, government

Definitely healthcare, or for certain industries (HFT/Finance/...) where for various reasons _everything_ must be run on prem.

As long as you fit your regulatory requirements, it's incorrect.

You could use it to query against any kind of B2B customer information and provide insight, citations and context without any of the data leaving your private server.

When building something similar powered by OpenAI I had a real pain in the ass anonymizing the data, then de-anonymizing the answers before showing it to the customer.

Also in my example, I'm sure using a string like "Pineapple Cave Inc." instead of the real business name hurt the AI's ability to contextualize the information and data and that hurt the LLM somewhat -- right?

Great answers above, but long term: Personal assistants. I truly think that’s a privacy line people won’t cross, even after seeing Alexa and Google Maps enter into our lives; I think people would rather have nothing than a robot that knows every detail of their health, schedule, feelings, plans, etc. in some vaguely defined server somewhere.

Don’t Google already have that information from your searches, emails, calendar, etc? Obviously you have to trust they don’t misuse it, but it’s basically the same thing as some personal assistant having it to me.

Yeah, but I think this is less of a technical line than an emotional one.

For example: I wanted my personal assistant to track hygiene, which is a natural use case. But then you arrive at the natural conclusion that either a) the user needs to enter the data themselves (“I brushed my teeth and washed my face and took X medications at Y time”), or b) you need some sort of sensor in the bathroom, ranging from mics or radio sensors up to a tasteful camera. And a million subtle versions of (b) is where I see people going “no, that’s weird, it’s too much info all together”

It's not about Google anymore; that ship has sailed for most people by now. But about giving all this data to yet another company. Also, it's not the same data at all.

Some data might never travel across a Google account, but very well over ChatGPT.

If you're processing personal data of other person, then you don't really have a choice in the matter: gain permission from them to transfer their data to a third party or self-host the model.

Proprietary and sensitive information. Personally, I use a self-hosted LLM because I don't trust how my conversations with hosted generative AI services will be used.

This. I also use open source self hosted LLMs for exactly this reason.

Sure, I use OpenAI APIs for certain heavy lifting tasks that don't involve sensitive information, but for anything sensitive it's self hosted LLMs all the way.

Personalized metaspaces, game worlds, content without paying a rent seeker copyright holder.

Education and research without gatekeepers in academia and industry complaining about their book sales or prestige titles being obsoleted

Whole lot of uses cases that break us out of having to kowtow to experts who were merely born before us trying to monopolize exploration of science and technology

To that end I’m working on a GPU accelerated client backed by local AI, with NERFs and Gaussian splatting built in.

The upside to being an EE with MSc in math; most of my money comes from engineering real things. I don’t have skin in the cloud CRUD app/API game and don’t see a reason to spend money propping up middle men who, given my skills and abilities, don’t add value

Programmers can go explore syntax art in their parent’s basement again. Tired of 1970s semantics and everyone with a DSL thinking that’s the best thing to happen to computing as a field of inquiry ever.

Like all industries big tech is monopolized by aging rent seekers. Disrupt by divesting from it is my play now.

This translates to "right now, porn" and aspirations. (n.b. NERFs that can be rendered client side take O(days) to train with multiple A100s)

Forgot re-creation/preservstion of existing content I paid for by translating footage into physics, color, and geometry models, map them to my clients render pipeline. Level 1-1 of New Super Mario Bros is pretty much completely translated. No copyright problems if I don’t distribute it :)

Like I said, most of my money is wfh design of branded gadgets. Not really the sort to care about the reach of others; if content industry collapses because people don’t need to spend money on it, meh. More interested in advancing computing. Pour money into R&D of organic computers, rather than web apps running on the same old gear with more HP under the hood. yawn

I want bioengineered kaiju sized dogs and drug glands that stoke hallucination I’m on another planet.

Humanity is a generational cup and string. Time to snip the 1900s loose.

Nope, they're using GPT for those

https://blogs.microsoft.com/blog/2023/08/22/microsoft-and-ep...

Reply on news.ycombinator.com