Hacker News Reader

Comment by kergonath | Hacker News Reader

> I think this is a game changer, because data privacy is a legitimate concern for many enterprise users.

Indeed. At work, we are experimenting with this. Using a cloud platform is a non-starter for data confidentiality reasons. On-premise is the way to go. Also, they’re not American, which helps.

> Btw, you can also run Mistral locally within the Docker model runner on a Mac.

True, but you can do that only with their open-weight models, right? They are very useful and work well, but their commercial models are bigger and hopefully better (I use some of their free models every day, but none of their commercial ones).

View on HN

I also kind of don't understand how it seems everyone is using AI for coding. I haven't had a client yet which would have approved any external AI usage. So I basically use them as search engines on steroids, but code can't go directly in or out.

You might be able to get your clients to sign something to allow usage, but if you don't, as you say, it doesn't seem wise to vibe code for them. For two reasons:

1. A typical contract transfers the rights to the work. The ownership of AI generated code is legally a wee bit disputed. If you modify and refactor generated code heavily it's probably fine, but if you just accept AI generated code en masse, making your client think that you wrote it and it is therefore their copyright, that seems dangerous.

2. A typical contract or NDA also contains non disclosure, i.e. you can't share confidential information, e.g. code (including code you _just_ wrote, due to #1) with external parties or the general public willy nilly. Whether any terms of service assurances from OpenAI or Anthropic that your model inputs and outputs will probably not be used for training are legally sufficient, I have doubts.

IANAL, and _perhaps_ I'm wrong about one or both of these, in one or more countries, but by and large I'd say the risk is not worth the benefit.

I mostly use third party LLMs like I would StackOverflow: Don't post company code there verbatim, make an isolated example. And also don't paste from SO verbatim. I tried other ways of using LLMs for programming a few times in personal projects and can't say I worry about lower productivity with these limitations. YMMV.

(All this also generally goes for employees with typical employment contracts: It's probably a contract violation.)

Yes these are indeed the points. I don't really care too much, it would make me a bit more efficient but I'm billing by the hour anyway so I'm completely fine playing by the book.

Not sure I can agree with the "I'm billing by the hour" part.

I mean sure, but I think of my little agency providing value, for a price. Clients have budgets, they have limited benefits from any software they build, and in order to be competitive against other agencies or their internal teams, overall, I feel we need to provide a good bang for buck.

But since it's not all that much about typing in code, and since even that activity isn't all that sped up by LLMs, not if quality and stability matters, I would still agree that it's completely fine.

Yes, it's important of course that I'm efficient, and I am. But my coding speed isn't the main differentiating factor why clients like me.

I meant that I don't care enough to spearhead and drive this effort within the client orgs. They have their own processes, and internal employees would surely also like to use AI, so maybe they'll get there eventually. And meanwhile I'll just use it in the approved ways.

What about 10 years ago when we all copied code from SO? Did we worry about copyright then? Maybe we did and I don’t recall.

It's roughly the same, legally, and I was well aware of that.

Legally speaking, you also want to be careful about your dependencies and their licenses, a company that's afraid to get sued usually goes to quite some lengths to ensure they play this stuff safe. A lot of smaller companies and startups don't know or don't care.

From a professional ethics perspective, personally, I don't want to put my clients in that position unless they consciously decide they want that. They hire professionals not just to get work done they fully understand, but to a large part to have someone who tells them what they don't know.

You raise a good point. It was kinda gray in the SO days. You almost always had to change something to get your code to work. But at lot of LLM's can spit out code that you can just paste in. And, I guess maybe the tests all pass, but if it goes wrong, you, the coder probably don't know where it went wrong. But if you'd written it all yourself, you could probably guess.

I'm still sorting all this stuff out personally. I like LLM's when I work in an area I know well. But vibing in areas of technology that I don't know well just feels weird.

SO seems different because the author of the post is republishing it. If they are republishing copyrighted material without notice, it seems like the SO author is the one in violation of copyright.

In the LLM case, I think it’s more of an open question whether the LLM output is republishing the copyrighted content without notice, or simply providing access to copyrighted content. I think the former would put the LLM provider in hot water, while the latter would put the user in hot water.

“We” took care to not copy it verbatim (it’s the concrete code form that is copyrighted, not the algorithm), and depending on jurisdiction there is the concept of https://en.wikipedia.org/wiki/Threshold_of_originality in copyright law, which short code snippets on Stack Overflow typically don’t meet.

Nobody is seriously disputing the ownership of AI generated code. A serious dispute would be a considerable, concerted effort to stop AI code generation in any jurisdiction, that provides a contrast to the enormous, ongoing efforts by multiple large players with eye-watering investments to make code generation bigger and better.

Note, that this is not a statement about the fairness or morality of LLM building, but to think that the legality of AI code generation is something to reasonably worry about, is betting against multiple large players and their hundreds of billions of dollars in investment right now, and that likely puts you in a bad spot in reality.

> Nobody is seriously disputing the ownership of AI generated code

From what I've been following it seems very likely that, at least in the US, AI-generated anything can't actually be copyrighted and thus can't have ownership at all! The legal implications of this are yet to percolate through the system though.

Only if that interpretation lasts despite likely intense lobbying to the contrary.

this is "Kool-aid" from the supply side of LLMs for coding IMO. Plenty of people are plenty upset about the capture of code at Github corral, fed into BigCorp$ training systems.

parent statement reminds me of smug French in a castle north of London circa 1200, with furious locals standing outside the gates, dressed in rags with farm tools as weapons. One well-equipped tower guard says to another "no one is seriously disputing the administration of these lands"

I think the comparison falls flat, but it's actually really funny. I'll keep it in mind.

Your mother was a hamster and your father smelt of elderberries?

Other forms of LLM output is being seriously challenged however.

https://llmlitigation.com/case-updates.html

Personally I have roughly zero trust in US courts on this type of issue but we'll see how it goes. Arguably there are cases to be made where LLM:s cough up code cribbed from repos with certain licenses without crediting authors and so on. It's probably a matter of time until some aggressively litigious actors do serious, systematic attempts at getting money out of this, producing case law as a by product.

Edit: Oh right, Butterick et al went after Copilot and image generation too.

https://githubcopilotlitigation.com/case-updates.html

https://imagegeneratorlitigation.com/case-updates.html

This comes down to a question of what one can prove. NNs are necessary not explainable and none of this would have much evidence to show in court.

Sure there's evidence: Your statements about this when challenged. And perhaps to a degree the commit log, at least that can arouse suspicion.

Sure, you can say "I'd just lie about it". But I don't know how many people would just casually lie in court. I sure wouldn't. Ethics is one thing, it takes a lot of guts, considering the possible repercussions.

"I do not recall"

Yup, Gates style would work. But billionaires have a tendency to not get into serious trouble for lying to the public, a court, congress and what not. Commoners very much do.

Most my clients have the same requirement. Given the code bases I see my competition generating, I suspect other vendors are simply violating this rule.

I have good results running Ollama locally with olen models like Gemma 3, Qwen 3, etc. The major drawback is slower inference speed. Commercial APIs like Google Gemini are so much faster.

Still, I find local models very much worth using after taking the time to set them up with Emacs, open-codex, etc.

How is it different from the cloud? Plenty startups store their code on github, run prod on aws, and keep all communications on gmail anyway. What's so different about LLMs?

I think it's a combination of a fundamental distrust of the model makers and a history of them training on user data with and without consent.

The main players all allow some form of zero data retention but I'm sure the more cautious CISO/CIOs flat out don't trust it.

I think that using something like Claude on Amazon Bedrock makes more sense than directly using Anthropic. Maybe I'm naive but I trust AWS more than Anthropic, OpenAI, or Google to not misuse data.

>How is it different from the cloud? Plenty startups store their code on github, run prod on aws, and keep all communications on gmail anyway. What's so different about LLMs?

Those plenty startups will also use Google, OpenAi or the built in Microsoft AI.

This is clearly for companies that need to keep the sensitive data under their control. I think they also get support with adding more training to the model to be personalized for your needs.

It’s not different. If you have a confidentiality requirements like that, you also don’t store your code off-premises. At least not without enforceable contracts about confidentiality with the service provider, approved by the client.

I would take any such claim with a heavy rock of salt because the usefulness of AI is going to vary drastically with the sort of work you're tasked with producing.

You can set up your IDE to use local LLMs through e.g. Ollama if your computer is powerful enough to run a decent model.

Are your clients not on AWS/Azure/GCP? They all offer private LLMs out of the box now.

That was my question too.

Personally I am trying to see if we can leverage AI to help write design documents instead of code, based on a fairly large library of human (poorly) written design documents and bug reports.

I also kind of don't understand how it seems everyone is using AI for doing their homework. I haven't had a teacher yet which would have approved any AI usage.

Same process, less people being called out for "cheating" in a professional setting.

Also it’s like saying you can host a database on your Mac.

Unless you have experience hosting and maintaining models at scale and with an enterprise feature set, then I believe what they are offering is beyond (for now) what you’d be able put up on your own.

premises, not premise.

https://www.grammar-monster.com/easily_confused/premise_prem...

Have you tried using private inference that uses GPU confidential computing from Nvidia?

Reply on news.ycombinator.com