Hacker News Reader

Comment by janalsncm | Hacker News Reader

Once you go in on infrastructure you have become a small data center. You will need to maintain it, continuously finance it, and secure it. And what if someone wants to back out 6 months later? Now you’re signing contracts with the implication that you’re willing to take a friend to court.

You can get around this by hosting in a 3rd party data center, but now you have the same trust issue again, but with more steps.

It’s all overkill for most people anyways IMO. This lady was just using it as a personal journal. Basically a glorified ELIZA. That kind of thing can be done with really small models locally these days.

View on HN

> This lady was just using it as a personal journal. Basically a glorified ELIZA. That kind of thing can be done with really small models locally these days.

And she's facing felony charges. Yeah, folks on hacker news can talk about technical solutions all day long, but that's not the fundamental problem. The fundamental problem is that we've allowed our digital infrastructure that includes everything from banks to schools to doctors to become adversarial to us. In some cases predatory.

It's effectively untrustworthy, like being charged a felony for walking across a bridge incorrectly. That's not a "I'll host a small model in my garage" problem. It's a we need the government to do its job problem.

Well this is why I think small local models are the opportunity of a generation. Because it’s a bit transgressive these days to say you don’t need a big centralized service that spies on you and requires a subscription. It’s not the absolute best, but it’s yours.

I agree in general that we technical solutions to social/political problems are suboptimal. But technical solutions don’t require the political system to work, which is a very attractive property at a time when politics seems broken.

There is a counterpoint that retreating into technical solutions only cedes more ground and makes things worse, and for that I don’t have a good reply other than the fact that life is short.

I mean, this is also just kind of how mandatory reporting works.

If you tell a teacher, a therapist or a priest that you are going to kill somebody, in many states they do actually have to report that and can be held liable for the resulting crime if they do not.

Every time we have a mass shooting in the US it inevitably comes up that they wrote it down or told somebody and then the next logical question becomes “how could we have stopped a mass casualty event?”

>the next logical question becomes “how could we have stopped a mass casualty event?”

'No Way to Prevent This,' Says Only Nation Where This Regularly Happens

I think the issue here is that I don’t want my automated IT to be a mandatory reporter.

My paper journal isn’t a mandatory reporter, neither should my google doc or my claude session.

Even with local models, I don’t want the eventual creep to require local software to report (eg, CSAM reporting from local storage and compute on iphones, etc etc)

We need actual privacy laws in the united states. Between this, other cloud providers, the advertising tracking infrastructure, phones that constantly ping your location, flock and it's ilk, etc. It's over, no longer can you live semi regularly and still maintain your privacy by maybe living out in the sticks a bit. This is the panopticon.

But ultimately it’s not a technology problem it’s a society problem

The people who are accountable and responsible are delegating their authorities to policy and now the policies are just implemented by machine systems and there’s decreasing human intermediation.

The humans that are still inside the system increasingly have less control such that it’s it’s increasingly difficult to find anybody who is the actual customer service type representative who has the authority to make a meaningful impact.

> You can get around this by hosting in a 3rd party data center, but now you have the same trust issue again, but with more steps

No this is not true — colos are very hands off.

You don't need to operate a mini AWS to self-host, and I don't know why people on HN want to make it sound like self-hosting if you don't have nation state security

> colos are very hands off

That’s a pretty broad statement? Maybe a more correct one would be that some are hands off enough that some people might find them worth it.

And that really only addresses one problem. The other one is the coordination problem. You go in on a $30k GPU with 3 other people. So each person puts down 7.5k. But then one person wants to pull out, and so every remaining person needs to put in an extra $2.5k on top, and they’re questioning whether $10k for a chatbot is really worth it.

I've never heard of a colo looking into their customers servers. They're not in the business of monitoring their clients.

I don't understand the coordination problem – if you want to pull out you have to find someone else to buy your share?

I've seen something like it once, but it was really just the FBI coming in a seizing a rack.

This level of hardware and cost just isn't necessary. You don't need to team up with your friends to go all in on a hardware purchase, which I've never heard of anyone doing anyway. Go spend some time over at r/LocalLLaMa, you'll see.

Yep. As someone who has run my local home “data center” through a synology nas/docker server for 15 years. It’s not so bad.

The worst part of data centers are the users. So with a home server/ friends server it’s easier to control.

> Once you go in on infrastructure you have become a small data center.

A PC at home with a 3090 in it is more than enough if you want to talk to a chatbot about your day.

> Once you go in on infrastructure you have become a small data center.

Wow. I never knew that my Frankenrouter, gaming PC, and handful of laptops, switches, and WiFi APs were a datacenter. The things you learn...

> This lady was just using it as a personal journal. ... That kind of thing can be done with really small models locally these days.

That kind of thing can be done with MS-DOS's 'EDIT.COM'. If you're nasty, it can be done with 'ed', The Standard UNIX Text Editor.

This is also where TEE's with attestation come in - https://redpill.ai/ for example (god I hate their name, but I am a fan of the product)

Reply on news.ycombinator.com