christina97

My earlier research suggests NVIDIA does not actually cap spikes, it caps the average over short periods of time. So setting the power limit is no guarantee.

But it’s okay to be down when the whole internet is down.

While I’m sympathetic to this argument (it would be great if the internet were a safe place), in practice this thinking leads to governments trying to impose legislation that hurts legitimate uses but does little to protect from the long tail of harm. There’s little that can be done about North Korean state sanctioned cybercrime without a great firewall.

If the perpetrators of this hack were caught and in a developed country, they would certainly be prosecuted for their crimes and not get off light (especially if any data is actually leaked).

I recently set up the 26B A4B model up on vLLM on an RTX3090 (4-bit) after a hiatus from local models. Just completely blown away by the speed and quality you can get now for sub-$1k investment.

I tried first with Qwen but it was unstable and had ridiculously long thinning traces!

You’re absolutely right!

By and large cooling, just like a data center.

If it’s so easy then why don’t we have a high quality classifier?

So in other words… they internalized a number of heavily subsidized externalities of individual transportation by car?

This is frankly one of the most frustrating things about LLMs: sometimes I just want to drive it into a corner. “Why the f** did you do X when I specifically told you not to?”

It never leads to anything helpful. I don’t generally find it necessary to drive humans into a corner. I’m not sure it’s because it’s explicitly not a human so I don’t feel bad for it, though I think it’s more the fact that it’s always so bland and is entirely unable to respond to a slight bit of negative sentiment (both in terms of genuinely not being able to exert more effort into getting it right when someone is frustrated with it, but also in that it is always equally nonchalant and inflexible).

Right, but the question is whether the companies producing foundation models will capture that value or not. Right now it seems like tokens might end up just being a commodity sold at cost plus, and companies higher up in the supply chain will make the money. Electricity changed the world but electricity companies capture very little of that value.

No the US government legitimately sells this kind of insurance (and reinsurance):

https://www.dfc.gov/what-we-offer/our-products/political-ris...

I mean they are “startups” on the way to mega-companies. They need internal tooling to match.

We are in this fascinating stage where tokens that are nominally entirely fungible at a roughly equivalent intelligence level; yet at the same time there is huge market segmentation and differentiation in the non-tangible aspects of those tokens.

A reminder on this topic that copyright does not protect ideas, inventions, or algorithms. Copyright protects an expression of a creative work. It makes more sense eg. with books, where of course anyone can read the book and the ideas are “free” but copying paragraphs must be scrutinized for copyright reasons. It’s always been a bit weird that copyright is the intellectual property concept that protects code.

When you write code, it is the exact sequence of characters, the expression of the code, that is protected. If you copy it and change some lines, of course it’s still protected. Maybe some way of writing an algorithm is protected. But nothing else (under copyright).

Am I reading it right, the widget is seen 5B times per day, and they recruited 8 people for testing to make sure their “redesign would work for everyone”…?

Author is clearly confused about the Anthropic case. The request rate at these generation endpoints is so high that the current batching delay is effectively negligible.

The article does not actually substantiate the claim in its title. All references are simply to articles that (at best) describe how people respond to dating apps.

I would not at all be surprised if some or even most dating apps had a team or org in charge of making the platform “good” for users (using some metrics that really do correlate to what we would think of as a desirable experience); and a somewhat disconnected group of people aiming to increase revenue. This is a pretty standard way of trying to align incentives.

It does not take a genius to figure out that to capture value in the long term requires producing some real value for users.

I believe parent meant that 1 mbps is the speed AFTER you hit 75 GB per month.

Whatever your personal feeling, judgement, or conviction on this matter; do not dismiss the other side because of a couple wingnuts saying crazy stuff (you can find them on both extremes as well as the middle). Stay curious as to why people have their own conviction, and seek the truth!

The problem with fentanyl is not that people are putting in too much. It’s that it’s extremely potent so it’s too easy to accidentally put too much in.

I derive the majority of my hobby satisfaction from getting stuff done, not enjoying the process of crafting software. We probably enjoy quite different aspects of tinkering! LLMs make me have so much more fun.

It’s not that hard. You would not expect 5 sec drift on phones that can sync time on the web at least once a day or once a week. A basic quartz crystal can keep time to within seconds per month of drift. High quality phones can do the same or better. Also the phone should keep track of system time as epoch time, and convert to local.

Let's say 1.5tok/sec, and that your rig pulls 500 W. That's 10.8 tok/Wh, and assuming you pay, say 15c/kWh means you're paying in the vicinity of $13.8/mtok of output. Looking at R1 output costs on OpenRouter, it's costing about 5-7x as much as what you can pay for third party inference (which also produce tokens ~30x faster).

I hate the ad business model as much as the next person, but this is a pipe dream. Meta had ~$50b in revenue on ads last quarter, and 3.54b “daily active people” whatever that means. That’s in the order of $1/“dap”/week, and there is just absolutely no way any meaningful proportion of their userbase would be paying that much for these apps.

I mean… it’s straight up an Amazon product, not like it’s an IETF standard or something.

NYC effective tax rates exceed 30% (if earning in excess of $100k).

The ministry of state security is not issuing warnings due to an arXiv paper… it’s a different type of “poison”.

That’s actually phenomenal. I love the little archetypes, it honestly mimics HN a bit too well…

It makes no sense to look at global averages for the employment market.

Relevant xkcd content: https://what-if.xkcd.com/29/