foota

Hah, I've been looking into something amusingly similar to track mmap syscalls for a process :)

More like RAM producers are providing supplies to the highest bidder, no? If this doesn't peter out supply will normalize at a higher but less insane price eventually.

Capital and availability?

Snowpiercer but with data centers? The breeze would help with cooling!

Eh, that doesn't math out. It's the bandwidth per storage density (or ultimately per price) that matters.

If you have great cost per byte but your bandwidth per byte is bad enough that the price per byte doesn't make up for it then you have an issue.

They've started making hard drives with multiple heads because of this issue, they increased density to the point where it's not useful to continue adding density if it doesn't come with more bandwdith.

The cost issues they're seeing (at least from what they've stated) are from users, not internally. Basically, it takes either $5 or $6.25 (depending on 5m or 1h ttl) to re-ingest a 1M context length conversation into cache for opus 4.6, that's obviously a very high cost, and users are unhappy with it.

I think 400k as a default seems about right from my experience, but just having the ability to control it would be nice. For the record, even just making a tool call at 1M tokens costs 50 cents (which could be amortized if multiple calls are made in a round), so imo costs are just too high at long context lengths for them to be the default.

Hey -- I have 0 PHDs so take this with a grain of salt :)

I had thought for a while about a way to store data that makes use of an idea that I had for sub-diffraction limited imaging inspired by STED microscopy.

First an overview of STED. You have a "donut" shaped laser (or toroidal laser) that is fired on a sample. This laser has an inner hole that is below the diffraction limit. This laser is used to deplete the ability of the sample to fluoresce, and then immediately after a second laser is shone on the same spot. The parts of the sample depleted by the donut laser don't fluoresce and so you only see the donut hole fluoresce. This allows you to image below the diffraction limit.

My idea was to apply this along with a layer in the material that exhibits sum frequency generation (SFG). The idea is that you can shine the donut laser with frequency A and a gaussian laser with frequency B at the same spot. When they interact in the SFG material you get some third frequency C as a result of SFG. Then, below that material would be a material that doesn't transmit frequencies C and A.

Then what you'd be left with after the light shines through those two layers is some amount of light at frequency B. The brightness inside the hole and outside of the hole would depend on how much of the light from frequency B converts into frequency C. Sum frequency generation is a very inefficient process, with only some tiny portion of the light participating, but my thinking is that if laser B is significantly less bright than laser A, then what will happen is that most of the light from laser B will participate in sum frequency generation where it mixes with laser A, and that you'll be left with only a tiny bit of laser A outside of the hole, so that you get a nice contrast ratio for the light at frequency A between the hole and the surroundings that then allow you to image whatever is below these layers below the diffraction limit.

In my idea the final layer is some kind of optical storage medium that can be be read/written by the laser below the diffraction limit. Obviously aiming this would be hard :) My idea was that it would be some kind of spinning disk, but I never really got to that point.

Late to reply, but.. yes, hence my reply that it would need to be integrated all the way down the stack.

But also, LLMs (or their current implementation) rely heavily on print caching for efficiency, without this costs are much higher. You can do neat tricks with it, but generally you're limited to playing with the end of the context to avoid breaking things.

I think some agents do add small context snippets to the end of the conversation that get used by the agent. You can do things like: conversation messages + context snippets + new message and then once the agent replies make the next turn conversation + new message + reply + ... This breaks the cache only for the latest message (not too bad) and let's you give the model current up to date information. This is how stuff like the "mode" or "what time is it now" are handled I believe.

I would go for out of touch, not cynical. A lot of people really think AI is the devil.

Some others mentioned pijul, but I will put in my two cents about it. I have been looking to make use of it because it seems really nice for working with an agent. Essentially you get patches that are independently and can be applied anywhere instead of commits. If there is ambiguity applying a patch then you have to resolve it, but that resolution is sort of a first class object.

Not really, because the LLM loop doesn't have the ability to get updates from the agent live. It would have to somehow be integrated all the way down the stack.

And they are doing better.

So.. it sounds like they're doing a lot better to me? 19 cases in the fall, 4 between the recall in Novemberish and Jan, and 1 between them and now that occurred in Jaunary?

Also lol at this quote in the article "Six vehicles passed the school bus while it was stopped, the agency said. It is still investigating." What it doesn't note is that the other 5 seem to have been human driven passenger vehicles. From the NTSB report: "located in Novi, Michigan, replied “No” to the prompt. The ADS-equipped vehicle then resumed travel and passed the school bus while its stop arms were still extended. A passenger vehicle following the ADS-equipped vehicle similarly passed the school bus. In total, six vehicles passed the school bus while it was stopped. A crash did not occur.", so it sounds to me like 4 people passed it, waymo was like wtf I'm pretty sure that's a stopped bus, a human incorrectly identified it as not a bus, waymo passed it, and then one more person passed after the waymo.

I don't think this checks out. Would the model do the same thing when presented with the exact same inputs? Yes. Is it more likely to do the same thing at the same intersection? Probably. But if you repeat a similar setup somewhere it might not. Bad behavior still exists and should be fixed, but it doesn't mean they're bad drivers in general.

Somewhat unrelated, but if I have downloaded node modules in the last couple days, how should I best figure out if I've been hacked?

Hah, I did the same exact thing and came here to say that :) I was looking at wiring diagrams and telling myself I could wire up some arduino circuits for it but gave up when when I realized I could just press the button!

edit: although mine was an ancient system from the early 90s. It was just replaced with a modern system a couple months ago. At my previous apartment I had wanted to set up a system that would allow either my then partner or I to activate the callbox and have it set for a VOIP number since we could only put one number on the box.

I added a super cheap and bad embedding database in a project that allows the agent to call a tool for searching all the content it's built, it seems to work pretty well! This way the agent doesn't need to call a bunch of list tools (which I was worried would introduce lost of data to the context), and can find things based on fuzzy search.

It would be fascinating if someday you could implement parts of the browser using WASM modules.

I realize this comes up every so often, but I was just looking at this the other day :) A related idea is wang tiles, which are a way to construct a tileset such that you can place them without ever running into a contradiction.

Could it be because nuclear is highly centralized? I would expect that something like solar/wind power would be better for decentralization (in a war).

Even if you don't blow up a nuclear plant, it seems like cutting the power from one would be relatively easy.

Copyright for me not for thee? :) That's a good point though. Maybe they could round trip things? E.g., use the model trained only on internal content to generate training data (which you could probably do some kind of screening to remove anything you don't want leaking) and then train a new model off just that?

> I don't think this would qualify as clean room (the Library was involved in learning to generate programs as a whole)

Hm... I mean this is really one for the lawyers, but IMO you would likely successfully be able to argue that the marginal knowledge of general coding from a particular library is likely close to nil.

The hard part here imo would be convincingly arguing that you can wipe out knowledge of the library from the training set, whether through fine tuning or trying to exclude it from the dataset.

> But what about training without having seen any human written program? Coul a model learn from randomly generated programs?

I think the answer at this point is definitely no, but maybe someday. I think it's a more interesting question for art since it's more subjective, if we eventually get to a point where a machine can self-teach itself art from nothing... first of all how, but second of all it would be interesting to see the reaction from people opposed to AI art on the basis of it training off of artists.

Honestly given all I've seen models do, I wouldn't be too surprised if you could somehow distill a (very bad) image generation model off of just an LLM. In a sense this is the end goal of the pelican riding a bicycle (somewhat tongue in cheek), if the LLM can learn to draw anything with SVGs without ever getting visual inputs then it would be very interesting :)

I don't know how far it would get, but I imagine that a FAANG will be able to get the farthest here by virtue of having mountains of corporate data that they have complete ownership over.

I think the more interesting question here would be if someone could fine tune an open weight model to remove knowledge of a particular library (not sure how you'd do that, but maybe possible?) and then try to get it to produce a clean room implementation.

Ah yes, open API, famously a user accessible means of accessing a website.

Well that aged poorly.

Hm.. but that's just tool calling, right? MCP is just that there's a lot more tools than normal.

Does tool calling in general bloat context, or is there something particular about MCP?

One thing I have read recently is that when you make a tool call it forces the model to go back to the agent. The effect of this is that the agent then has to make another request with all of the prompt (include past messages), these will be "cached" tokens, but they're still expensive. So if you can amortize the tool calls by having the model either do many at once or chaining them with something like bash you'll be better off.

I suspect this might be why cursor likes writing bash scripts so much, simple shell commands are going to be very token heavy because of the frequency of interrupts.

That's... weird? Maybe it's blocked in your country? The link opens just fine for me.

Those were tested numbers, not advertised though. I don't see how you'd get a drop from 200 to 120 miles, that's a 40% drop. Maybe in a gasoline powered car, but EVs can regeneratively break, so I don't think it'd make that much of a difference.

Reading some more, there are a handful of different ratings. the old European one: NEDC, the new European one: WLTP, the US EPA, and China's CLTC.

Generally the ratings from lowest to highest go EPA, WLTP, NEDC, then CLTC. The EPA rating is just a tad high I've read when you look at fast highway driving (e.g., 75 MPH), but should be within ballpark range.

I think you're under estimating the range of modern EVs.

Maybe most EVs in the wild, but no way for EVs being sold today. There are only 5 cars on this list below 200: https://www.edmunds.com/car-news/electric-car-range-and-cons..., and more than half above 300.