14,490 karma · joined December 3, 2007
But on the other hand, the compile time of Rust is really counterproductive. On large, established projects, most prompts I write now spend more time compiling on my machine than they spend outputting tokens. In a way this is great because the tokens are the expensive part, but it does mean that when faster LLMs will come, they will not meaningfully improve iteration speed for me.
I wonder how much of the compile time of Rust is inherent to the type system. There might be room for a language with slower runtime speed (GC?), but as good of a type system as Rust, so long as compile times are much faster. Switching languages has never been easier, anyone have any suggestions? IMO hard requirements are algebraic types and error handling based on them.
MoE models are the path to local models with traditional system architectures, but they are antithetical to what Taalas was doing. If you spent all the money to etch 125B weights into silicon, you'd want to activate them all for each token, instead of only touching 6B. You cannot match the 125B sparse model with a 27B dense one, but you might be able to match it with a 60B or so one.
We might have built them, we sure as hell didn't design them. And no, we do not have a decent idea about how it works.
The SSME uses the same kind of fuel-rich combustion chambers for both preburners. This presents a problem for the oxidizer side because any leaking along the shaft between ox pump and turbine in either direction would be catastrophic. This is resolved by an extremely expensive and complex multi-stage helium-purged rotating seal.
One of the cool parts about most FFSC designs, and the raptor in particular, is that they can afford slight leaking from high pressure to low pressure through any seal in the system and it's broadly safe, and only marginally reduces performance. This allows them to be made much more cheaply.
Nothing but economics.
Inference requires dramatically more memory, meaning it gets radically more expensive, if you are doing backprop alongside. This is why it is not done, despite the obvious advantages. If this doesn't require storing as much data for updates, it might make such systems more feasible.
Seriously, though, making robots.txt have the weight of law would be a massive improvement for the ecosystem in general.
If flight sims were less of a niche category, I think VR would be mainstream. VR just improves them that much. Sadly, the entire flight sim market is probably insufficient to sustain the VR market.
EU opposes the Houthis because they are a horrifying slave state that distributes children as sex slaves to their troops.
EU opposes Russia because it is a horrifying fascist dictatorship that has repeatedly invaded it's neighbors, and constantly talks about it's historical claims to EU territory.
In the NA conflicts it's harder to draw such a clear line, because the conflicts typically have about 15 sides each, but the basic algorithm you can use is generally that EU tries very hard to pick sides based on morality, not geopolitical interests.
unsafe {
mem::transmute::<&T, &mut T>(t) //takes a &T, returns a &mut T
}
in the compiler because this is something that a lot of people think might be safe (I'm running single-threaded, everything would be so much simpler if I just mutated this bit while no-one's looking), but is in fact pretty much always UB of the nasal demons type. But there are more ways to step on this problem than the most apparent way, and the compiler is not able to protect you from all of them.To be clear, just declaring a block to be unsafe does not immediately do anything in Rust, it just allows a set of primitives that are not normally available, so it is possible to use unsafe judiciously without immediately stepping into a million landmines. You just have to be careful and ideally read the docs and the nomicon page for the operations you do, especially if they are very long.
Why would anyone ever type that comment as a response to the comment above?
ESA neither launches or builds rockets. They are an end-user, and they own the launch infrastructure in Kourou. Both these roles will exist long after Ariane is gone.
Arianespace does not build rockets. It's a launch services provider, they purchase the rockets from other companies, handle all the logistics and manage the launchpad. It's in no way married to Ariane 6, and it has in the past and still does sell launches on multiple different rockets from different companies. There will be demand for these services even after Ariane the rocket is gone.
ArianeGroup is the company that builds Ariane 6. They are a complex co-owned project between Airbus and Safran, with workshare distributed between France and Germany. They make other things than the Ariane 6, notably the French SLBM. Eventually the Ariane 6 will go away, because it will be obsolete, but that will not necessarily mean that ArianeGroup will go under, because they have an another subsidiary called ArianeWorks developing a new rocket, called Themis.
The government contracts that made SpaceX were all open bid, where SpaceX bid an better solution than anything that any of the established companies did, at literally half the price. Falcon 9 was a massive gift to the US government, in that it drove the entire industry to halve the prices they extracted from the government. Dragon was just as good.
That they also built a profitable launch company out of it just highlights how much of an outlier SpaceX is. They sold the US government a dollar for fifty cents, made a profit doing it, and ended up driving half the rockets already flying out of the business.
It's often said that you can't fix behavioral problems with technology, but I've found that tooling that strictly enforces rules is really useful.
But also, the model weights are in a single mask rom layer, high up in the metal stack. They could manufacture the die specialized for a given geometry of a model up to that layer, wait for updated weights, and then get the final product out in weeks after they got the weights, instead of many months which is what it would take to redesign the whole chip for the new weights.
But it turns out that when a fascist has nukes, there's not much you can do to convince them to not be fascist.
The greatest failure of the past 20 years was the belief that if we open up trade and grow rich together, rational self-interest will lead to liberalization and peace. Instead, it just gave our enemies more resources to spend on self-destructive imperialist ambitions.
It doesn't really contain anything new or earthshattering, but if you find diving into the history of programming as a discipline interesting, it might be for you.
I would recommend it to anyone who liked the rambling style of "The big OOPs", but imo that one was better. I still liked it.