MauranKilom

But you posted this to a site that is literally called Hacker News... To be clear, I am not supporting any attempt at undermining your project, but people are pointing out to you that your results will be called into question if your only defence against "hacking" is "I hope people don't figure out how to do that".

Because no message has ever had its meaning twisted by overzealous spellcheck. No ducking way.

I don't have much to contribute, but the name "Rephsis" seems borderline unpronounceable to me (maybe that's why the empire is crumbling? :P). I don't know whether you had some inspiration behind it, but to me it has a strong "I combined letters until I had a word that looked new" vibe.

The screenshots in the Steam page do look impressive though! As an outsider, I could mistake this for a recent Civ game. Congrats on making it here!

Do they? I think it's mainly about how much energy you are willing to invest into making rational decisions over emotional/"gut" ones. I mean, in the end, it probably doesn't matter which kind of shaving cream you buy, so it's easiest to go with the one that (due to e.g. ad-created familiarity bias, or just "niceness" of visual presentation) you have the best feeling about.

In other words, you introduce an invalid state to every object and make constructing objects a lot more cumbersome. The first is the exact opposite of the (imo highly desirable) "make invalid states unrepresentable" principle, and the second is also a pretty extreme cost to productivity. I wouldn't say this is never worth it, but it's a very high price to pay.

Thanks for this, it provides an intriguing approach to thinking about transformers (or predictors in general)!

For extracting the fractal from the residual stream, did I understand it correctly as follows: You repeatedly sample the transformer, each time recording the actual internal state of the HMM and the (higher-dimensional) residual stream. Then you perform a linear regression to obtain a projection matrix from residual stream vector to HMM state vector.

If so, then doesn't that risk "finding" something that isn't necessarily there? While I think/agree that the structure of the mixed state representation is obviously represented in the transformer in this case, in general I don't think that, strictly speaking, finding a particular kind of structure when projecting transformer "state" into known world "state" is proof that the transformer models the world states and its beliefs about the world states in that same way. Think "correlation is not causation". Maybe this is splitting hairs (because, in effect, what does it matter how exactly the transformer "works" when we can "see" the expected mixed state structure inside it), but I am slightly concerned that we introduce our knowledge of the world through the linear regression.

Like, consider a world with two indistinguishable states (among others), and a predictor that (noisily) models those two with just one equivalent state. Wouldn't the linear regression/projection of predictor states into world states risk "discovering" the two world states in the predictor, which don't actually exist there in isolation at all?

Again, I'm not doubting the conclusions/explanation of how, in the article, that transformer models that world. I am only hypothesizing that, for more complex examples with more "messy" worlds, looking for the best projection into the known world states is dangerous: It presupposes that the world states form a true subspace of the residual stream states (or equivalent).

Would be happy to be convinced that there is something deeper that I'm missing here. :)

> I think that they should not have allowed to overload = and , operators

Comma operator I would agree is really only used for shenanigans. But operator= is as crucial as the copy/move constructors for ownership, so I'm not sure how you picture this...

> apparently in C++ a structure cannot really have zero length

Yes it can: https://en.cppreference.com/w/cpp/language/attributes/no_uni...

> I think it should be possible for your own structure or union type to overload any operators that are not already defined

I see that you are trying to solve the problem of "when reading this code, I don't know if these are the built-in operators or custom ones". I would be interested in learning about situations in which this was a problem for you, because I have not made that experience myself. From my perspective, you need to know the types of x and y to know in the first place which operations are possible - default or custom - so this rule does not really buy you much. But again, I don't think I understand your problem well enough.

FWIW, you can define a custom operator-> and operator* (they have to return pointers, or types that themselves have operator->/operator* which are then applied recursively). I am not convinced that mixing pointer-related semantics into e.g. object assignment is a sufficiently elegant solution from a conceptual perspective though.

As a younger programmer, I used to listen to people hating about everything C++ does, offers, or wants to be. Specifically https://yosefk.com/c++fqa/ [1]. Then I tasted the RAII (probably up there with Dynamic Programming for "horribly named but super powerful paradigms") koolaid, and threw my caution to the wind. It's been a pleasant ride since, even if I've been living in a somewhat comfy C++ environment with few third-party integration woes and a reasonable compiler upgrade pace.

Sure, I agree, you should carefully evaluate which language best serves your needs, and what tools in a language you should use or stay away from. But some game dev ranting about which C++ features they consider useless (without even articulating their background or constraints) is worthless imo. Especially if this is the credit they give to their name:

> After over 20 years working with C/C++ I finally got clear idea how header files need to be organized.

[1] Yes, that FAQ parody is now very outdated, seeing as it precedes even C++11. I can believe that it was on-point at the time of writing, but C++ grew up since then.

Regardless of all the "he said, she said", libel or not libel, email response times, GMP, threats and whatever drama:

Is there anyone who is seriously contesting the "it is intended to cure/prevent a disease, therefore it is a drug, therefore it needs FDA approval to be sold legally" line of reasoning?

More humoristically: https://xkcd.com/2475/ and https://xkcd.com/2530/

Less humoristically: https://en.wikipedia.org/wiki/Thalidomide

FWIW, "number of flights you take" also drowns out your eating habits in environmental impact. Compared to how much they cost, flights have stupid CO2 equivalents.

However, I don't know why you are comparing a single billionaire vs a single X kid household. Like, the number of each (or even of private jets) are not even _remotely_ in the same ballpark. Which is why "number of kids" is not at all a strange place to focus on environmental impact, but "billionaire lifestyle choices" is.

Don't you think that we would be able to observe some anisotropy in the data if that were the case? (Note: I have no clue, but it seems unlikely...)

If you (or anyone reading it) include yourself in that "we", you can help with taking that step by writing and/or donating to the nonprofits that are driving these cases. Like the ICCL that posted this article, or e.g. NYOB (https://noyb.eu/).

As a notable example, the MP3 format was hand-tuned to vocals based on "Tom's Diner" (i.e. a female voice). It has been accused of being biased towards female vocals as a result.

I find this post very highly accurate in my personal experience (minus the "do it on a Saturday" part). And this is in a company whose departees often tend to eventually return precisely because it's actually not among the "horrible companies". Neither would I consider it "bleak and depressing", unless you actually believed those falsehoods.

I would like to also highlight this article by Joel Spolsky: https://www.joelonsoftware.com/2006/04/11/the-development-ab... (It's concerned with the everything else in a company, but it applies equally to the senior vs junior responsibilities imo).

If you are under the impression that "writing good code" is the be-all-end-all of being a software engineer, and that being a senior means you get to enjoy it more, then you're probably working in a company that is really good at maintaining this part of the abstraction layer.

That's because the more senior you get, the more you inevitably get to be part of the system that allows more junior devs to keep their mind on writing good code. That means dealing with the meetings, timelines, uncertainties, bug triage, maintenance, documentation, compliance and customer/expectation management. So they don't have to.

> even humans hallucinate massively

Simpler example: Dreams.

A fair comparison (on a technical level) to GPT-4 RAG would be a doctor in a relevant field who also has internet access. I think this would be indeed interesting to compare to assess the resulting quality of care, so to speak!

(The other models being only partially able to source good references is unsurprising/"unfair" on a technical level, but that's not relevant for assessing their safety.)

Agreed - I am a bit troubled by the maintainer stance here, even if they have to wade through a bunch of AI nonsense every day and this report is fluffed up way too hard.

"We assume 64 bit overflow is not going to happen because nobody can store that many bytes" could be valid if the existence of those bytes was required for reaching this code. But if user input can lead to UB being triggered here, fixing the code is indeed prudent, even if everyone were fully convinced that current compilers are not outsmarting themselves.

The OpenAI announcement mentions being able to provide an image to start the video generation process from. That sounds to me like it will actually be incredibly easy to anchor the video generation to some consistent visual - unlike all the text-based stable diffusion so far. (Yes, there is img2img, but that is not crossing the boundary into a different medium like Sora).

FYI on https://projects.blender.org/blender/blender/pulls/116089: `inline` first and foremost affects the One Definition Rule. Very briefly put: If you have a function defined (!) in a header (and call it in different .cc files), you will get linker errors unless you mark it as inline.

The effect on the actual inlining optimization could be anything between "none", "the compiler might weigh it differently purely due to linkage" and "it is actually taken as a hint", plus a similar set of considerations for link-time optimization. A much stronger inlining signal to the compiler (specifically for functions that are only used in the file they are defined in) is to define them in an anonymous namespace.

The original intent behind marking the function `inline` could reasonably have been an attempt at actually achieving inlining. A measurable benefit is not obvious (but also not impossible).

At least the image scaling does not appear to be in linear? Going by the notes in [0]:

    Gimp and stb_image_resize results are subtly different from others, even for Bilinear. Looks like they do convert source sRGB image to linear, do filtering there, and convert back to sRGB.

You can definitely see that as a problem with bright lines (e.g. collar highlight in einar*.png) being darker in the upscaled Blender versions.

[0]: https://aras-p.info/img/misc/upsample_filter_comp_2024/

Arguably, minimal frequency + phase adjusted so the falling sinusoid has only negative values in the playing area is "more optimal" (more room for error).

Two big question marks after reading this and the linked home page (partially pointed out in other comments):

- If there's a flaw in your application code that causes a crash (as is the motivating example in the essay), then restoring the entire program into the state it was in just before the crash happened would just cause it to crash again ad infinitum. Sure, this model helps against "my VM instance got preempted", but that's a pretty different category of "crash" (and also notably unrelated to supervision trees).

- "External" state (like an API endpoint being down/returning gibberish) can be part of the reason why your program got into a bad state. In fact, that's disproportionately likely, since external "weirdness" is comparatively hard to cover exhaustively in tests. In such a situation, the suggested computation model would never be able to recover even when restarted, because it would forever retain the (bad) API response. Effectively, this is just caching all the non-pure effects of your computation, and we all know about cache invalidation being a hard problem...

I hear it's one of the factors regarding the location of Intel's upcoming German fab.

That's an interesting project, going to keep an eye on it! I have a hobby/DIY project involving sending several hundred UDP packets per second to various ESP32s across the house (driving LED strips live based on music playing on my PC). Much to my chagrin, the ESP32s tend to choke/get stuck quite frequently at higher packet volumes, and I've not found a good way to debug the issue further. See [1] below. While I haven't crossed the bridge of leaving the comfort of the Arduino IDE for flashing the ESP32s, trying out/beta-testing an open WiFi stack (once available) that I know I'll have a chance to thoroughly trace/debug would be very enticing.

[1] I did find out so far that my WiFi setup occasionally "clumps" packets, causing like 10 packets to hit a given ESP32 at an instant (instead of a few ms apart) - not great, but should not be disastrous. However, this seems to cause the ESP32 WiFi stack to just slow to a crawl: It responds to pings much slower (like, in 100+ ms range) (to my surprise it actually responds to PING requests out of the box in the first place...) and/or doesn't really process any more packets in general if I continue sending at the same rate as normal. But backing off on the packet stream usually gets it back on track, strangely enough. This also happens if I do nothing in the main loop except clear packets as they come in, so it's not in my code.

My wireless headphones get an unbearable amount of pops and cracks when I'm wearing them close to my (running) microwave. But since I wear them virtually always, I am in effect virtually never next to my microwave when it's on :)

Most recently, I was using bash.org as my go-to non-HTTPS site for captive portal purposes (back when HTTPS Everywhere was a useful extensions). But the built-into-Firefox HTTPS only treatment handles captive portals gracefully already, so I didn't actually visit in a while.

Goodbye!

I am well aware of the techniques you mention, and intimitely familiar with inspecting include trees under the light of "does it make sense for this code to be recompiled if this code changes?", particularly across module boundaries. To clarify, I'm talking about cases of multiplicative object code growth from layering templated abstractions (as per my reply) causing a single object file with a clear singular purpose to grow beyond linker limits. Not because it does too many things or needlessly compiles in a whole bunch of mistakenly-in-interface-headers templates from other modules all over the place; those would be easy to fix. But this case is not solved by "move code into submodules" (there is nothing to further sub-modularize) and "move templates out of headers" (that translation unit uses and instantiates precisely the templates that are reasonable to involve). Yes, those are the easiest fat to trim from compilation times, I know and agree, but I'm trying to point out to you that not every template in a header is a mistake, and that object code size can grow beyond expectations even if those basic techniques have been exhausted. (And to be clear, we of course use incremental builds, but that's not even relevant to object size limits).

Again, I am not objecting to the general advice you give, but maybe you can take a step back and appreciate that not every code base and C++ use case falls into the range of what you have seen and interacted with so far. There is no doubt that a vast majority of C++ code bases would benefit immensely from improving compile-time encapsulation, but that is not the be-all-end-all of solving long compilation times, and it does not give you grounds for dismissing concerns of people who have done that and still face different problems.

> You instantiate what you need, you move your template code into submodules and out of interface headers, and you're set.

You seem to have completely missed my point. I implore you to consider for a second that I might be familiar with what you are trying to tell me, and that there are indeed complications beyond the basic techniques you advocate for. Let me illustrate with an example.

https://godbolt.org/z/5j43WrM68

Here we have a very simple class that uses a few std library types. It is a template, but say that we know that it is only valid to use it with the shown basic arithmetic types (note 1). The class implementation and according explicit template instantiation definitions have therefore been moved to a separate TU (not shown) and we only have the shown code in the header. This is a simple application of those "basic C++ techniques" you mention.

What happens to the compilation time if we enable the explicit template instantiation declarations in the header? Measuring on godbolt is noisy, but by repeatedly changing the source file (e.g. just adding spaces) you can quickly get a bunch of measurements. The lowest value of 10-20 measurements is going to be a reasonable proxy for real-world compilation performance. And if you perform these measurements with and without the ETIDeclarations enabled, you will notice that they cause this short piece of code to take 100-200 ms longer to compile. Mind you, these are explicit template instantiation declarations - they tell the compiler that it does not have to generate code for these template instantiations. However, they do still force the compiler to instantiate all the declarations of all involved transitive templates. Explicit-template-instantiation-declarating X<int> requires the compiler to (internally, in some abstract way) note down the existence of every last std::vector<int>::const_reverse_iterator::operator-=(size_t) and so on (note 2). That's a lot of nested templates that it needs to (abstractly) "declare", even if it never has to actually generate code for them, which explains why the mere presence of the ETIDeclarations slows down compilation measurably - in every single TU that includes this header!

Note 1: It is, in my experience, relatively rare that the set of valid template arguments is closed and not open - in a way, an open set of permissible types is the whole point of writing templates. And such a templates intentionally being provided across a module boundary is also all but rare for lower level libraries in a bigger code base.

Note 2: It's easy to underestimate the amount of template nesting in e.g. the standard library too. Here's a single typedef in std::vector (with two further levels of typedefs expanded):

  using const_reverse_iterator = _STD reverse_iterator<_Vector_const_iterator<_Vector_val<conditional_t<_Is_simple_alloc_v<_Alty>, _Simple_types<_Ty>,
        _Vec_iter_types<_Ty, size_type, difference_type, pointer, const_pointer, _Ty&, const _Ty&>>>>>;

I agree with the thrust of your argument that there are a lot of compile-time-encapsulation tools you can employ to reduce build times.

However, I will still challenge you on your build time/object size claim. Take a codebase structured around lazy tasks (i.e. every function is a coroutine), factor in runtime parallelism/SIMD dispatch, multiply with loop abstractions (think std::ranges) and you get frighteningly close to linker limits in no time (which, of course, also implies minutes of compilation per source file). Each of those pieces is templates through and through, and there's no explicit template instantiation way out of any of them.

Also, I have to (sadly) point out that explicit template instantiation declarations only take you so far. Yes, they reduce compiler time spent in codegen, but also cause the compiler to create all required transitive (!) declarations in every TU, which will quickly offset those gains when talking about template types that have lots of small member functions.

The part that makes higher error correction even more pointless is that errors can only be tolerated well in certain parts of the QR code. If the position markers are damaged, a scanner likely won't even recognize that there is a QR code visible, which obviates any error correction. The article mentions this in the very last sentence/caveat.

It feels a bit like the error correction exists mainly so you can embed fancy logos in your QR code... Avoiding damage to some parts but not others seems unaligned with real world scenarios...

Can you comment on the part starting with

> What if an open source project is used directly by consumers, and causes them harm? The public policy is clear: they must be compensated.

It's expressly not clear what the implications here are, according to the article.