cafxx

Filed a CL for this, hopefully it gets merged ~soon.

https://go-review.googlesource.com/c/go/+/729920

That's not how it's implemented (it returns false if you're inside a Do() on a unsupported platform), although I agree the wording should be clearer.

I find this example mildly infuriating/amusing:

    func Encrypt(message []byte) ([]byte, error) {
        var ciphertext []byte
        var encErr error
    
        secret.Do(func() {
            // ...
        })
        
        return ciphertext, encErr
    }

As that suggests that somehow for PFS it is critical that the ephemeral key (not the long-term one) is zeroed out, while the plaintext message - i.e. the thing that in the example we allegedly want secrecy for - is totally fine to be outside of the whole `secret` machinery, and remain in memory potentially "forever".

I get that the example is simplified (because what it should actually do is protect the long-term key, not the ephemeral one)... so, yeah, it's just a bad example.

The point is that no-one is thinking to add knobs, or allow alternative GCs.

> GOGC was introduced in Go 1.5

yes, that's quite literally what I meant by "GOGC has always been there". 1.5 was released 10 years ago, just 3 years after 1.0.

So to summarize: there is one knob (that has been there from basically the beginning), plus a second one (if you squint hard enough: GOMAXMEM), and absolutely no plans to add further ones, or to add alternative GCs.

Your previous message was referring to Go "getting" knobs, but GOGC has always been there.

The older GC algorithm won't stay, IIRC the plan is for it to be removed in 1.27 (it's kept for now just to give a fallback in case of bugs in the first release).

I still would be wary, even in that case. Go does not guarantee that the address of an allocation won't change over the lifetime of the allocation (although current implementations do not make use of this).

If you really store just references to the same arena, better to use an offset from the start of the arena. Then it does not matter whether allocations are moved around.

Not sure what you are referring to. There are no knobs involved in the things I mentioned (aside from the one to enable the experiment, but that's just temporary until the experiment completes - one way or the other).

> If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type.

A word of caution. If you do this and then you store pointers into that slice, the GC will likely not see them (as if you were just storing them as `uintptr`s)

There's a bunch of activity ongoing to make things better for memory allocation/collection in Go. GreenTeaGC is one that has already landed, but there are others like the RuntimeFree experiment that aims at progressively reduce the amount of garbage generated by enabling safe reuse of heap allocations, as well as other plans to move more allocations to the stack.

Somehow concluding that "By killing Memory Arenas, Go effectively capped its performance ceiling" seems quite misguided.

I think that the idea of not initiating writeback immediately derives mostly from the days of spinning rust, where read latencies would be noticeably impacted if you initiated writeback too aggressively: reads, contrary to writes, are synchronous by default, and spinning rust rarely allowed high (by modern standards) IOPS, so it made a lot of sense to buffer writes as much as possible to minimize the number of I/O operations spent on writes, as this would leave as many of those IOPS as possible available for reads.

This is probably much less of a concern today, as NVMe drives - beside having many orders of magnitude higher IOPS capacity - also have (at least on paper) much better hardware support for high I/O concurrency. It may still make sense, even today, if your hardware (or stack) limits IOPS.

Would be nice also if they fixed the ubiquitous "network errors" that happen approximately every single time...

Not to be the devil's advocate here, but almost certainly it can be the case that data was used to define heuristics (potentially using automated statistical methods) that a engineer then formalized as code. Without that data that specific heuristic wouldn't exist, at least very likely not in that form. Yet that data does not have to be included in any open source release. And obviously you as a recipient of the release can modify the heuristic (or at least, you can modify the version that was codified), but you can not reconstruct it from the original data.

I know my example is not exactly what is happening here, but the two sound pretty affine to me and there seem to be a fairly blurry line dividing the two... so I would argue that where "this must be included in a open source release" ends and "this does not need to be included in a open source release" starts is not always so cut and dry.

(A variant of this, that happens fairly frequently, is when you find a commit that says something along the lines of "this change was made because it made an internal, non-public workload X% faster"; if the data that measurement is based upon did not exist, or if the workload itself didn't exist, that change wouldn't have been made, or maybe it would have been made differently... so again you end up with logic due to data that is not in the open source release)

If we want to go one step further, we could even ask: what about static assets (e.g. images, photographs, other datasets, etc.) included in a open-source release... maybe I'm dead wrong here, but I have never heard that such assets must themselves be "reproducible from source" (what even is, in this context, the "source" of a photograph?).

That being said, I sure wish the training data used for all of these models was available to everyone...

...or maybe Windows should just offer an API for marking a file for deletion once it's not in use anymore (I understand unlink semantics may not be possible, but that's not what my suggestion above is saying)

Or you could simply invoke a GC periodically (or every N times a key is removed from the map, or similar schemes).

Another simple way, if we don't like the idea of triggering GCs manually,is to allocate the same buffer both off-heap and on-heap: use the off-heap one for actual key storage, and the on-heap one just to generate heap memory pressure.

Why not just allocating the blobs off-heap? (That is something you probably want to do anyway if it's cryptographic material, to avoid being at the mercy of the GC leaving copies around)

ByteBuffer.allocateDirect should do that IIRC. This allows you to use the standard ConcurrentHashMap while being able to get a stable pointer for use by the rust logic.

'Oumuamua was closer to 8:1 https://en.wikipedia.org/wiki/%CA%BBOumuamua

In the example you linked it's comparing two threads reading from the same or separate cache lines, no? If so, that's not really the point I was referring to (as the two variables I mentioned as example are accessed by a single thread, not by two threads).

A point that AFAICT is not articulated in the article is why the two cached fields should be in their own dedicated cache line (e.g. why readIdxCached_ can not share the cache line with writeIdx_).

Yes, that's the whole point. I did not claim you can control every single module in the QR. I just said that you can get different QR codes for the same semantic content.

Yes, that's what I meant.

Sorry, yes, I should have clarified I meant "no-one has done it in the current batch of diffusion-based experiments". As you point out it has been done multiple times before, e.g. https://research.swtch.com/qart (online demo https://research.swtch.com/qr/draw/).

Sure, but as I mentioned it has absolutely nothing to do with the CRC calculation, that is what you claimed.

Also not sure the diffusion model has been taught yet that it could in some cases choose to deliberately ignore a (small) fraction of some of the elements/blocks, if it helps with composition.

Or even better just wire up a QR decoder to the loop and automatically reject images that don't decode correctly, and let the model sort out how far it can go.

None of what I wrote impacts the CRC calculation.

The funny thing is that no-one AFAICT has realized that the same content can be encoded in different-looking QR codes. Beside the obvious (different error-correction levels), the content itself can be changed while maintaining its semantic meaning (e.g. "https://example.com/foo", "HTTPS://EXAMPLE.COM/foo", or "HtTpS://eXaMpLe.CoM/foo" are all semantically identical) and even the QR encoding itself can be tweaked (e.g. by changing the version and mask, see the demo on https://www.nayuki.io/page/qr-code-generator-library). Each combination would yield a different-looking QR code that would encode the same meaning, and it could therefore allow the diffusion models even greater freedom.

I'm sure somebody will get to this soon.

For complex SELECT queries, I do:

WITH cte1 AS ( /* whatever / ), cte2 AS ( / you can refer to cte1 here / ), cte3 AS ( / you can refer to cte1 and cte2 here */ ), ...

If I need access into multiple statements that I can't refactor into a single one, explicitly using temporary tables is a viable option (some engines implement CTEs and other operations using implicit temporary tables anyway). Otherwise, yes, views.

Yup, common occurrence also here in Tokyo. If you are not Japanese you can expect quite a few rejections, as well as having to pay extra "guarantor" fees, and in general higher than average rent.

And, mind you, I'm probably among the lucky ones... if you're from other Asian countries you can probably expect even worse.

Not sure I buy the argument.

By that logic, Apple should only allow phone calls from callers they consider trusted, because any call originating from a non-trusted phone number may be a scam phone call, and someone may fall for it.

Do scam phone call exists? Yes. Do people fall for it? Yes. Does apple block all untrusted phone calls, with no opt-out, as a result of this? (AFAIK) No.

(I could also make the same or similar arguments for web pages, music, podcasts, books, movies, or even apps on their other OS and so on... but phone calls seemed the simplest one to present succinctly.)

> no one would tell a senior engineer just a single sentence "please implement mult command"

It definitely happens (at least it did to me) to be given programming tasks without context. Maybe my comment wasn't crystal clear on this, but I was not providing advice for interviews to the specific company mentioned in the OP, rather general advice for senior roles. In this case the claim that "no one would tell ..." is hard to maintain.

> this would not affect my evaluation of this stage one way or another.

Same point applies here. We all know different interviewers have different ways to evaluate. Therefore it's helpful to cover your bases. Hence my comment to not skip that part. For some interviewers it won't matter, but for others it will be a rather important factor.

> and tell that [...] it is not a viable solution

Just for the sake of discussion: if I were the interviewee I would then definitely ask why it's not viable, and if I did not get a satisfactory answer it would for sure affect my impression of the process and, by extension, of the company I'm interviewing for. Not claiming that every interviewee is like this, just that this is the case for some (I am simply not pretending to be an exception on this).