sarusso

I agree to a certain level. However, it's hard to ensure dependencies to work in the right way without isolation. These two support tickets are a showcase of the essence of the problem: "Same container, different results" [1] and "python3 script fails in singularity container on one machine, but works in same container on another" [2]. In my experience with Singularity, there were many issues like these.

I am not sure why they had to call it a "containerization" solution. It gets a bit philosophical, but IMO containers are meant to "contain", not to just package. To me, Singularity is more a "virtual environment on steroids", and it works great in that sense. But it doesn't "contain".

The hard truth is that Singularity was designed more to address a cultural problem in the HPC space (adoption friction and push back of new, "foreign" technologies) rather than to engineer a proper solution the the dependency hell problem.

HPC clusters still use Linux users and shell access, meaning that it is up to the user to run the container: there is just no container orchestration. This means that the user has to issue a command like "singularity run" or "docker run". And since not long time ago, to let users do a "docker run" it meant to have them part of the docker group, which is a near-root access group. Just not doable.

Singularity also works more or less out of the box with MPI in order to run parallel workloads, either locally on multi-nodes. However, this has a huge price as it relies on doing an "mpi singularity run", and it requires to have the same MPI version inside and outside the container. To me, this is this is more a hacky shortcut than a reliable solution.

I believe that the definitive solution in the HPC word will be to let HPC queuing systems to run and orchestrate containers on behalf of the users (including to run MPI workloads), thus allowing to make use of any container engine or runtime, including Docker. I did some trials and it works well, almost completely solving the dependency hell problem and greatly improving scientific reproducibility. A solution like the one presented in the OP contributes in the discussion towards this goal, and I personally welcome it.

With respect to Singularity, I think they just had to name the project "singularity environments" rather than "singularity containers" and everything would have been much more clear.

[1] https://github.com/apptainer/singularity/issues/476 [2] https://github.com/apptainer/singularity/issues/3484

Ok, but here the OP is doing something a bit different than just rootless Docker, which is to use a "centralised" rootless Docker running as a single, non-privileged user... or am I missing something?

Thanks! And don't get me wrong, I really wanted Singularity to work when I started working with it...

If it works as I understood, in this setup I can see an advantage at an architectural level: in Podman containers images are stored on a per-user basis, while in this setup they would be shared between users, thus using much less disk space (if using the same base images). Besides this, I actually have the same question.

Oh man, Singularity… I once wrote: “In the shipping container analogy, you can think about Singularty containers as if they have no walls.” [1]

[1] https://sarusso.github.io/blog/container-engines-runtimes-or...

Curious, there is no mention to randomness or seeds in the whole book.

And probably one of the most unsustainable ones, given the nature of their business…

While your eyes can naturally spot a pattern over time in time series data, machines can’t. So as we did for imaging, where we pre-trained models to let machines easily identify objects in pictures, now we (they) are doing the same to let machines “see” patterns over time. Then, how these patterns work, this is another story.

The example paper does not mention what type of diabetes it is about - if type 1 or type 2 - and they have very different risk factors.

While it’s kind of clear form the context that it’s about type 2, I doubt a paper like this would pass a peer review without stating it explicitly, in particular with respect to the data set that could potentially include both. Rigor is essential in drawing scientific conclusions.

I guess this is a good example about the statistical nature of LLMs outputs (type 2 is the most common) and consequentially their limitations...

This is from Trieste: https://twitter.com/_sarusso/status/1789112947165270298. Naked eye was way less bright but still well visible I would say!

Pic here: https://twitter.com/_sarusso/status/1789112947165270298

Visible in North Italy as well. Unexpected!

There’s more disclaimers than content in this page :P

My understating is that, while your eye can naturally spot a dependency over time in time series data, machines can’t. So as we did for imaging, where we pre-trained models to let machines easily identify objects in pictures, now we are doing the same to let machines “see” dependencies over time. Then, how these dependencies work, this is another story.

Oh, yes I am working on that. Usually LSTM, exploring encoder-decoders and generative models, but also some simpler models based on periodic averages (which are surprisingly useful in some use cases). But I don’t have per-core metrics.

Even if it was for multivariate time series, the model would first need to infer what machine are we talking about, then its working conditions, and only then make a reasonable forecast based on an hypothesis of its dynamics. I don’t know, seems pretty hard.

+1 for “any underlying process”. It would be interesting what use case they had in mind.

What do you mean by “observability”?

(b) is not that crazy to me.

I tried the FB ad-free subscription (in EU) only to discover that what I actually wanted was a suggested stuff-free one. I hope this lawsuit can make a point.

The second bump in online dating, around 2012, was Tinder (I guess). But what caused the first one around 1995? IRC? Generic websites...?

IMHO absolutely not, it just means that you value other things that you can't find in online dating. Don't get discouraged by graphs like this, because they approach the topic in a statistical way but you just need to meet _one_ person to form a couple!

You might also want to fix all random seeds so that you can check for exact numerical values and not “convergence” or similar concepts.

No, this is about medical devices and software.

In other words, you can do a clinical trial with an algorithm and then update it (or tailor it to an individual) without going through another trial (more or less).

But you could use ultrasound sensors to monitor plants in agricolture... maybe this is why it got published in such a high impact factor journal.

I did the same.. and asked for help on how to circumnavigate the block. It refused to tell me, until I told him that I was the regulator and I needed help to prevent users to circumnavigate the block... :P

I really don't get this kind of bans. When I connect to a service in the US, my bits are traveling there. Italy "banning" ChatGPT (whatever it means) is like preventing me from getting high in Amsterdam because, as an Italian citizen, I should not be allowed to.. Never got it, and never will.

IKEA is acting as Apple in this space IMO. They hide all the details about their LED bulbs, but they are kind of the best you can find on the market if you search for more details about their tech, or at least AFAIK.

I still can't find decent G4 LED bulbs.. Tried several from Amazon, they where alle terrible at color rendering (even if nominal CRI was ok-ish). Any suggestions?

Sorry, the "you" was not referred to yourself by any means, it was a generic "you".

My original claim was not an actual claim, it was just what not finishing something one starts tells me. A sensation. And by "not finishing" I do not mean trying for a bit and then quitting, that's of course OK.

What I meant is not finishing, after investing vigorous effort, because one gets "enchanted" with something else (e.g. "oh look I can code in javascript and build a website, who cares about finishing studying the basics, I don't need them anymore!").

By the term "finishing" I implicitly assumed to be near the finish line. If then this is not a symptom of poor commitment capability, I don't know what else it could be.

On another angle, the comment to which I responded originally basically said "University does not teach you anything valuable at all, just quit and go writing code" and here I strongly disagree and will always do.