kinow

Churn of the user base could be playing a role in this, but I think it may not be too significant. In Europe there are multiple universities with HPC masters, which provide new users/devs to HPC. I worked with HPCs in New Zealand, and now I am doing the same in Spain. We hire multiple people from other HPC centers in Germany/UK/Italy, and equally lose people to those sites.

I think the field is actually increasing with AI, digital twins, more industry projects (CFD, oil, models for fisheries, simulations for health diseases, etc.).

I think hpc devs need an extra set of skills that are not so common. Such as parallel file systems, batch schedulers, NUMA, infiniband, and probably some domain-specific knowledge for the apps they will develop. This knowledge is also probably a bit niche, like climate modelling, earthquake simulation, lidar data processing, and so it goes.

And even knowing OpenMP or MPI may not suffice if the site uses older versions or heterogeneous approaches with CUDA, FPGA, etc. Knowing the language and the shared/distributed mem libs help, but if your project needs a new senior dev than it may be a bit hard to find (although popularity of company/HPC, salary, and location also play a role).

Seeing the comments, I think one thing missing is the availability of water. The authors mention it briefly, saying it's not part of the report,

> We focus primarily on temperature rather than precipitation or soil moisture, as water availability can be more readily controlled through human interventions, such as irrigation and paddy field construction.

Traditional rice cultivation normally involves flooding. So I am not sure how much one can rely on irrigation for rice. When I was younger, in Sao Paulo some Japanese immigrants were trying to raise Japanese rice species with flooding when I was young, but they gave up and moved to other crops as the weather in the southern area and Uruguay were better (that's what I was told growing up -- FWIW we had corn, sugarcane, orange until there was a huge pest in Barretos region, bamboo and lots of other fruits and veggies in the farm).

A couple of years ago my company (BSC in Spain) had an internal talk about impact of climate change in European vineyard. I don't remember the article they were talking about, but it was similar to this one: Climate change impacts and adaptations of wine production https://www.nature.com/articles/s43017-024-00521-5

From what I recall, what the researcher explained was that the change in water sources like rivers and lakes is already affecting wineries that are not able to grow certain crops due to the lack of water.

But due to the reduction in water levels, some wineries and types of grapes were not being able to be harvested in specific parts of Italy. The article above mentions something similar,

> Existing producers can adapt to a certain level of warming by changing plant material (varieties and rootstocks), training systems and vineyard management. However, these adaptations might not be enough to maintain economically viable wine production in all areas.

I guess there might be some genetic modification, and other techniques that do not require flooding and use less water. But that will likely affect small/medium producers, as well as and communities that depend on rice cultivation in certain areas.

Even with genetically modified rice, it might not be viable to bring water, or move families to other areas. So higher temperatures and the reduction of area that is suitable for harvesting rice might make genetically modifying rice useful only to a few.

I guess larger producers may be able to afford workarounds but that may increase cost to end users. Which is already a lot higher in Brazil than 10 years ago for normal and for the Japanese rice.

It is common in Python too. Reduces memory and eliminates stack errors in some cases. Althoughin Python I think the developer always needs to implement it and cannot rely on compiler/interpreter to optimize that.

I had fun working with QGIS some years ago, connecting it to GeoServer, mapserver, importing shapefiles, and customizing a few maps. I didn't use as much as the GIS engineers I worked with, but it was definitely a great open source tool.

I had to use ArcGIS too, and while sometimes it performed well, when it didn't it was quite painful to have to deal with the local vendor to implement our features, and troubleshoot bugs in their software.

I think during the pandemic a lot of kiwis returned from overseas. Once it was over they slowly started migrating as the economy wasn't really good.

I did the same also to stay close to my wife's family for a few years before returning.

It is pretty common I'd say, not big news. And living here in Spain, apparently the exact same happens.

Young people normally study and work here. Many choose an Erasmus program or find job that pay 2 or 3 times more, especially in Germany, The Netherlands, Poland. We find it really difficult to hire good developers, especially seniors. Juniors are not too hard.

Cool idea, and agree with this sentence in the final paragraph

> It would be cool to make a similar visualization for RISC-V and compare it with ARM64.

Or even compare a subset of the instruction set to see what's missing on different archs.

I remember learning about IIIE (triple-I-efe) years ago while working with a computer vision researcher that used it to serve images generated with Jenkins pipelines.

Glad to see the project of server I used is still running well. The maintainer was a really nice person to work with too: https://cantaloupe-project.github.io/

It helped me that I was already familiar with some OGC & GIS tiling technologies, as what IIIF is doing is not too different. The image processing layer is different though, as you can zoom in, out, rotate, scale, etc..

There were several JavaScript clients, the one I used was one adopted by a museum, but I cannot recall the name now.

I think it is the first time I hear about a merge of two workflow tools.

I work with workflows and HPC, and the most common is multiple workflow managers in an hpc center, or even within a department. Which is not always necessary (sometimes it feels like just because there is budget and someone skilled they prefer to reinvent than reuse).

I hope this means that trend is stopping.

Another thing I hope I will start seeing more are workflow managers sharing libraries. Most workflow managers need to handle similar tasks like submitting tasks to cloud, slurm, containers. Or even performing graph operations, visualizing graphs.

Yeah, I remember reading about an article on France govt adopting element/matrix. Surprised it didn´t go mainstream in other departments/companies/people.

I am quite happy with my AMD 13 laptop. It replaced my Thinkpad T530 after its ~10 years of service. So far, nearly one year after buying it, everything works well. Ubuntu, Docker containers, clion+pycharm, sometimes Blender and other apps. Plus several tabs on Firefox. Camera and audio are better than my old thinkpad, so i have nothing to complain about yet.

There were issues like configuring Linux (extra monitor, logitech mouse, tablet, some software) but I found everything on Framework forums or googling a bit.

That was enjoyable. I miss the days when I would buy old pieces, or find some in old dumpsters in Sao Paulo and try to use old video cards and memory modules to create little franksteins (a lot cheaper than this, but still fun).

I found interesting to learn there are businesses around converting used servers into desktops. Sounds like a good initiative to avoid some e-waste (assuming the desktops are easy to maintain).

This looks amazing. Took me some time to find the author (https://fabiomaia.eu). Will drop a message to ask if it'd be possible to have a similar tool for Python. I find it useful to teach new engineers how to find the source code in the cpython repository, check if the code is Python or C, and understand what the code does (some times the docs are a bit lacking/confusing for newcomers).

I believe it's a navigation tool, with pointers to important parts of the code. Useful for those that want to learn about the code base but do not know where to get started.

I also think the same. While in Java the stdlib lacks a few functions, long ago Apache Commons became the de-facto complement for the Java stdlib, being replaced/complemented by other libs over time, and eventually even becoming obsolete with newer versions of Java. But I always had the impression that having Apache Software Foundation components (with a good release/security process) helped Java to mitigate a lot of attacks.

Very sad to hear this. I interacted with Gregg a few times in Open Source projects, and he was always very easy to work with, even though he appeared to be quite busy and involved in multiple initiatives.

I haven't adopted uv yet watching to see what will be their move. We recently had to review our use of Anaconda tools due to their changes, then review Qt changes in license. Not looking forward to another license ordeal.

ECMWF is on GitHub, and I think they mirror some work in their BitBucket.

Research in Spain has both GitLab and GitHub from what I could see (my company has 2 internal GitLab, a few orgs on GitHub).

Many projects from EOSC or used in EOSC ptojects and deliverables have GitHub repos.

Many projects I use from a Manchester Uni team from ontologies and metadata and provenance are hosted on GitHub.

CERN, Max Planck, INRIA use GitHub, although not sure if exclusively or if they mirror. But I know some of their international collaboration happens there.

So GitLab is definitely used in Eu institutions. Maybe all. But that does not mean GitHub is not also used primarily or as alternative or mirror by these same institutions.

I think Microsoft servers include GitHub? If so, that'd have a huge negative impact on research and academia in EU, as well as software development (even some web pages using JSresources from GitHub pages directly).

Thanks for the summary and good to know it integrates well with Vue.js!

I buy some arts materials from China, but only simple things that I cannot find in EU or tyat are just the same product re-sold a lot more expensive here. I'd be glad to buy in EU if that's cheaper.

I still buy EU arts materials that are more expensive than Chinese products, but that are (at least supposedly) better tested for toxicity.

I noticed in the past year or two art stores like Casa Piera/Arte Miranda have had more products like watercolor paper and paints from China. I hope new regulations will make sure these are compliant with EY regulations, without raising the price to consumer too much.

In our case we had a junior position in the data team, and advertised it in Linkedin and asked others to spread word to friends/bluesky/etc.

Somehow, that person heard about the position, and was already trying to transition from architecture to IT (had enrolled in an open university course of CS -- online, remote).

We really try to interview as many as we can, give a fair chance to all (ignoring background, gender, etc., focusing on what's needed for the position).

Human resources did the first triage (assessing character/personality [one of those simple psychology assessment tests], level of English, visa/work permit/etc.).

Then the managers for the position (department leader + group leader) reviewed CVs and prepared brief interviews, followed by a final technical interview. That's it.

That person did not have a lot of experience, little visible in GitHub, having only studied IT for a few months, but he was really trying hard. The managers liked his attitude, and believed he was the best fit for the position.

He's now learning Git, Docker, Python, NetCDF, etc. In fact, today we will have a quick chat near the waterfountain/coffee machine as he asked me if we could chat as he had questions about pyproject.toml and building Python packages.

I believe if instead he had showed some projects with Python and NetCDF, and said he was looking into enrolling in a CS degree or take some courses, that could have been a replacement for the CS degree.

So tell your friend to try to choose what s/he prefers (data, programming, performance, DevOps, web, algorithms, compilers, etc.), and try to either create a public portfolio, or study hard so that in an interview s/he can answer questions well, showing some knowledge & interest (e.g. in this case, knowing what's NetCDF is, what's GRIB/BUFR, xarray, etc., even without a lot of hands-on experience could have been an advantage).

There was one last year, but I missed it. I will check on that, but ping me a message if you learn that it's widely available, please. Thanks!

Agreed. Another thing that helps is writing. If you have a blog, it helps tremendously as it's possible to learn a lot from an engineer when the posts contain some thoughts about what that person learned or worked on, some opinions, techniques and tools used, etc.

Even these Linkedin posts are fine, as long as it's not an ad to some tool, or just full of buzzwords.

The kind of "behind the trenches" posts are great too, to explain something that happened in a project, a challenge you had, how you debugged a problem. This saves a lot of time in the interview as I would read it all and use that to discuss with the candidate.

> That said I understand why this is an issue and seeing a (representative) code sample is invaluable.

Yeah, I'd also hate to be in that position where I may be the best candidate for a position but somebody else got it just because they have public projects. But at the end, the managers/HR/technical team/etc. are all trying to minimize risks to the company and projects.

> Sounds like a contradiction. If it's super competitive shouldn't it be easy to find candidates?

I know, it should be the other way around. I asked some locals, and so far some of the best answers I got for this are that the best brains here are abroad, at the Netherlands, Germany, etc.. There's a lot of applicants for every position we advertise (especially if we need someone from DevOps or Web -- not so much for Fortran, Scientific Programming, but that's normal).

But most don't pass the first filter. We try to select those that fill the basic criteria, even if they don't fill all the boxes (it's alright to learn on the job for us), even select some that do not pass to call for a short interview with HR or even with manager, but it's quite hard to find good candidates.

> How many candidates coming from EU companies have the work they do at their company made public like that? I never worked anywhere where this was the case. So what do I do then?

It's quite common for some companies in my current field, earth sciences. Many companies have public GitLab servers, or host their own containers (e.g. Mercator Ocean International, DKRZ, ECMWF, etc.).

As I mentioned in other comments, I'd interview someone that doesn't have public repositories or containers anyway. But in the end, if there are other candidates with good CVs, and interesting projects public on GitHub, etc., or if they collaborated to good Open Source projects (Dask, Xarray, Singularity, Jenkins, Python, fortran, etc.) changes increase.

> Only if you work in FOSS is your work public, but if you work from some bank or any other private company they don't expose their work on GitHub due to IP and legal concerns.

I worked in banks, insurance, credit bureau, telco, and government. In most of these the work was done in private CVS, Subversion, or Git servers.

But in most of these, we used Open Source projects, and normally we were allowed to send contributions back upstream, when we found bugs in numpy, python, etc.. There were some companies where I couldn't contribute, so I can only explain the systems I worked, and the tech stack we had.

> So to me it sounds like you're only selecting those who worked at FOSS projects/enterprises

We hire people without public repositories too. Having worked at FOSS projects/enterprises definitely helps.

It's the same as the degree you have. In the end it may be an advantage depending on the company. I, particularly, do not care much if someone is coming from physics, mathematics, economics, biology, or even architecture (we just hired an architect that enrolled in a CS degree in Portugal to work with data pipelines/NetCDF/xarray/etc.).

As long as the person has the technical knowledge required for the job, and some experience if needed, that's fine by me. But I worked with manager that only hired those coming from CS.

So even those with good public contributions or coming from enterprise/FOSS, acing a technical test, etc., nothing would help you to be selected when applying for that team.

As I replied in the other thread, then we will go through a discussion of what they had internally. Many cases I find the good engineers are able to explain that without too much trouble.

Asking them about decisions in the project, issues, pros and cons of tools, etc., normally clarifies whether they understand things or not.

And as I replied in the other comment too, unfortunately if there is another candidate with very similar profile & good answers, then I'll give my technical assessment to the manager of the position, and it'll be up to the manager and humans resources to choose based on risk to the company, what they can see from public papers/conferences/repositories/etc.

In some cases, even if the work is not public, they can tell us what projects they worked. I work with climate models, so if they mention they don't have anything public, but they were using IFS, FESOM, ICON, SCHISM, or another model that I know/work with, then I'd direct questions on that model, to check what part of the model they worked on or used, who they worked with, etc.

For me not having something public is not a blocker for hiring. Just something that simplifies the hiring process -- although, lately I've had a lot of candidates with many personal projects, no branches, no pull requests, not showing experience with Git, etc. (i.e. in some cases it seems some people try to create many repositories just to show that they have something... which can be a red flag too).

No bubble here, I think. I'm also based in EU, Spain, where job market is super competitive right now, and it's quite hard to find candidates.

As I said, it doesn't need to be only public repositories. If I interview for containers, then I ask if they have something public on DockerHub, GitLab/GitHub registries. If the answer is no (happened just a few days ago), then I ask what they used. I expect them to say something like Artifactory, Nexus, a directory on a server or HPC, or just explain why they didn't need a registry. For me there is really no wrong answer here. But if they reply they don't have anything public, and have no idea what's a container registry, then that's probably bad -- this is what I think OP of this thread meant by eliminating candidates that are full of---.

> Where I live in Europe 90%+ of workers barring those currently in academia, have no public work because most companies don't publish their work, so you'd never hire anyone with that barrier.

My wife also works here, and she's a recruiter in a EU company, in cancer/research. Most applicants won't have things public in that case, but they can still explain things. If there is a candidate with similar profile, that managers liked, and they have a lot of good work, public, then it's up to managers. I just explain what I saw in the repositories, whether I'd work with the candidates, and the managers hiring choose based on risk for companies.

As you are based in EU, you probably have the same problem that the hiring process can be expensive for company, and risky if you eliminate candidates, and have to go back during the experience/trial period of candidates.

After some years applying for different positions, I started asking receuiters about their hiring process and straight out dropping out when there were technical tests like this.

Even when you nail the test, it is no guarantee you won't be just wasting your time.

I explain to the recruiter why I am turning down that opportunity and thank them.

Best jobs I had were mainly via my network of friends, or reaching out to engineers directly asking about their companies and open positions, then sharing CV and GitHub, then chatting about technologies used, bugs in production, and other past experiences.