rpep

I think there are some things this misses about the scientific ecosystem in Universities/etc. that can make the cloud more attractive than it first appears:

* If you want to run really big jobs e.g. with multiple multi-GPU nodes, this might not even be possible depending on your institution or your access. Most research-intensive Universities have a cluster but they’re not normally big machines. For regional and national machines, you usually have to bid for access for specific projects, and you might not be successful.

* You have control of exactly what hardware and OS you want on your nodes. Often you’re using an out of date RHEL version and despite spack and easybuild gaining ground, all too often you’re given a compiler and some old versions of libraries and that’s it.

* For many computationally intensive studies, your data transfer actually isn’t that large. For e.g. you can often do the post-processing on-node and then only get aggregate statistics about simulation runs out.

> I think it's storing array data as-is in straight binary.

You can't infer the stride from the raw binary; a 2-D array [[1, 2], [3, 4]] in C ordering just looks like:

1,2,3,4

and in Fortran ordering it's

1,3,2,4

So there must be additional metadata stored. Maybe it's just using the .npy format internally - https://numpy.org/devdocs/reference/generated/numpy.lib.form...

There's quite a bit of magic hiding under this. For e.g., try creating a NumPy array with different orderings, and the metadata looks the same:

import asdf

import numpy as np

x = np.array([[1, 2], [3, 4]], order="C")

y = np.array([[1, 2], [3, 4]], order="F")

tree = {"x": x, "y": y}

af = asdf.AsdfFile(tree)

af.write_to("example.asdf")

and you get in the metadata no distinction between the two arrays even though things like byteorder are included:

x: !core/ndarray-1.0.0

  source: 2

  datatype: int64

  byteorder: little

  shape: [2, 2]

y: !core/ndarray-1.0.0

  source: 0

  datatype: int64

  byteorder: little

  shape: [2, 2]

This makes me wonder what it's actually storing - is it actually doing something like pickling the NumPy array?

> can break things that there's no way to know until production

I would argue that any production system should have enough tests that upgrading a dependency that breaks compatibility should cause failure of the test suite in some way

... yes, I am actually.

My surname is Pepper and it didn't come up with me :D

Porting x86 to ARM is not really that difficult. You've got to recompile your code on the different machine anyway, so the architecture itself shouldn't be the major barrier.

This must be regional, because it certainly doesn't show that for me in the UK, top two results are the wikipedia articles for the vegetable (capsicum) and black pepper.

Google Voice only works in the US so not an option for most people.

The Metaverse is just an example of everything that's wrong with it. Why would anyone want to spend time in something that looks horrendously outdated?