jphoward

The game in easy/intermediate is very fun and satisfying! However it doesn't seem to scale too well beyond this - it feels unless you get 'lucky' with one row/column having a very low total, you end up with many many degrees of freedom for each move, which scale, I suppose, quadratically?

Sudoku at the hardest difficulty also has this, but usually the errors manifest a bit more quickly. It feels in this mode you could end up finding your error 'at the end' and have an absolutely diabolical time debugging your error?

Multi-GPU training is a double-edged sword. If you are at the stage where you are running your code in a iPython notebook then you are almost certainly not going to benefit from the multiple GPUs, and I strongly suspect you'd be better with fewer and larger GPUs, even if training time is prolonged.

The reason I say that is, if we go with PyTorch, you basically have 2 options for multi-GPU training.

- DataParallel - where you clone your model over each GPU, but each one functions independently, and after each 'training step' they pool their data. This has downsides, in that you don't get to process intermediate layer outputs and synchronise your batch normalisation layers - so you can't use it to train 'big' models. It just makes your smaller models train more quickly. However, you can at least use these in a 'normal' training script.

- DistributedDataParallel - this is 'proper' multi-GPU training - you can now train big models and put a little bit of data on each GPU, and have then synchronise their results after each layer. However, this can be very annoying to use - each GPU runs in its own background process which is either spawned or forked (depending on Windows/Linux) and you therefore cannot run it in an iPython notebook, or an interactive Python console. It also makes tracking metrics etc. MUCH harder - because you need to reduce your metrics over each GPU process (because otherwise you get 4 accuracies, 4 mean squared errors etc. if you have 4 GPUs, and each process only sees one of them).

I personally prefer having 1 GPU with 24 Gb RAM over 3 GPU with 12 Gb RAM - because I can have a larger batch size on each GPU, which is VERY VERY advantageous in large models where you can only have small batch sizes, and batch normalisation starts falling down. I'd rather wait 2x as long for a 'better' model to train.

James Powell frequently gives very deep Python talks at conferences like Pydata, and shows you his coding approach in real time on Vim. Plenty of them are on YouTube and are worth a watch. This one [1] entitled "So you want to be a Python expert?" has almost half a million views!

[1] https://www.youtube.com/watch?v=cKPlPJyQrt4&ab_channel=PyDat...

When I first read about JAX I thought it would kill Pytorch, but I'm not sure I can get on with an immutable language for tensor operations in deep learning.

If I have an array `x` and want to set index 0 to 10, I cannot do:

  x[0] = 10

I instead have to do:

  y = x.at[0].set(10)

I'm sure I could get used to it, but it really puts me off.

The hydraulic power utility is fascinating - the sheer amount of water lost via leaks from water piping in this century is huge, and so keeping a high pressure system running back then must have been tremendously difficult. Especially given a similar sized leak will have a much larger impact in a higher pressure system.

If you mean Aseem Malhotra, he is an idiot and nobody in the entire region of London that he trained in thinks he has an IQ above 100. Pre-COVID he was unemployably bad as a doctor, and has never had a formal job in the NHS since he completed his training. He has latched on to vaccines hoping he can make a quick buck that way.

NNT is an interesting metric that people were very sanctimonious about when I was at university, but has rightly lost favour.

This is because NNT is actually a terrible metric for any treatment that reduces risk over a long period, including the most important outcomes like mortality.

Imagine a drug that halves your risk of dying of heart disease. Heart disease kills 1 in 4 people in developed countries, and so realistically it would be a no brainer that everyone middle aged or over should take it.

If you did this in a randomised controlled trial, you might recruit 1000 patients, and randomise 500 to this new drug and 500 to placebo. If you ran the trial for a year, you might find 1% of the placebo arm died, and only 0.5% in the control arm. This demonstrates a risk reduction of 0.5%, or a NNT of 200.

An NNT, by conventional teaching, is underwhelming.

Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy. In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.

It's different if you have a single one-off therapy, where all the benefit will have been delivered by a certain timepoint, at which you measure the NNT. But for a lot of medicine it is not that simple, and the NNT sucks.

Generally in Python it's much better to use list comprehensions when possible, it offers significant speedups usually:

  def zip(ls1, ls2):
      return [(ls1[i], ls2[i]) for i in range(min(len(ls1), len(ls2)))]

And indeed you could even use a generator instead:

  def zip(ls1, ls2):
      return ((ls1[i], ls2[i]) for i in range(min(len(ls1), len(ls2))))

I could eat a knob at night.

I use pytorch the vast majority of the time, but I think there are 2 reasons tensorflow remains competitive.

First, is TF's deployment has always been ahead of Pytorch's (although the gap is closing, especially as Onnx becomes more popular).

However, the more important reason is how amazingly good value TPUs are in terms of their RAM and FLOPS. Although pytorch has XLA support, it just doesn't work as well as TF on TPU pods.

In Kaggle competitions, when the input data can fit on a consumer GPU everyone uses pytorch. When it doesn't, everyone uses the free TPUs on the Kaggle platform and reverts to tensorflow/keras.

I completely agree! I personally find these powerful new network releases border on the depressing, in that they aren’t really network releases but huge training systems of dispersed YAMLs. YOLOv4 was a case in point where I was too overwhelmed to try and integrate it into a project I was working on.

PS you are a hero of mine - I’m an academic medical doctor for who CS231n was my first foray into AI, and since then I’ve gone on to gold medal in a couple of Kaggle competitions and secured 5 years of higher research funding to pursue clinical AI. I am immensely grateful to you and Fei-Fei Li.

This is likely to share its name with the next Prime Minister of the UK...

"almost five times larger than France and Sweden" - the UK population is almost 7 times that of Sweden, so that's quite flattering. But what are the reasons for France's relatively small tech industry?

RadiAnt is OK - it's got a trial license model but when the license expires you can request an extension for free (big WinRAR energy). There's nothing as good as Horos that is similarly free and open source, sadly.

What you may prefer to do is use the excellent pydicom library for Python, which loads dicom files as numpy arrays, allowing you to convert to any other file format you fancy, whether that be image or video.

Medical training (both med school, and early career) in the UK is in a fascinating situation at the moment. The term "morale is at an all time low" doesn't begin to cover the (justified) anger I see in them at the moment. Independent government bodies admit inflation-adjusted junior doctor wages since 2004 have gone down by 30%. Last week, some doctors got 4.5% pay rises, which were met with incredulity given inflation is at 9%, the tremendous efforts during Covid etc., but juniors don't even get that - they get 2%.

To put it into context, a junior doctor in the UK makes £29,384, and they will leave university with close to 6 figure debt, with 7.3% interest rates.

But it's not even just that. Over the last few years, the UK has embraced the roles of non-medically trained physician associates (PA), which in the USA have been a success by relieving medically trained doctors from the more protocolisable jobs, so they can focus on the more difficult. However, a PA earns £39,766.76 - far more than the juniors with none of the responsibility. I won't go into details on the issues about how juniors have to maintain annual portfolios, change hospitals every year etc. etc., because the Royal Colleges say it's necessary to become a good doctor, whereas PAs don't need to do any of it.

So when juniors in the UK see this, that they could have gone to University for 2 years rather than 5, with lower entrance requirements, and end up in a far better financial situation in a shorter time, many of them just don't want to be a doctor anymore.

I can say, without a doubt, that junior doctors will strike in the UK within the next 2 years. I am no longer one, but I completely get where they are coming from, and support them.

But isn't it always on for "Hey Siri"?

But this is adjusting for the baseline illness, remember. This isn't just age-standardised mortality rates. It's disease-standardised mortality. Hopefully they've adjusted for confounders...

Is it me or is this very depressing and scary? I always assumed the creative industries would be relatively resistant to automation, with more algorithmic jobs more easily replaceable. But with this you never need to buy stock pictures ever again, right? I saw several stock-picture-esque images of indistinguishable quality, and yet with incredible specificity to the user-supplied description.

People don't feel guilt online, generally, because they either forget the person they are talking to is a vulnerable human, or they just imagine them as someone deserving of their vitriol.

However, as soon as they are sat in front of someone and are picking up on the visual clues in facial expressions and body language which make it clear that what they are saying is actually affecting a real person with feelings, almost everyone has a profound sense of empathy or guilt.

Because dependency hell isn't dreadful enough? Some entries will be completely missing from requirements.txt because the person assumes a certain version of python is present.

This is a beautiful example of how bad Wikipedia is within mathematics and a lot of the physical sciences. An introduction such as this would be unacceptable on a page about medical sciences:

"...is the peculiar manner in which the Fourier series of a piecewise continuously differentiable periodic function behaves at a jump discontinuity. The nth partial sum of the Fourier series has large oscillations near the jump, which might increase the maximum of the partial sum above that of the function itself"

I have a PhD in machine learning and also an interest in magnetic resonance imaging. Sometimes Gibbs phenomenon leads to an artefact (called Gibbs artefact) that we see in our images of the heart when we are doing a certain sequence called stress perfusion cardiac MRI. I am not an idiot, and yet I actually cannot understand the first paragraph of this page.

Mathematics articles on Wikipedia so frequently read in a way that is only interpretable by mathematicians or physicists. This is not the point of Wikipedia! You don't even have to pick an advanced topic - the Taylor Series is taught in high school maths in the UK, and look at the introduction to its Wikpedia article, it's ridiculous!

Articles about relatively esoteric topics in the biological sciences are almost always much better, and at least try to first introduce the topic in a way the average reader can understand, e.g. https://en.wikipedia.org/wiki/Supraventricular_tachycardia

This is such a great example of how people that know math(s) really struggle to explain things to people who don't, because they struggle to realise how much lingo they are using.

The first point explains something as basic as absolute and relative errors. Great.

1 page down (s)he's casually thrown in function notation with R representing the real numbers with no explanation. Anyone who isn't familiar with this notation is immediately intimidated. But I'd wager almost anyone who is familiar with it didn't need to read the page before it. But also, I think it's just unnecessary to be this specific in the first place, if you're trying to quickly explain concepts.

It's actually a huge problem in maths and even Wikipedia (I feel) is pretty guilty of this.

If you take a (relatively) key mathematical concept, such as the Taylor series, then the opening gambit is completely incomprehensible to anyone who isn't already knowledgeable in maths: https://en.wikipedia.org/wiki/Taylor_series

This, to me, seems in huge contrast with other scientific fields. Medicine wikipedia, for example, caters towards the "average reader" first, and doctors later. This is entirely appropriate for an encyclopedia. For example: https://en.wikipedia.org/wiki/AV_nodal_reentrant_tachycardia...

My understanding is Conway's game of life can be done by simply convolving a 3x3 array of all ones (except a central 0) through every pixel, and then looking at the results of that convolution to see if the pixel is dead or alive, e.g. https://stackoverflow.com/questions/46196346/why-does-my-gam...

If that's the case, would a nice solution be to model it as a recurrent neural network, e.g. pytorch or jax, with a single convolutional layer, with a single kernel, which you don't backprop to, and then try to minimise to loss with reference to the target image, by backpropagating to the input image?

It then becomes highly analogous to Google's deepdream, but instead of optimising the image for the output class, you optimise it to the output image.

The results that should be used to compare their results to others involves training on just under 1.3 million images across the 1,000 classes.

Their best results involve 'pretraining' on a dataset of 300 million examples, before 'tuning' it on the actual ImageNet training dataset as above.

SOTA is 'state of the art'. Image recognition is a task classically appraised by calculating the accuracy on the ImageNet dataset, which requires a system to classify images each as one of 1,000 pre-determined classes.

I _love_ numpy, and I am getting excited about jax, too.

However, I do have one request for it. Getting the argmax of a multi-dimensional array, in terms of the array's dimensions, is difficult for new users.

np.argmax(np.array([[1,2,3],[1,9,3],[1,2,3]])) is 4, rather than (1,1). I understand why, but it seems strange to me that argmax cannot return a value the user can use to index their array.

Having to then feed that `4` into unravel_index() with the array's shape as a parameter seems less elegant than say passing a parameter of "as_index=True" to the argmax.

I found when I started programming I never used OOP. Then I used it too much. And then recently I use it incredibly sparingly. I think this is most people's experience.

However, there are certain situations where I cannot imagine working without OOP.

For example, GUI development. Surely nobody would want to do without having a Textbox, Button, inherit from a general Widget, and have all the methods like .enable(), .click(), and properties like enabled, events like on_click, etc.?

Similarly, a computer game, having an EnemyDemon inherit from Enemy, so that it has .kill(), .damage(), and properties for health, speed etc.?

I'd really like to know how the most anti-OOPers think situations like this should be handled? (I'm not arguing, genuinely interested)

All good points! Great work on the project and thank you for working to make Pandas more friendly!

What I think would be a nice approach would be someone creating a new Dataframe class that merely inherits from a Pandas dataframe but just implementing .filter(), .summarise(), .select() etc. as methods, using the dplyr syntax. If they return their own dataframes then chaining follows naturally.

I know that isn't how dplyr works, but it feels more Pythonic, and this solution isn't entirely analogous to dplyr anyway.

This approach seems a little complicated, though I'm sure with some use I could learn to enjoy it.

I find JAX really exciting. The idea of numpy with autograd is exactly what Pythonistas want. The elephant in the room, though, is “why not Pytorch?”

Everyone knows JAX is what Google realised Tensorflow should have been when they realised how much of a joy Pytorch was to use. I actually think JAX does offer some advantages, not least true numpy interoperability. However, not mentioning *torch a single time in the blog post seems a little disingenuous for a Google-owned deep learning enterprise.