IanOzsvald

I continue to build my London based monthly directed-exploration LLM days where we try to collaboratively push through a benchmark. We're doing ARC AGI 2026 again in a couple of weeks: https://playgroup.org.uk/ Recently we built parts of GPT2 from scratch and worked on agent based benchmarks

I've spent 6 months building a network of curious LLM hackers in London who want to push the edge of what we can do.

This Friday we pull GPT apart and rebuild bits by hand. A few weeks back we tackled ARC AGI 2026. Prevoiusly we did fine tuning and making GPT funny.

It is a sort of a guided hackathon (generally I plan goals for the day) and collaborative study group.

Much fun, no money, lots of smart folk asking good questions: https://playgroup.org.uk/

+1 Merlin. I also stop and do a few minutes with Duolingo in the park, then take a breath and just listen to the wind and birdsong.

There's a profiler cell magic for Notebooks which helps identify if you run out of VRAM (it says what runs on CPU and GPU). There's an open PR to turn on low-VRAM reporting as a diagnostic. CuDF is impressive, but getting a working setup can be a PITA (and then if you upgrade libraries...). Personality I think it fits in the production pipeline for obvious bottlenecks on well tended configurations, using it in the r&d flow might cost diagnostic time getting and keeping it working (YMMV etc)

Thanks :-) I use it for all my talks and finally decided I'd better start sharing it a bit. It really is useful to understand the memory cost of things like Pandas operations

I'm the co-author of High Performance Python, Micha and I are working on the 3rd ed (for 2025). Lots of bits of the book came from my past conference talks, they're available here (and the public talks will generally be on youtube): https://speakerdeck.com/ianozsvald

Mostly that content has a scientific focus but the obvious thing that carries over to any part of Python is _profiling_ to figure out what's slow. Top tools I'd recommend are:

* https://pypi.org/project/scalene/ combined cpu+memory+gpu profiling

* https://github.com/gaogaotiantian/viztracer get a timeline of execution vs call-stack (great to discover what's happening deep inside pandas)

* my https://pypi.org/project/ipython-memory-usage/ if you're in Jupyter Notebooks (built on https://github.com/pythonprofilers/memory_profiler which sadly is unmaintained)

* https://github.com/pyutils/line_profiler/

@munhitsu gave me a demo at the weekend (I'm on Android and it is iPhone only), it seemed pretty slick and very easy to use, though I confess not something I personally need right now

I've just spent the morning uninstalling and reinstalling different versions of Nvidia driver (Linux) to get nvcc back for llama.cpp after Linux Mint did an update - I had CUDA 12.3 and 12.4 (5GB each), in conflict, with no guidance. 550 was the charm, not 535 that was fine in January. This is the third time I'm going this since December. It is painful. I'm not in a hurry to return to my cuDF experiments as I'm pretty sure that'll be broken too (as it has been in the past). I'm the co author of O'Reilly's High Performance Python book and this experience mirrors what I was having with pyCUDA a decade back.

Don't forget that a sequence of numpy operations will likely each allocate their own temporary memory. Numba can often fuse these together, so although the implementation behind numpy is compiled C you end up with fewer memory allocations and less memory pressure, so you still get your results even faster. Numba also offers the OpenMP parallel tools too. I have a nice sequence of simulations in my Higher Performance Python course going from raw python through numpy then to Numba showing how this all comes together. Just try having a function with a=np.fn1(x); b=np.fn2(a); c=np.fn3(b) etc and compile it with @jit and you should get a performance impact. Maybe you can also turn on the OpenMP parallelizer too.

Really Numba will speed up numpy and some scipy (there's partial API coverage) and math based pure python. I think it is unlikely it'd be used away from math problems. As another commenter mentioned it can be used to accelerate numpy-array based Pandas (but not the newer Arrow based arrays), and again that's for numeric work.

That's my book :-) Micha and I are working on the 3rd edition right now. Cheers!

How about hyperlinking each title so it can easily be opened in a new tab?

You may want to look sideways to companies such as hedge funds. They have DNN teams and experiment with LLMs, you may find interesting optimisation opportunities with such teams. Charge according to opportunity that you open up, not electricity saved!

In the UK I use Redber for beans, I drink decaf (Swiss Water prices) fresh ground in the afternoon and have 2-3 caf cups in the morning. Redber had a wide selection and several roast levels. No caffeine after noon. I use a 2-cup espresso Bialetti stove top, the second smallest.

Indeed I was the one who got confused by the name! Thanks for attending the discussion Jay and I'm happy to see Daft being discussed here

PyPy uses a modified Mark and Sweep garbage collector, CPython uses Reference Counting. C extensions such as NumPy (and so Pandas, sklearn etc) are compiled expecting Reference Counting. A translation layer is needed for memory management from PyPy to extensions like NumPy and that introduces overhead (historically - a lot of overhead).

I use a FLIR One on Android. I've charted internal leaks (where cold air blows in through cracks) and external (where heat escapes through eg old windows). Wait for a cold day (eg 0C), heat the house, investigate everything you can. My 1930s London house has SO many leaks, I've spent 3 years slowly fixing them. I have a talk I should give on using 12 Govee hygrometers to back-calculate moisture (absolute humidity) coherent per room, as I was charting moisture loss to trace air leaks.

Hey Ritchie. Re legacy I'm thinking about wider teams in large organisations (eg SWEng system support teams) and IT mandating library upgrade frequency - switching to new libraries can have widespread impacts and the cost can be high. Polars (and Vaex) are definitely here to stay, but I think integration to existing teams may take a while. I followed the PRs around numpy data sharing but I wasn't sure on the end result. Is the data sharing copy-free (always?)? I wasn't sure what the impact was if Rust and NumPy are utilising the same bytes (or even if that was possible). Can you share some detail? Edit - reading the updated thread I your reply https://news.ycombinator.com/item?id=34298023 which says "1D often no copy", can you add any colour to when a 1D no copy can't happen and whether 2D no copy is an option?

This https://docs.dask.org/en/stable/spark.html notes "However, Dask is able to easily represent far more complex algorithms and expose the creation of these algorithms to normal users [compared to spark]" linking to: http://matthewrocklin.com/blog/work/2015/06/26/Complex-Graph...

I'd argue a little differently. I'm co-author of O'Reilly's High Performance Python book and I've been teaching a course around this for years, often to quants.

1. Pandas if you stay in RAM, if the team and org already know this, but learn about reduced-ram types (eg float32 rather than float64, categorical for strings and dt if low cardinality, new Arrow strings in place of default Object str). Pandas 1.5 has an experimental copy-on-write option for more predictable (but probably still not "predictable") memory usage, try to use a subset of team-agreed functions (eg merge over join) due to varied defaults that'll confuse colleagues (eg inner Vs left and other differences). Buying more ram is normally a cheap (if inelegant) fix.

2. Dask as it is an easy transition from Pandas (and it scales numpy math, arbitrary python non-math functions and lots more), lots of cloud scaling options too. Stays within Python ecosystem for reduced cognitive load. It is probably less resource efficient than Vaex/Polars

3. Ignore Dask and stick with Spark if your team already uses it, as it'll scale to larger workloads and you've taken the cognitive and engineering hit (pragmatism over purity)

Vaex and Polars are definitely interesting (hi Ritchie!), and great if you're doing research and are comfortable with potentially changing APIs but you have no legacy systems to worry about. You might buy yourself a lot of future manoeuvring room. You'll find fewer clues to tricky problems in SO than for Pandas, and have a harder time hiring experienced help.

Can other data scientists comment?

I'm 15 years in with python and scientific work. For a lot of years I liked conda but then it got crazy slow. Next I started making conda environments and installing packages with pip. Now I'm experimenting with mamba ("fast conda") and that's pretty good.

Conda envs mean I can experiment with different versions of Python (I'm a co author for O'Reilly's High Performance Python so eg 3.11 and 3.12 are pretty interesting right now). Conda "should" also make identical teaching environments (I teach my own courses). Pip was a pragmatic choice to get installations in minutes not hours in the years when conda was silly-slow.

The above is also all for short -lived research work (my typical client mode for scientific work), so it is probably different to anyone doing long-run dev work, production deploys, or for those not needing non-Python binary support (eg GPU/C/Fortan lib support).

With our infant using a digital thermometer in the ear, we routinely observe that the left ear is hotter than the right. We take 3 measurements in each ear, we see a similar variance as you mention per ear. Have you observed a similar difference? As adults with the same device we don't see a difference in our own ears.

I did some checking and found it hard to get clear numbers. It seems that the majority of cars in London will be compliant but the majority of vans will not be, so business is affected over private households.

The scrappage ("upgrade") scheme applies to non compliant cars and vans, with a focus on low incomes, charity and small businesses, it starts in January and feels reasonably generous: https://tfl.gov.uk/modes/driving/scrappage-schemes

The scheme pays more if you replace an ICE with an EV or take a public transport pass which further supports the goal of improving London's air. As a London resident, parent and driver I'm strongly supportive of improving our air.

I do some of this in data science when I'm giving strategic support to teams. Be aware that managers generally see you as a cost who doesn't bring financial benefit to the team (whilst the team are desperately asking for practical training so they can deliver quicker). Find ways to couch your offering in business benefits - you've helped teams avoid expensive bugs or via pragmatic refactoring they've responded to market change faster. Finding believable examples IMHO is actually quite hard. I've found managers far more open to strategic support (to help the team "go in the right direction"), with things like teaching code reviews and best practices bolted on as secondary benefits.

I interviewed author Ritchie Vink on my newsletter (NotANumber) some months back, he's smart and the library has a nice design. I still barely know anyone trying it, it did get a write up just recently here: https://news.ycombinator.com/item?id=32998040 On one of my recent Higher Performance training courses a hedge fund person said they'd tried it with mixed success - some things faster than Pandas, others more expensive on RAM. I'm hoping to have a play soon but have only lightly tinkered so far. The parallel-by-design nature is nice, but I think the API is still evolving rapidly making it harder to develop with.

Here's a benchmark for 3.8-3.11b: https://www.phoronix.com/review/python-311-benchmarks/4

The geometric mean of the 3.8 to 3.11b benchmarks was a 45% speedup.

What's your plan for redframes? It looks really new? I'm the co-author of O'Reilly's High Performance Python so I'm always on the lookout for pandas alternatives. Are you looking at speed implications too? Bigger-than-RAM use cases? "Easier than pandas" only (which, of course, is totally huge in its own right)?

Numba focuses on scientific use cases speeding up most of numpy and some of scipy. Whilst it can compile some parts of pure Python (eg numeric array loops), generally you have to copy the data in to the numba side which can be slow in volume. Numba isn't really meant for pure Python speed-ups, it gets rid of numpy specific inefficiencies. Personally I'm pretty excited to see a fresh JIT working its way into core CPython (I'm the co-author of O'Reilly's High Performance Python).

I author a Python data science focused newsletter https://buttondown.email/NotANumber , I've built it up over 4 or so years to a couple of thousand readers. I don't use automatic link gathering but hand-write advice in each newsletter (and yes, this can take some time). I've started to interview respected people which offers a nice chunk of value for my readers.

For promotion I've found that sharing appropriate ideas in a private community (e.g. data science slacks) where I'm known & trusted can be a good lead source. Tweets on particular topics help. LinkedIn helps with specific advice. Linking to relevant articles in another newsletter and getting a reciprocal mention helps (but the link has to be useful to the readers and the other audience has to care about my newsletter, else no point), that only works if you know the other author.

In each of my strategic client sessions I mention the newsletter. One chunk of value I offer is to do job listings - having your job in front of my curated audience who trust me offers much more value than e.g. LinkedIn (and a _much_ smaller reach).

In short I think try to find relevant communities, do not spam (obv.), share relevant links, offer to share something cool from someone else with no requirement for reciprocation (if it is good - share it anyway) and maybe they'll reciprocate in some way. It takes a lot of effort but is a sure fire way to build up an engaged audience who trust you.

Years back I built IPython Memory Usage[0] which shows how much RAM and time was used per cell in Jupyter Notebooks (and originally the IPython shell). This is very useful for diagnosing why some Pandas and NumPy operations use a lot of RAM [1] which can also point at slow-downs, so you can compare different approaches to find more efficient solutions rather than randomly trying stuff until something no longer breaks.

0: https://github.com/ianozsvald/ipython_memory_usage

1: https://github.com/ianozsvald/ipython_memory_usage/blob/mast...