I am a C/C++ programmer mostly. Is there good documentation for python development when I care about performance and memory footprint out there as a book or something that anyone can recommend? It was fun reading this but I like to know more.
Python is good enough 90%. You get faster 2x 10x, etc code by picking better algorythms or solutions. In Python, you should not be caring about 10% or 30% speed improvement. It's not worth it, It's not Python's strength.
When you need faster you go to C based libs (the python libs that need to go fast are already in C), https://en.wikipedia.org/wiki/Cython https://en.wikipedia.org/wiki/Numba etc.
[1] https://www.pypy.org [2] https://www.graalvm.org/docs/reference-manual/languages/pyth...
There are other approaches which can work too, like compiling C to LLVM IR using Clang so a function can be inlined by an LLVM based JIT at runtime.
Second Edition of High Performance Python in the works
I'm very pleased to say that Micha and I have started work on a Second Edition of High Performance Python with O'Reilly, planned for early 2020. This book will use Python 3.7 and will add tools that barely existed 4 years ago including Dask and probably some Tensorflow, amongst lots of other goodies. I'll let you know how the book progresses via this list. So far I've updated the Profiling chapter, added some advice on 'being a highly performant developer' and have rebuilt the Cython/Numba/PyPy code for the Compiling chapter.
If you have to read everything into memory, because you're doing some sort of transforms like a list to a dict or such, explicitly deleting variables can help, as well, rather than waiting for them to go out of scope, but this should be pretty rare.