Memray: Python memory profiler | Hacker News Reader

Memray: Python memory profiler

bloomberg.github.io

196 pointsbx37621 comments

Memray is incredibly usable. In a few projects I was able to go from 0 to identifying my "problem children" in less than 15 minutes.

I don't use it often, but when I need to I'm glad it's a part of my toolbox!

Interesting feedback. I’ve found memory profiling in python to be vastly annoying, and tooling less than adequate. I’ll give it a try.

What kinds of projects did you find Memray useful for? I work on a FastAPI app with a ton of API endpoints and the occasional memory leak. So far I’ve had a hard time using Memray (or any other profiler) to figure out which endpoints in particular might be causing the leak. Would love to hear from anyone who’s been in a similar spot before.

Mostly a complex scientific computing application. (sorry to not be more specific, but it was indeed not a webapp)

I have a lot of praise for using Memray as part of working out some memory kinks in a codebase I’m working on. It allowed me to quickly iterate on some hunches and then enshrine the resulting memory savings in tests via pytest-memray.

Testing memory usage somehow has never occurred to me. Neat idea, thanks.

Does anyone know how it compares to scalene?

https://github.com/plasma-umass/scalene

The main difference is Scalene is a sampling profiler, and Memray appears to be an instrumentation profiler. Instrumentation profilers usually give more precise information at the cost of a slowdown.

From Memray:

> Traces every function call so it can accurately represent the call stack, unlike sampling profilers.

Memray: a memory profiler for Python - https://news.ycombinator.com/item?id=31102089 - April 2022 (48 comments)

memray is awesome. Used it to demonstrate massive memory savings wins for presentations

My favorite feature is the jupyter integration. Just decorate a cell with %%memray_flamegraph

I'd really like a version of this to spit out a csv or something for going straight into making plots

bloomberg! they seem to be sponsoring a bunch of good work for OSS. Thank you!! I also happen to meet some of their engineers at Gophercon this year. Wouldn't have thought they do so much! Thanks!

Bloomberg solves genuinely big & interesting data challenges

Ive used Memray and it works great locally. But when I deployed my application over long running processes (i.e. in production) because I want to see memory usage over a long period of time, the profiler outputs get really large, like hundreds of gbs. They cause disk outages and also take forever to download and visualize with the flamegraphs. What do people use to understand memory usage of long running workloads in production?

This is great! I managed to spot some inefficiënties in my ML inference pipeline by generating some flame graphs using Memray.

Does it also work with CUDA memory?

All my memrays gather 'round her.

Miner's layday, stranger to blue water.

Note that there is no support for Windows (only WSL).

Now if only there was one of these for Node.js...

Memray is definitely best in class!

Is this working on Windows?

Not natively. WSL only.

Reply on news.ycombinator.com