Hacker News Reader

Comment by dotancohen | Hacker News Reader

Maybe you're the person to ask. My Python Qt app is sluggish when it loads tens of thousands of values from SQLite. Various variations of pagination and lazy loading hurt usability. But isolating the issue and testing with C++ and Rust show that those two languages don't have the performance hit of Python.

I could use C++ and stick with Qt, but I'd much prefer Rust. Rust has no good Qt bindings. What are my options?

The app makes use of QV/HBoxLayout, QWidget, Qdialog, QPushButton, and other really standard features. It reads from the filesystem, reads and writes to SQLite, and outputs sound from mostly ogg files at various speeds (through VLC behind the scenes). I stick with Qt because I like how it integrates with KDE and other desktops flawlessly.

View on HN

> What are my options?

> I stick with Qt

These come up on search results if you combine Qt plus the language.

Go:

* https://github.com/mappu/miqt

Java:

* https://github.com/OmixVisualization/qtjambi

Nim:

* https://github.com/jerous86/nimqt

* https://github.com/seaqt/nim-seaqt

Zig:

* https://github.com/rcalixte/libqt6zig

Thank you. I don't see Java as having a performance edge over Python ))

Same for the other languages - they likely won't have the performance of C++ or Rust. But I'll test them anyway, even just for the academic exercise the endeavour is worthwhile. Thank you.

Java is actually pretty fast, and certainly much, much faster than python. There are plenty of reasons Java isn't my preferred language, but speed is not one of them.

This is great to know, thank you. I still remember the days of the knock, knock jokes about Java.

The jokes were mostly about startup time for the JVM (only happens once as when its in RAM it stays there and the loading time doesn't repeat for the next java program you run). It was also a lot more notable in the 90s when java was a browser plugin to run applets and browsers were single threaded (so the browser froze for a few seconds while the JVM loaded).

I think it's been out of date for a long time with modern computers having much more RAM and speedy SSDs (and also JVM optimizations). For actual run time performance once its loaded the JVM has had great performance in the early 00s already and improved since. And for server side it was never an issue as you don't restart the server very often (so the JVM is always already loaded).

I'm not necessarily advocating java, I think rust is def the best solution in most cases -- but java definitely has a significant performance edge on python. The JVM will never be quite as fast as native code, but it's very fast these days and it gets better every year. Certainly much, much faster than python on the average.

Thank you, I will evaluate Java for this purpose, though I almost certainly won't actually use it in the rewrite.

Java is about an order of magnitude faster than Python. It's a JIT-compiled statically typed language, after all, and Oracle's JIT compiler is the best in class.

That said, given your original request, I don't think it has much to do with language choice. Displaying a very large number of rows in a data grid or similar is a classic GUI task in line-of-business apps, and the answer has always been either virtualization (lazy loading) or pagination.

One trick to improve UX when virtualizing data is to do preloading asynchronously in the background - ideally on a separate thread - and start it before the user scrolls all the way to the end (e.g. when there's still a couple of pages of data left).

Yes, that is exactly what I have been doing. Thank you.

One thing that I would like to discover is how to adjust the scroll bar size so that it appears as if more records are loaded than actually are.

In Qt you should use the model/view framework if possible. It gives a virtualized list with a render callback. https://doc.qt.io/qtforpython-6/overviews/qtwidgets-model-vi...

Thank you. I will definitely peruse this.

https://areweguiyet.com/

Also did you test similar functionality in C++, I haven’t compared many implementations but I app I use that has an SQlite db being read with Qt(C++) is pretty sluggish whenever you touch the DB.

Maybe store the values in a dict and only read/write from sqlite when needed. Dicts are very fast in python.

Yes, that's what I'm doing. It's the initial load that is heavy.

I've also tried implementing lazy loading and pagination, with a pseudo-infinite scroll technique based on SQL LIMIT pagination, but the resulting CPU and memory spikes are noticeable.

This website is useless if accessibility compatibility is not mentionned.

> I could use C++ and stick with Qt, but I'd much prefer Rust. Rust has no good Qt bindings. What are my options?

There are several Qt binding for rust, for example cxx-qt. I haven't tried myself but it looks maintained. Why is it not good?

Otherwise, the most promising equivalent to Qt in Rust would be Slint.

Thanks. I forget why I discounted cxx-qt. It think it didn't support some common Q* object but I don't remember which. I'll experiment with it and with Slint, which I've been eyeballing anyway. Thank you.

Is Cython and/or avoiding copies an option?

Cython is a great idea, thank you. I'll definitely try that. The code is all properly typed anyway.

I'm not sure what you mean by avoiding copies. I'm rather stringent with memory, I don't have superfluous copies of data in memory. Or did you mean something else?

Thank you.

Sometimes Python code makes copies when we’re not expecting it to. Hard to know but performance optimization books/blogs discuss it.

I'll Google for that, thank you!

Reply on news.ycombinator.com