Out of curiosity, what throughput do you have that would require GPU labelling?

The most resource heavy bit is text rendering I guess, but that could be cached per class-name and reduced to a memcopy. Otherwise drawing rectangles is pretty quick on a CPU to the point where I'd imagine the memory transfer to the GPU is probably comparable to the draw ops?

I've got OpenCV down to around 10ms per image (single thread, python) without the caching idea I mentioned above.

Interesting idea! I'll have to do some studying as to how to make that happen, so I doubt I'll be adding that option anytime soon but I'll be adding it to the icebox.

Thank you for the comment!