Hacker News Reader

Comment by turdnagel | Hacker News Reader

With graphics cards prices coming down, I'm considering purchasing one to mess around with GPU-based ML. Is a model like YOLOv6 runnable on a modern single GPU? If so what would get me the best bang for my buck?

View on HN

For training, more GPU RAM will allow you train with greater resolutions in less time and better performance. Before feeding data to a model, it needs to be resized to a "network dimension" (YOLOv4 default is 416x416 px if I recall correctly). For training, it will group several samples and train with them at the same time, in "batches". For better generalization you want bigger batches (so more different images are feed at the same time). With a 3060 (non-Ti) you'll have 12GB of GPU RAM, with that I think you can run the default settings (network size, batch size and subdivision of batches) for the YOLOv4 model. If you want to go to 512px, you might have to increase the subdivision (create more subbatches) or reduce the batch size. If I recall correctly, you could find 3070 with less than 12GB of RAM, so in trying to purse faster training times (I'm not talking about inference, using the model to actually recognize something) you might not be able to train with a broader range of options that can improve your accuracy.

I'm training YOLO on a 2080TI works fine but YMMV. Waiting for a 4080(TI).

Relatedly someone linked a "all in one ML imaging software" called chaos vision or something, and the first thing on the linked page was "you probably need an Nvidia rtx 3090, or another Nvidia card with 24GB of memory"

I've tested a 'machine vision for image tagging' self-hosted service and it seemed reasonably responsive, CPU only, too - but I ran a pre-trained model for that.

I would wait for the 4000 series from nVidia, which should be this fall, and then making a purchase. The best bang for your buck will likely be a 4070, or a discounted 3080.

Reply on news.ycombinator.com