Finetuning LLMs on a Single GPU Using Gradient Accumulation

lightning.ai

104 pointsashvardanian3 comments

This won’t help if your model won’t fit on a single gpu right? So I’m this example your model has to be under 16gb if you memory?

I thought Raschka was a tenured professor. Did he leave academia?

From https://sebastianraschka.com/

"I used to hold a position as an Assistant Professor of Statistics at the University of Wisconsin-Madison (on a tenure track from 2018-2025). However, with a heavy heart, I recently resigned in 2023 to concentrate fully on my work at the Lightning AI startup, which I had joined in January 2022."

Reply on news.ycombinator.com