It depends on the gifted program. I was in one in high school and rather enjoyed it, though it was a pretty wide range of "gifted" (top 10% of my age cohort). I wouldn't characterize my experience as raising the the grade, but rather by diving into different material alongside different people. The material was ostensibly advanced/sophisticated sure, but it was categorically different from what would be encountered in the higher grades at the same "level."

A friend of mine was in a much more rigorous gifted program (through Stanford) and I think it was the best thing that ever happened to him. He had a very poor home life but he's phenomenally brilliant (he's one of a specific handful of people I've personally met who I use that term for). All he really enjoyed doing from the time he was 12 was reading math and physics books. Going through the gifted program put him on a path that exercised his talents in a way that he found personally very fulfilling. He ended up finishing undergrad at Harvard before the age most kids become sophomores, and then completed a PhD from Harvard before the age most people even begin one.

Then he went on to work for the NSA and, later, a hedge fund. Those things probably look soulless to a lot of people, but he's very happy.