I would be surprised if GPT-3 uses 700GB of RAM. It may be true, I don't know. But I am running 70B parameter models (quantized to 5 or 6 bits, biggest is 48GB loaded) on my 64GB Mac M2 Max Studio now and they are usable and the machine is still usable too. With an M2 Ultra and 192GB of RAM I imagine you could do a lot more.

I'm not arguing that these models hold up against GPT 3.5 and I still use GPT 4 when it matters. But they work and it's more like the difference between Premier League & Division 1, rather than PL & a five-a-side team from Bracknell.

Even a few years ago I could not have imagined this.

Given the pace of work on optimisation and my assumption that the M3 Studio I buy next will probably have 256GB of RAM at much the same power levels as I use now, it seems eminently possible it's a year or two away.