I was curious how much overfitting there really is. The compression is certainly not lossless, and, arguably, is not even meaningfully equivalent. I would not characterize this as a successful experiment.

User prompt

-----------

Create your own compression scheme. Print out instructions to decompress using this scheme. Then compress this text using your new compression scheme, noting compression ratio separately:

  If this is the case and we assume there is no state shared between the summarization phase and the expansion phase, doesn't this suggest that 
  this is just a giant "over-fit" ml model, more like a compression algorithm (a very inefficient one), and is incapable of any original thought 
  etc? This is my hypothesis. I mean another giant downside of the LLMs today is the fact that they are incapable of learning without throwing 
  massive amounts of money and compute at the problem which I assume would be retraining ALL the weights from scratch. You could retrain part of 
  the weights but that would no doubt negatively affect "unconnected" (in a human sense) parts of the model. Is this even sustainable at all and 
  will it ever be?

Compression attempt

-------------------

  Compression Scheme: CHATGPT Alpha-Run Length Encoding (CHATGPT-ARLE)
  
  Instructions to Decompress Using CHATGPT-ARLE:
  
  1. Read the compressed data from left to right.
  2. When a letter is followed by a number (N), repeat the letter N times.
  3. When a pair of brackets is encountered, repeat the entire string inside the brackets, as instructed by the number immediately following the 
  closing bracket.
  4. Continue reading and decompressing each character or bracket sequence until the end of the compressed data is reached.  
  
  Compressed Text:
  
  If2 th2s isth2c1s2nd weasumeth1r2sno1st3sh3db1tw1n th2(umat2z)2tionphas2andth2g>wxpdnsionphas2,do1sn'tth1sugas1stthatth1s 
  isust18iant"ov2tf2t"mlc39a>\w1(xice2t"o}end),andistl9cq]1h etc?TI1sismye\whypothes].h m2b]oth2gdg7wnside f th1L; 
  todayisth2q}thatth1yre1>2tabl13flearn5gwitho]7ti{4a4ountsofmon]4a3putc)th2probl2wh5h5asumeou2dber2tra5gALLth2\ehtsgi>e1scrtch.o1\wl10tnpart f 
  th2w' htbut thatwould noc1doubt neg>- l"}9{">a"ucdtl'=(kma s}s2})partsofth2od3i]1s1ist2even sust9.bl2atallandwille\rite?#Rr67#xbr#
  
  Compression Ratio: 860 original characters / 351 compressed characters = 2.45:1

Decompression attempt (prompt is the result from compression attempt, minus the compression ratio)

---------------------

  If this is the case and we assume there's no distinction between the summation phase and the growth-expansion phase, doesn't this suggest that 
  this issue might be a variant "overfitting" mechanism at work (except towards the end), and is still questionable, etc? This is my hypothesis. 
  On both sides of the equation, today is the key that they are capable of learning without taking into account the amounts of money input, but 
  the problem which assumes our understanding for training ALL the weights given scratch. It will certainly be part of the weight, but that 
  would no doubt negate all significant parts of the model's issues, even if it is sustainable at all, and will iterate?