Training NanoGPT on My Journal

Wise, and agreed

> Please remember that you can NEVER (and I hate using absolutes) find real happiness in the actions of others, or from any other external source.

I haven't really found "real happiness" yet, but I still get a lot of happiness from talking to all my friends and knowing that I am loved and appreciated <3

Even when I lose or fail to make one, there are always others that I can hang out with to feel better.

I think "real" happiness is highly subjective when there are just as valid views like "happiness is only real when shared".

At the end of the day, what makes you happy might be a miserable time for others.

Personally, finding happiness in the action of others is fine for me, it is expecting reciprocation that makes some bitter and that is just unneeded burdens.

It is too bad. Reading The Policeman’s Beard is Half Constructed it is pretty clear to me that it was one or both of:

1) nearly every grammatically accurate combination of a human-chosen lexicon being constructed by a computer and then heavily human-filtered

2) a human writing a simple program, getting the feel of the basic computer process and then repeatedly emulating it for artistic/comedic effect

My money is on 2, as it is more fun and less work, especially when you consider it was the early 80s. Still at least the work of a human-computer hybrid in some sense.

The least hypothesis is that the "good" version of Racter that supposedly composed the novel never existed, and that it was all a hoax made up by someone who wanted to bask in some publicity and sell the Racter that actually got released. As you say, the output is suspiciously good for something they insisted was accomplished on a home computer of the era and, oh no, they can't release the software that did it for Reasons... yeah, pull the other one, it's got bells on.

Also, this is interesting:

https://electronicbookreview.com/essay/constructing-the-othe...

> There is little evidence of The Policeman’s Beard’s process of production, likely due to its digital conception, as well as the normalcy of its creators. Chamberlain and Etter were two friends, with the former an amateur computer enthusiast and the latter employed as a computer programmer; neither felt it necessary to document production. Further, all of the referenced sources focus on Chamberlain, and Etter’s name has virtually disappeared from any record of activity since Racter. Even then, his name is mentioned only once, and between commas, in The Policeman Beard’s front matter. There does not appear to be any photographic evidence of Etter (while there is plenty of Chamberlain), nor evidence of additional contributions to any technological field. There is, frankly, no convincing evidence for Thomas Etter’s existence other than a few quotations attributed to him in news articles about Racter, and assurance from Chamberlain himself (correspondence with the author, who also confirmed that Etter is now deceased).

Yeah, if you only want coherent language, 1B parameters is more or less good enough. If you also want general context awareness and some basic world knowledge, you will need to up it another order of magnitude. Either way, this is nothing that you'd run on dated hardware. Smaller transformers can be useful for very specific task, but autoregressive LMs are just insane when it comes to hardware requirements.

I remember Andrej mentioning in [1] that is a challenging part, what materials/lectures/papers have you found or what ideas on what to do about that so far for that?

[1] https://www.youtube.com/watch?v=kCc8FmEb1nY