Hacker News Reader

Parent Full thread

thulle

Does it even differentiate between lowercase and uppercase letters? Results for me were:

> How long is a piece of string, when the string is 4km long?

4096 meters.

> How long is a piece of string, when the string is 4Kim long?

4Kim

> How long is a piece of string, when the string is 4Km long?

4096 meters.

after that I got the "Something went wrong, please try again." error.

View on HN

When it comes to tokenization, yeah, it does differentiate. But its training set is certain to have many examples of different capitalizations and misspellings, so it can understand them, too. Which is usually what you want...

GPT-4 really takes this to eleven. For example, it can correctly parse and process this request: "Tll m shrt str bt ncrns; 10 sntncs. Wrt t l vwls prprl."

though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG

just run the 13b model 4bit quantized locally, it's already better than the 7b-8bit and you can turn down the temperature to 0 to get repeatable results.

Reply on news.ycombinator.com