Hacker News Reader

Has anyone tested OTP 170B from Meta? Seems like it's public

Just to get OTP up for inference would require a very large spend. To use GPT-NeoX (20B parameters) for inference requires 45GB of vRAM minimally. Its hard for me to imagine using a 170B model for fun somewhere unless one has a large GPU farm or lots of money.

GPT-NeoX-20B was specifically targeted to fit on A40s, A6000s, and a pair of 3090 Tis. Anything larger than that is going to be a real struggle for people who don’t own computing clusters to use.

How do you instruct BLOOM? Does it need fine-tuning?

have you considered SQL generation with two inputs:

1. data from the information_schema of the database

2. a natural language question

I've played around with gpt3 but not providing a full schema for the database is a glaring issue with accuracy of the SQL generated.

I've done something similar with gpt3 (codex) and had good results