Hacker News Reader

Comment by ismailmaj | Hacker News Reader

Could it be Mistral Large? This beats GPT-4 on my personal test.

I tried a bunch of my recent prompts to GPT-4 from daily use - this was often just slightly worse, sometimes slightly better. Fast too (tokens per second) while also not being overly wordy - very much appreciated that.

Refusals are a bit "I am just a language model"-y which GPT-4 has gotten away from. Also it's more refuse-y if I broach something rudely (which again I've found GPT-4 to have become much better at.)

Way better at everything than whichever Gemini I've been trying recently (can't tell for sure what I'm using when I use it.) But that one isn't even in contention for any use at all IME.

Overall it felt like I need to try it in daily use to work out if it's a contender with GPT-4 as a daily driver.

The clincher for me will be if the API to it is not so exorbitantly priced as GPT4, and if mistral can make using LoRAs economical.

Curious what your use case is that API tokens are actually significant expense at this time. I get it if you’re reselling a product but myself: abusively and naively messing around, I don’t see major prices as an end user. I sometimes hit the rate limit in ChatGPT, maybe like twice a week, but otherwise my monthly API bill in the last 6 months usually is in the dollars frame. Not tens or thousands of dollars, just dollars

Back when GPT4 came out mid last year my bills were slightly more dramatic (i am what you would probably call a semi heavy individual user) but they never surpassed $200 USD for a single month.

GPT4 is very cheap when you're hitting the api with common knowledge questions or reasoning about known things, and asking for short replies. It's very expensive when you're loading up the context with a ton of data and asking it to convert some data points into a narrative structure.

Oh, dude, you'll easily end up with $200 per hour if you are working on generating synthetic data...

It doesn't on mine it still is really wrong but compared to mixtral when you tell next that it did a mistake it corrects it to the right answer where mixtral was making things up that were even more wrong.

Reply on news.ycombinator.com