Hacker News Reader

Comment by foundval | Hacker News Reader

You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon.

If you want to learn more, there is a writeup at https://wow.groq.com/now-available-on-groq-the-largest-and-m....

(disclaimer, I am a Groq employee)

View on HN

We also added Llama 3.1 405B to our VSCode copilot extension for anyone to try coding with it.

Free trial gets you 50 messages, no credit card required - https://double.bot

(disclaimer, I am the co-founder)

would be great if there was a page showing benchmarks compared to other auto completion tools

Groq's TSP architecture is one of the weirder and more wonderful ISAs I've seen lately. The choice of SRAM in fascinating. Are you guys planning on publishing anything about how you bridged the gap between your order-hundreds-megabytes SRAM TSP main memory and multi-TB model sizes?

There is a lot out here.

I gave a seminar about the overall approach recently, abstract: https://shorturl.at/E7TcA, recording: https://shorturl.at/zBcoL.

This two-part AMA has a lot more detail if you're already familiar with what we do:

https://www.youtube.com/watch?v=UztfweS-7MU

https://www.youtube.com/watch?v=GOGuSJe2C6U

Thanks!

You can chat with all these models for free and ultra-low latency using this hosted website https://nat.dev/chat for free by GitHub Founder

Just checked it out. Is pay-as-you-go API access available at all? It says 'Coming Soon'

https://console.groq.com/settings/billing

I think you answered it yourself? It’s coming soon, so it is not available now, but soon.

It's been coming soon for a couple of months now, meanwhile Groq churns out a lot of other improvements, so to an outsider like me it looks like it's not terribly high on their list of priorities.

I'm really impressed by what (&how) they're doing and would like to pay for a higher rate limit, or failing that at least know if "soon" means "weeks" or "months" or "eventually".

I remember TravisCI did something similar back in the day, and then Circle and GitHub ate their lunch.

I've found Bedrock to be nice with pay-as-you-go, but they take a long time to adopt new models.

And twice as expensive in comparison to the source providers’ APIs

405B is already being served on WhatsApp!

https://ibb.co/kQ2tKX5

How do you get that option?

And available via poe:

https://poe.com/s/LCAyUbAgUx8UcVMhM3Re

At what quantisation are you running these?

Reply on news.ycombinator.com