Hacker News Reader

Baking weights in makes a lot of sense for inference speed and power efficiency and has the added benefit of putting many end-users on the hardware refresh treadmill.

View on HN

Yes, but how much would be a static Sonnet 3.5 be worth today? Its just about 2 years old. I'm not even going to ask about 3yo models like GPT4.

Okay, but the models today will be useful for a lot longer than sonnet 3.5. They're already more than capable to do nearly anything you throw at them given enough time and human assistance. The next step up is faster, cheaper and better user experience. I have only had two instances where I needed to reach for 5.6 sol and that only totalled around $2.7 in api costs.

I would imagine it would look something like this:

Ground breaking/novel research -> SWE -> day to day assistant conversations -> chat support bot...

The first one might not make sense but it gets them the pipeline for when it does make sense in the future. Plus for a lot of things using LLMs they'd be fine with older tech, especially if running it was even faster and cheaper.

Reply on news.ycombinator.com