I don't put a lot of stock on evals. many of the models claiming gpt-4 like benchmark scores feel a lot worse for any of my use-cases. Anyone got any sample output?
Claude isn't available in EU yet, else i'd try it myself. :(
Claude isn't available in EU yet, else i'd try it myself. :(
The one for chat: https://www.anthropic.com/claude-ai-locations
The one for API: https://www.anthropic.com/supported-countries
The latter has Norway in it, while the former does not. One wonders why.
At some level of accuracy and consistency (human order-of-magnitude?), the pricing of the service should start approaching the pricing of the human alternative.
And first glance at numbers, LLMs are still way underpriced relative to humans.
It would be an ironic thing that it was open source that killed the programmer; as how would they train it otherwise?
As a scientist, should I continue to support open access journals, just so I can be trained away?
Slightly tongue in check, but not really.
Too little relevant training data in niche, state of the art topics.
But to the broader point, isn't this progress in a nutshell?
(1) Figure out a thing can be done, (2) figure out how to manufacture with humans, (3) maximize productivity of human effort, (4) automate select portions of the optimized and standardized process, (5) find the last 5% isn't worth automating, because it's too branchy.
From that perspective, software development isn't proceeding differently than any other field historically, with the added benefit that all its inputs and outputs are inherently digital.
On the people-direction side, I expect the span of control will substantially broaden, which will probably lead to fewer manager/leader jobs (that pay more).
You'll always need someone to do the last 5% that it doesn't make sense to data engineer inputs/outputs into/from AI.
I do however wonder, at what point do I just describe the hypothesis, point to the data files, and have it design an analysis pipeline, produce the results, interpret the results, then suggest potential follow-up hypotheses, do a literature search on that, then have it write up the grant for it.
Programming became high-level programming (of compilers) became library-glueing/templating/declarative programming... becomes data engineering.
If science was reproducible form articles posted in open access journals, we wouldn’t have half the problems we have with advancing research now.
Slightly tongue in check, but not really.
Programmers (specifically AI researchers) looked at their 300K+ a year salaries and embraced the idea of automating away the work despite how lucrative it would be to continue to spin one's wheels on it. The culture of open source is strong among SWEs, even one's who would lose millions of unrealized gains/earnings as a result of embracing it.
Artists looked at their 30K+ a year salaries from drawing furry hentai on furaffinity and panic at the prospect of losing their work, to the point of making whole political protest movements against AI art. Artists have also never defended open source en mass, and are often some of the first to defend crappy IP laws.
Why be a luddite over something so crappy to defend?
(edit to respond)
I grew up poor as shit and got myself out of that with code. I don't need a lecture about appearing as an elitist.
I'm more than "poking fun" at them - I'm calling them out for lying about their supposed left-wing sensibilities. Artists have postured as being the "vanguard" of the left wing revolution for awhile (i.e. situationalist international and may 68), but the moment that they had a chance to implement their tactics in the art world (open source AI art), they shunned it and cried and embraced ludditism.
Compare this to the world of AI right now. AI has somehow "legally circumvented" copyright laws and we are living in a de-facto post-copyright world. Huggingface and Richard Stallman as an entity/community and individual have done more to democratize access to and give the poors real access to social and economy mobility than any artists have done in the last 10 years, anywhere in the entire world.
You should embrace shit jobs going away, especially in a world where the speed to "re-skill" is often on the orders of hours when AI is involved. I am pointing out that the well-paid AI professional had much to lose and embraced losing it anyway, while the furry artist acted greedily over their pretty awful situation.
Wow who would've thought a large income allowed you to take risks and embrace change?
Imagine being a copywriter for 25 years, on 30k, paying a mortgage, running a car, feeding a family, trying to save on what's left... And all your clients dry up. You've got no other skills, you invested your career in copywriting. You don't have the savings to pivot and your kids need new school uniforms now, not when you reskill to a new career.
You lost your clients. Now your home. Maybe your wife and kids too.
Money is a buffer from risk most don't have.
I hope you never feel this and get to keep the luxury of poking fun at other people for being risk averse without the buffer. Maybe bring some compassion to the table tho? Furry art or copywriting, it isn't anyone's place to judge the merit of the income.
I'm currently in EU and I have access to it?
https://www.anthropic.com/claude-ai-locations
Perhaps you meant Europe the continent or using a VPN?
edit: They seem to have updated that list after I posted my comment, the outdated list I based my comment on: https://web.archive.org/web/20240225034138/https://www.anthr...
edit2: I was confused. There is another list for API regions, which has all EU countries. The frontend is still not updated.
I was just able to sign up, while not being able to a few weeks ago.
But it still errors out when trying to sign up from Germany:
That's really weird, I just signed up with no issues and my country together with some other EU countries was listed. Now when I try to signup a new account, it says that my region is not supported.
I still have the sms verification from them as proof.
Same, although they are helpful for setting expectations for me. I have some use cases (I'm hesitant to call them evals) related to how we use GPT for our product that are a good "real world" test case. I've found that Claude models are the only ones that are up to par with GPT in the past.