Is Devin a new LLM? Perhaps equiped with code and deploy plug-ins? The comparisons against other LLMs would suggest so.
The real world eval benchmark puts Claude 2 way ahead of GPT-4, which doesn't sound right.
The real world eval benchmark puts Claude 2 way ahead of GPT-4, which doesn't sound right.
I'm guessing it's a fine-tuning of some existing LLM model or API, but this largely seems to be an "agent" and UI that includes some SWE like workflow coding to allow more complex requests to be asked than just an LLM could provide.