These things have the reasoning skills of a toddler, yet we keep fine-tuning their writing style to be more and more authoritative - this one is only missing the font and color scheme, other than that the output formatted exactly like a research paper.
I have to say I am really underwhelmed. It sounds all authoritative and the structure is good. It all sounds and feels substantial on the surface but the content is really poor.
Now people will blame me and say: you have to get the prompt right! Maybe. But then at the very least put a disclaimer on your highly professional sounding dossier.
Prove it or fight me
They're optimizing for the sales demo. Purchasing managers aren't reading the output.
But I admit that I didn’t spend huge amount of time designing the prompt.
Basically how progress of everything ever looks like.
The next huge jump will have to again make a qualitative change, such as enabling AI to handle a new class of tasks - tasks that fundamentally cannot be represented in text form in a sensible fashion.
But I do expect the next qualitative change to come from this area. It feels exactly like what is needed, but it somehow isn't there just yet.
Perplexity, OTOH, has almost completely replaced Google for me now. I'm asking it dozens of questions per day, all for free because that's how cheap it is for them to run.
The emergence of reliable tool use last year is what has sky-rocketed the utility of LLMs. That has made search and multi-step agents feasible, and by extension applications like Deep Research.
Yet what's essentially "cat [62 random files we googled] > prompt.txt" is now being confidently presented with academic language as "62 sources". This rubs me the wrong way. Maybe this time the new AI really is so much better than the old AI that it justifies using that sort of language, but I've seen this pattern enough times that I can be confident that's not the case.
That's not a very charitable take.
I recently quizzed Perplexity (Pro) on a niche political issue in my niche country, and it compared favorably with a special purpose-built RAG on exactly that news coverage (it was faster and more fluent, info content was the same). As I am personally familiar with these topics I was able to manually verify that both were correct.
Outside these tests I haven't used Perplexity a lot yet, but so far it does look capable of surfacing relevant and correct info.
I boycotted ai for about a year considering it to be mostly garbage but I’m back to perplexifying basically everything I need an answer fo
(That said, I agree with you they’re not really citations, but I don’t think they’re trying to be academic, it’s just, here’s the source of the info)
No, these AI companies are burning through huge amounts of cash to keep the thing running. They're competing for market share - the real question is will anyone ever pay for this? I'm not convinced they will.
There are plenty of open questions in the AI space around unit economics, defensibility, regulatory risks, and more. "Will people pay for this" isn't one of them.
The obvious crutch of this new AI stack reduced go-live time from 3 weeks to 3 days. Well worth the cost IMHO.
The leadership of every 'AI' company will be looking to go public and cash out well before this question ever has to be answered. At this point, we all know the deal. Once they're publicly traded, the quality of the product goes to crap while fees get ratcheted up every which way.
There seems to be such variance in the utility people find with these models. I think it is the way Feynman wouldn't find much value in what the language model says on quantum electrodynamics but neither would my mom.
I suspect there is a sweet spot of ignorance and curiosity.
Deep Research seems to be reading a bunch of arXiv papers for me, combining the results and then giving me the references. Pretty incredible.
Every time I see a comment about someone getting excited about some new AI thing, I want to go try and see for myself, but I can't think of a real world use case that is the right level of difficulty that would impress me.
Many of the AI companies ride on the hype are being overvalued with idea that if we just fine-tune LLMs a bit more, a spark of consciousness will emerge.
It is not going to happen with this tech - I wish the LLM-AGI bubble would burst already.