Vanna.ai: Chat with your SQL database

How about a chat interface, where you correct the result and provide more contextual information about those columns?

Those chats could be later fed back to the model and ran a DPO optimisation on top

Billions wasted on a pointless endeavor.

10 years from now folks are going to be laughing at how billions of dollars and productivity was flushed down the drain to support Microsoft Word 2.0.

AI is a bubble. Do yourself a favor and short (or buy put options) the companies that only have "AI" for a business model.

Also short Intel, because Intel.

Hmmm, no. Altavista was the go-to search engine at the time (launched 1995), and was a crawler (i.e. not a curated catalog/directory) based search. Lycos predates that but had keyword rather than natural language search.

Google didn't launch until 1998.

And basically when you next write queries, it just auto completes for you. This would improve the productivity of the analysts a lot. With the flexibility of them being able to tweak the query. Here if something is not right, the analyst updates. The Copilot AI keeps learning and giving weights to recent queries more than older queries.

Unlike the previous solution where if something breaks, you can do nothing till you clean up the ETL and redeploy it.

Instead of Herculean data infra projects, this is a good time for figuring out new policy abstractions, and finding more productive divisions of labor between different days stakeholders and systems. Machine-friendly abstractions and structure are tools for predictable collaboration and automation. More doing, less waiting.

More practically, an increasing part of the Louie.ai stack is helping get the time-consuming quality, guardrails, security, etc parts under easier control of small teams building things. As-is, it takes a lot to give a great experience.

So yes, I can understand if there is incentive for the startups to invest in Data Engineers to make well maintained data models.

But I do think, the most important value here is not the chatgpt interface, it is getting DEs to maintain the data model in a company where product/biz is moving fast and breaking things. If that is done, then existing tools (Power BI for instance has "ask in natural language" feature) will be able to get the job done.

The google moment, the other person talks about in another comment, is where google or 1998 didn't require a webpage owner to do anything. They didn't need him/her to make something in a different format. Use specific tags. Use some tags around key words etc. It was just "you do what you do, and magically we will crawl and make sense of it".

Here unfortunately that is not the case. Say in a ecom business which always delivers in 2 days for free, a new product is launched (same day delivery for $5 dollars), the sales table is going to get two extra columns "is_same_day_delivery_flag" and "same_day_delivery_fee". The revenue definition will change to include this shipping charges. A new filter will be there, if someone wants to see the opt in rate for how many are going for same day delivery or how fast it is growing. Current table probably has revenue. But now revenue = revenue + same_day_delivery_fee and someone needs to make the BO connection to this. And after launch, you notice you don't have enough capacity to do same day shipping, so sometimes you just have to return the fee and send it as normal delivery. Here the is_same_day_delivery_flag is true, but the same_day_delivery_fee is 0. And so on and on...

Getting DE to keep everything up to date in a wiki is tough, let alone a BO type solution. But I do hope getdot.ai etc. someone incentivizes them to change this way of doing things.

I am not chasing this rabbit. Someone else will almost certainly catch it first. For now, this is a fun toy I enjoy in my free time. The moment I try to make money with it the fun begins to disappear.

Broadly speaking, I do think this is approximately the only thing that matters once you realize you can put pretty much anything in a big SQL database. What happens when 100% of the domain is in-scope of an LLM that has iteratively optimized itself against the schema?

Some findings after I played with it awhile:

- Langchain already does something like this - a lot of the challenge is not with the query itself but efficiently summarizing data to fit in the context window. In other words if you give me 1-4 tables I can give you a product that will work well pretty easy. But when your data warehouse has tens or hundreds of tables with columns and meta types now we need to chain together a string of queries to arrive at the answer and we are basically building a state machine of sorts that has to do fun and creative RAG stuff - the single biggest thing that made a difference in effectiveness was not what op mentioned at all, but instead having a good summary of what every column in the db was stored in the db. This can be AI generated itself, but the way Langchain attempts to do it on the fly is slow and rather ineffective (or at least was the case when I played with it last summer, it might be better now).

Not affiliated, but after reviewing the products out there the data team I was working with ended up selecting getdot.ai as it had the right mix of price, ease of use, and effectiveness.

But sure, why not!

- Generate SQL

- Generate optimized SQL

- Fix query

- Optimize query

- Explain query

Disclaimer: I am the solo developer behind it.

It's an open source BI tool that does just that.

I am a fan of a semantic layer (I made an open source one but there are others out there), and think having AI talk to a semantic layer has potential. Though TBH a good UI over a semantic layer is so easy it makes the natural language approach moot.

https://github.com/totalhack/zillion

old demo here: https://gist.github.com/lostmygithubaccount/08ddf29898732101...

planning to finish it...soon...

NLQ is a somewhat new feature for us, so we don't have a great library to pull from for RAG. Experimenting, I found that having a few-shot examples with some CoT (showing examples of chaining attributes retrieval) sprinkled around did help a lot.

Even still, some queries come out quite ugly, but still functional. I'm thankful that DuckDB is a beast when tackling those :D

> Did you folks experiment with building an Agent-like interface that asks more questions before the LLM finally answers?

That's something I want to figure out next:

1) try to check if a generated query would work but would generate absolutely junk results (cause the model forgot to check something) and ask to rephrase

2) or show results (which may look "real" enough), but give an ability to tweak the prompt. A good example is something like "top 5 products on Cyber Monday" <- which returns 0 products, cause 2024 didn't happen yet, and should trigger a follow up.

We built something similar to query DB. Created two versions, one of which was agent based that had a maker-checker style of generation. Basically one generates and the other checks its correctness and if objective has been acheived.

The accuracy improves in the agent driven framework, at the cost of latency.

I'm always suspicious that is just because RAG is so much more accessible (both compute wise and in terms of expertise required). There's far more profit in selling something accessible to the masses to a lot of people than something only a niche group of users can do.

I think most people who do actual fine tuning would still probably then use RAG afterwards ...

Your boss asks you to look into something, and you do it through a combination of structured and semantic research. Perhaps you get some books that look relevant, you use search tools to find information, you use structured databases to find data. Then you synthesize it into a response that's useful to answer the question.

People say RAG is temporary, that it's just a patch until "something else" is achieved.

I don't understand what technically is being proposed.

That the weights will just learn everything it needs to know? That is an awful way of knowing things, because it is difficult to update, difficult to cite, difficult to ground, and difficult to precisely manage weights.

That the context windows will get huge so retrieval will be unnecessary? That's an argument about chunking, not retrieval. Perhaps people could put 30,000 pages of documents into the context for every question. But there will always be tradeoffs between size and quality: you could run a smarter model with smaller contexts for the same money, so why, for a given budget, would you choose to stuff a dumber model with enormous quantities of unnecessary information, when you could get a better answer from a higher intelligence using more reasonably sized retrievals at the same cost?

Likewise, RAG is not just vector DBs, but (as in this case) the use of structured queries to analyze information, the use of search mechanisms to find information in giant unstructured corpuses (i.e., the Internet, corporate intranets, etc).

Because RAG is relatively similar to the way organic intelligence conducts research, I believe RAG is here for the long haul, but its methods will advance significantly and the way it gets information will change over time. Ultimately, achieving AGI is not about developing a system that "knows everything," but a system that can reason about anything, and dismissing RAG is to confuse the two objectives.

I think that might be a bit more positive than I would be. Broadly speaking, I think you could say that the downsides of the legacy technology in question aren’t larger than the collective switching costs.

But I’d definitely agree that when something has been around that long, it’s prob not all bad.

8/10 projects I look into don't have any indexes set up.

Use of ORMs with little thought into lazily loaded relations lead to 100s of queries being done per request.

It's pretty mad. Do not underestimate the propensity of a developer to stick to the only tool they know how to use. Unfortunately ORMs like Eloquent make it way too easy.