Hacker News Reader

saigalon Show HN: We open sourced our entire text-to-SQL product

The autonomous car analogy is a good one. The technology is overall so far superior to a human (probably scrolling TikTok) driving but the moment it makes a mistake we remove the AEV which would be to to higher societal benefit.

Decisions should be made against an alternative, not against some fictitious perfect solution.

saigalon Show HN: We open sourced our entire text-to-SQL product

Couldn’t agree with this more. Exactly.

saigalon Show HN: We open sourced our entire text-to-SQL product

For companies that are willing to put in some effort, the self hosting option is a great one. There are certain use cases where this works now, and is already in production. These tend to be use cases with some constraints and don’t deal with very sensitive data.

saigalon Show HN: We open sourced our entire text-to-SQL product

we've encountered a lot of instances when people know SQL but just want a first draft of SQL to expedite the process. we see this a lot from data analysts too.

saigalon Show HN: We open sourced our entire text-to-SQL product

yes. some want BYOC solutions. others don't want to even be perceived as being used to train an LLM. not to mention CCPA, GDPR, etc etc etc.

lots of questions around what data is being sent to the LLM, or just schema.

saigalon Show HN: We open sourced our entire text-to-SQL product

The question was around row level security

saigalon Show HN: We open sourced our entire text-to-SQL product

Yes totally agree. You can easily sniff out products that are simple wrap of GPT

saigalon Show HN: We open sourced our entire text-to-SQL product

Search bar within the SaaS interface that allows user to ask data questions and returns back NL answer or specific cut of data

saigalon Show HN: We open sourced our entire text-to-SQL product

Yes. Correct.

saigalon Show HN: We open sourced our entire text-to-SQL product

We also encounter a lot of build vs buy conversations with businesses.

saigalon Show HN: We open sourced our entire text-to-SQL product

Enterprises are spending lots of time and money on this. The biggest issue that has slowed down sales cycle at this stage has been data governance. Most folks think it’s about accuracy or latency (which of course is an issue) but data governance can make this whole thing a non starter.

saigalon Show HN: We open sourced our entire text-to-SQL product

https://discord.com/invite/A59Uxyy2k9

discord invite in case anything comes up

saigalon Show HN: We open sourced our entire text-to-SQL product

there is a middle ground here. the most complicated queries will need the intel and business context of a smart data scientist. there are however so many types of queries where automation would make the world so much easier and allow more self-serve type data inquiries. too often the rhetoric around these topics is binary as in "it works" or "it doesn't work." in reality, there are certain use cases that work now and others that don't yet.

saigalon Show HN: We open sourced our entire text-to-SQL product

fantastic. let us know how it goes :-)

saigalon Show HN: We open sourced our entire text-to-SQL product

is there an easier way?

saigalon Show HN: We open sourced our entire text-to-SQL product

if you expect correct 99% of the time, you will be waiting for a very very very long time for most, except for the most constrained, use cases

saigalon Show HN: We open sourced our entire text-to-SQL product

"...what it expects them to do"

therein lies the nuance. some people expect to get a natural language answer back. others expect to get a data table back. others expect to get correct SQL back. this is why it's so important to understand the use case and not bucket everything together.

saigalon Show HN: We open sourced our entire text-to-SQL product

I agree with you. The larger point with text to SQL, however, is that it will not work if it is a simple wrap of an LLM (GPT or otherwise). Text to SQL will only work if there is a sufficient understanding of the business context required. To do this is hard, but with tools such as Dataherald a dev's life gets a whole lot easier.

saigalon Show HN: We open sourced our entire text-to-SQL product

Yes please do. We’d love your feedback and or to hear whether you see material improvement over what you have now

saigalon Show HN: We open sourced our entire text-to-SQL product

allow me to clarify.. Dataherald isn't intended for developers because they don't know SQL, it's intended for developers who want to build text to SQL into their products

saigalon Show HN: We open sourced our entire text-to-SQL product

We’ve seen demand from all types of SaaS applications where the user might need data— software that helps customer support staff answer data questions, CRM, payroll software, just to name a few.

saigalon Show HN: We open sourced our entire text-to-SQL product

"Anything more natural requires a lot of context and needs to solve ambiguity.

this is precisely why we created Dataherald-- to make it much easier to add that business context so that NL to SQL could actually be good enough to get into production

saigalon Show HN: We open sourced our entire text-to-SQL product

while i agree, there is clear demand for people to use natural language to SQL. we have tremendous conviction around the desire for natural language tools, but of course the technology and product need to deliver desired results.

saigalon Show HN: We open sourced our entire text-to-SQL product

i agree that there will be "early adopter" type use cases and others that might take a while (e.g. healthcare with hipaa compliance)

it is still the early days. goal is to give the developer tools to do this easier.

saigalon Show HN: We open sourced our entire text-to-SQL product

hmm not sure I understand the question

saigalon Show HN: We open sourced our entire text-to-SQL product

Tbh the original intention was to be the "data analyst" but we found over time (and with literally 100s of user conversations at small cos and enterprises) the embedded use case was more interesting and made for a better business, which was not at all what we expected.

saigalon Show HN: We open sourced our entire text-to-SQL product

"The agent is LLM agnostic and you can use it with OpenAI or self-hosted LLMs."