Hacker News Reader

Comment by bambax | Hacker News Reader

> I don't think I need to explain why it's a bad idea for a database to be so careless about data validation.

Well, loose typing can be extremely useful, and having a type of "ANY" would not replace it.

I have built recently an accounting reconciliation system to find discrepancies in data coming from a large variety of sources: some from proper database engines (MySQL MariaDB), but most from proprietary systems that export to CSV. It's amazing how corrupt data can become: dates that are invalid, numbers that aren't numbers, strings strings strings everywhere.

Being able to store the data into tables that have types, but can accept anything, is simply great.

View on HN

Your... solution to bad ETL data is to go "let's keep it this way"?

You can already "store whatever you want" in a serious database that respects types by default. It's called a blob or if you must, a text/varchar.

Yes! keeping it that way helps with traceability. The point is not to fix the data, it is to understand where corruption happens.

Are you saying that you have an application where you want the loose-typing-with-integer-affinity semantics for a column (or some other particular affinity)? It would be entirely reasonable to have a specific type for each loose-with-affinity variant. But I don’t think those should be the default.

> loose typing can be extremely useful

"Loose typing" enforced in a strict typing system can be useful in certain scenarios, but it is regrettable that it instead replaced the strict typing discipline for some time in software. Strict typing should be the default, because it is the most accurate description of data in the vast majority of cases.

You're using CSV, what did you expect? CSV was never meant for data exchange between two systems that do not know of each other's existence. Basically every CSV file is its own dialect.

I am not "using" CSV, I am forced to use it.

You are implicitly dismissing any future in which the auditors actually do their job. Is that quite safe?

> I don't think I need to explain why it's a bad idea for a database to be so careless about data validation.

...Meanwhile MongoDB being successful for years with no sign of decline.

Reply on news.ycombinator.com