pklee

Very well deserved !! Amazing contributions !!

https://www.joelonsoftware.com/2001/07/21/good-software-take... - Reference. I read this for the first time. Thank you for sharing this.

This is pure gold !! Thank you so much eugene and gang for doing this. For those of them which I have encountered, I can 100 % agree with them. This is fantastic !! So many good insights.

Have you considered marker. Does a very good job of turning PDF into markdown. - https://github.com/VikParuchuri/marker

This is pretty cool. I tried with dolly and then I tried with T5-base, both of it did not give me result. It broke for me. Has anyone tried it ?

This is fantastic. This is what we do on our platform. + Copy and keep what worked and what did not so you can fine tune :)

If this is using OpenAI which it seems is what it is using, It is only sending column headers / column names. Not the data. If you are concerned about column names, you could also mask it on the way out and back in. If you are looking for an end to end database connect and query, please reach out to me.

They are decently good, I could not find major differences for the cases I was trying. The key is to control the temperature. Make sure it is low, otherwise the randomness increases tremendously. Infact you can feed the same input from openAI into NEOX and it generates results. There are many NEOX open playgrounds that allow you the control the temperature etc.

GPT3 model generates a SQL. You can sqldf on top of your data.table. We will be demo'ing at one of the events shortly. BTW, you could do somewhat similar with other LLMs such as GPTJ and GPT NEOX if you have worked with them

Very well said !! data is code

2 chapters in. Very good and makes you think. Truly impressive way of teaching how data is pivotal. The introduction reminds me of Peter Norvig's talk. I also like the point below of data is code, it is very profound.

Reminded me of this - https://www.youtube.com/watch?v=awG09liCMD4

The concept of a data-mesh is more of a business concept as opposed to tech. IMHO the idea being proposed is that of a conceptual data-server (not to be confused with database server) much like a HTTP server / Mail Server where people can engage with data as a first class citizen and create "data" products. This is especially true as we move from HTML to somewhat HDML (Hyper data markup).

By making data as the product (abstracting all the gory details), you are fundamentally engaging with data through a UI or an API. As you expose these products they become accretive while fundamentally encapsulating the domain expertise within them.

Radiolab had a similar post sometime back. That was about philippines - https://www.wnycstudios.org/podcasts/radiolab/articles/post-...

Of course. Assume somewhat of a big services company. There are many customer related problems that can be solved using products. Assuming such a product can be created. What should be the structure of the team that handles such products ? What should be the equity structure etc. Does that help ?

That was an awesome video. Thank you for sharing !!

What do you mean by dynamic scheams ?

This looks great. Thanks for sharing. I hope at some point this becomes a gate for RMF process.

I love the phrase "Enthusiasm is not a skill". I think it is great. I am not for or against in terms of your idea. I just wanted to comment on the phrase :)

Sorry I had meant this for @curiousperson23. Thanks

Would love to understand this more. pklee.hackernews@gmail.com

Yeah I am not just talking about analytics.. even basic merging across data-sources and simple visualizations based on the blended data and the ability to go across environments.

Yes completely. What I am seeing more and more is people try to shovel everything into big-data, but it becomes a nuclear powered can-opener which I am not sure is needed. The UI for this oscillates between excel with a macro or here is SaaS. Absolutely nothing in between.

Interestingly I was listening to this podcast yesterday - https://gimletmedia.com/shows/science-vs/76hgkmv/havana-synd... Analysis all the possibilities through interview with experts. Hope you find it interesting

I would say Scott Ambler. His book "building object applications that work" was amazing and it helped me understand how to design systems

I understand the distinction now :), thank you. Funny.

I am not sure about MS Access. From what little I know 2 people opening from network drive would mostly result in database being corrupt.

What a great guy and an amazing human being. RIP Professor :(.

This is absolutely valid. There is a comprehensive analytics platform that I work on where we went from (a litany of) REST services to our own domain language. It not only allowed us to go across databases (Graph / RDF / RDBMS) but even across processing engines (R/Python). This is really where I think the industry should go. I am sure someone will point out there is a great talk by Brian Kernighan on this on youtube.