cricketlover

there's kanban

Do we need another language for querying relqtional data? Why not just use SQL?

I understand what you're saying, but in the example for accounting, we can also solve this problem using NoSQL. Because the most important feature we're talking about there is transaction support. Similarly, schema-on-write can be provided by a library.

To me it seems like NoSQL works better when there is less to normalize, which is the case with microservices. Those services struggle with the support for a distributed transaction when they have to make a distributed transaction. This problem will be very easily solved in SQL (assuming its not shared to completely denormalize everything for performance).

Note that this normalization problem also shows up in schema-on-write. If multiple people are contributing to a schema from different teams, then it will become hard to maintain a schema-on-write.

Great work and it is also something I've been trying to build. Is this open source? Any plans on sharing the high level design?

Good post but given my lack of experience, I could only follow about half of it.

What would be a good place for me to understand how Query Processing Engines excute and optimise in general? I want to see and understand actual system calls behind them.

Agree. Very unclear why they won't simply use a secure socket or why a user space tunnel will be needed.

I surmise that the reason might be that a user space tunnel might be faster (like maybe they can do UDP over TCP or something to gain speed improvements).

Good post nevertheless.

Anybody with an archive link? These days my only social media is hackernews.

How can we prepare for such a future? And secondly, the most important thing in software is not the actual writing of code, but making sure that it solves the business problems, makes the right trade offs, is maintainable, testable and bug free. 90% of the time is not being spent writing code.

Yes. LLM can write some pieces of code. Yes, software can write some code. But can it maintain a million line codebase, can it prioritize issues, can it commit to timelines, can it talk to other people to resolve the ambiguities, can it make the trade offs, can it push to production, can it debug in production?

None of this actually involves writing code. At best, LLMs can take out interns and SDE1s but not more than that (think aechitects, staff plus roles)

Pardon my ignorance but I was hung up on this line.

> Out-of-sync document stores could lead to subtle bugs, such as a document being present in one store but not another.

But then the article suggests to upload synchronously in S3/DDB and then sync asynchronously to actual document stores. How does this solve out of sync issue? It doesn't. It can't be solved is what I'm thinking.

> Data, numbers

How much data are we talking about?

Great post. I was asked this question in an interview which I completely bombed, where the interviewer wanted me to think of flaky networks while designing an image upload system. I spoke about things like chunking, but didn't cover timeouts, variable chunk size and also just sizing up the network conditions and then adjusting those parameters.

Not to mention having a good UX and explaining to the customer what's going on, helping with session resumption. I regret it. Couldn't make it through :(

I would love to know how some of these APIs work internally.

How fan we capture unencrypted packets from the network? I thought you had to run tcpdump or something like that to be able to do that. But you won't be able to run tcpdump if you don't have access to the interface (source or destination), no?

won't there be more noise while predicting just 20s in advance? The longer the duration, the less effects we will see of temporary events like network blips etc. no? sorry I'm new to software engineering and just trying to learn.

I went through the post and I have absolutely no clue what this person is talking about. But I want to be in a place where I can understand what the person is saying.

How can I reach that point? I was lost at quantized, could understand bit packing, and was even more lost when the author started talking about things like Hamming Distance.

Please help me out. I want to grow my career in this direction.