Grimm1

Not entirely, certain services are free forever and this reduction is one of them.

I agree it has generally been a sham for a long time. If we actually believed strongly in free markets we would have let the various financial institutions die in 2008. I personally believe smaller regional players would have stepped in and we may have had a stronger recovery after a worse initial depression as those players fill the gap.

Let's also see the real consequences of the rapidly increasing inflation too in the coming years from the last couple stimulus bills. I wouldn't be too quick to look at QE as a panacea.

Seems like we've been treating symptoms instead of addressing various root causes for a long long time.

Ah yup you are definitely correct, that is what I meant.

I get that I can run this as a SPA, but then don't I lose out on the prerendered HTML fetched from the server? That's what I mean by isometric with Node.

I partially judge the novelty of the problem I'm solving by the level of access to information about it I have. You realize you're on the edge of knowledge when you're deep down in the research papers and there's only a couple papers on the subject and you implement from there. The solutions you come up with from that tend to be the most rewarding and unfortunately relative to all the mundane work few and far between. I think I run into that type of problem a few times a year, if even.

I talk about encryption which cuts the confusion nicely even if it isn't a perfect replacement but I find lay people don't care about the general field of cryptography they care about privacy and from that, encryption. I've been able to get a few people onto signal for instance by going that route.

Unfortunately language isn't a static thing and is somewhat defined by the dominant usage of something over time (unless you're in the rare country that has fully prescriptive language). I had been using crypto as shorthand for cryptography for a long time but cryptocurrency quickly subsumed that. If I talk to any of my non technical friends, and even technical friends who don't touch cryptography, crypto = cryptocurrency. I think the war is lost here.

Congrats Kevin and Guru on the launch. Happy to see this on here, I was wondering when you’d do your Launch HN. These are real problems that a lot of data teams have and I’m glad to see this making it into the wild!

I see Deno getting a lot of play here, and I think rightfully so. The recent globally distributed chat example in like 20 lines of code is basically the level of accessibility I like seeing brought to otherwise very complicated distributed systems setups.

That said how is Deno being seen in relation to Node, iirc Deno is being made to correct on a lot of the issues Node originally had but I’m curious about adoption and how this is going to fit in the JS world where Node has become so cemented, what’s the killer feature to move those people over?

Or is that not the play, will this peacefully coexist?

I'd say many aggregators do offer value to the rest of humanity but I imagine there are probably some exceptions and also not all scraper services offer no value it's just different value to different people.

Some scraping services make their money by offering scraping services to companies for specific information and you could argue they provide value to other businesses that way, but not to the broader "rest of humanity".

So I'm not sure it's as simple as just "aggregator" good "scraping service" bad as value provided takes on many different forms, and that's what makes this difficult.

I guess it may come down to your take on what you think of middlemen, because they are all effectively middlemen in the data economy.

Edit: I was rereading your comment, in respect directly to the value added to the content, then yes maybe it is more clear that aggregators are in principle different because they do add that value where scraping services that sell the data do not offer any enrichment to the content creator. I personally think protecting content aggregators that republish the data to create visibility or other value for the content creator to the extent that they're not worried about being sued for that is probably a worthwhile thing to happen because of the net benefit to our ability to find information/content.

Heh at the end I think I may have been called out for my harsh response to the post being referenced at the start here on HN since it seemed to raise some hackles. Borrowing from other social media terminology here, "subtweeted" but by a blog post, not sure how I feel about that.

Honestly it almost makes me feel compelled to start writing more for the industry but to be honest, my opinion is most ideas are not great, or not novel, up to and including my own. So writing about them seems likely not to be beneficial. I guess you could argue I could let the industry be the judge of that though.

I asked before making a mistake, I also asked to do some light market validation. Try to keep the ad hominems down. If you can't make an argument against the point and have to attack me as a person, it's just validation. Most of my contributions here are in my upvoted comments like my op, which clearly a good amount of people share.

Generally speaking I don't want to submit most things, largely and because like this post I find most thoughts people have to be garbage, including my own, most things and people simply aren't interesting or useful.

I'd say most crawlers are looking to provide enriched value for content at their end use. Google is just an aggregator (the biggest by far) but other aggregators are looking to provide similar value.

I'm actually cool assuming there is implicit harm and no benefit, but by that logic we need to tear google down too. I'm cool making that trade but it has to be done equally.

If you can't make that trade then you've weighed the value provided by an organization like google to be more valuable than the copyright of these content creators and I want other players who may want to be able to challenge google to have the same protections and access google does to have a chance at providing the same value.

Personally I find it trite but whatever floats your boat.

This one in particular is not worthwhile on the front page of HN, that's my take. They're most definitely useful for beginners, or maybe people just learning about databases.

I'm not going to not post simply because you find it disagreeable, there are plenty of people here who seem to agree with me.

Blameless post mortems are great, for your team. I am not his team mate, and I don't really feel a kinship with every developer under the sun. And for what it's worth I don't blame this developer for anything. If anything I lament the institutions that failed them on the way to this point in time. To me this is a symptom of systemic rot.

First its not my "compiler errors" scenario it's the person who initially replied to me. Sure whatever, I don't think I ever insinuated I thought that was a good idea, it runs in parallel with the issue I have.

No I did, but since I disagree with your earlier point about how much existing knowledge they have it kind of by default means I disagree with what they took away from this incident.

It's also highly speculative so like I'm not going to go back and forth on it.

Needing a vendor to hand hold your likely highly paid dev seems like a bad fix to me.

Also not having an index isn't an error it can be a valid choice based on your situation and query load which is why people should know the situations when they're needed.

I think people should simply be better. A lot of people don't like hearing that though so usually I keep it to my private chats where people seem more willing to cop to that fact.

I know we disagree, I know you're going to continue disagreeing, I know I don't want to have the conversation.

No you've missed my point, the author seemingly didn't know that ForeignKeys applied indexes by default in MySql. It's not "Concept I didn't understand", clearly they're capable of understanding because they did after they ran into the issue. It's about not having had basic knowledge to begin with.

But he didn't see compiler errors, he caused monetary cost to his employer.

When I deploy something that unintentionally causes a large monetary bill to my employer, then yes I do believe that indicates a gap in knowledge so I don't in anyway believe I'm being uncharitable. Or and this would be worse, a lack of caring. (Which is not what I think happened here though)

I won't respond to your imposter syndrome bit I don't really think it's relevant to my point.

Cool, and if this was a case of bike shedding over something that hadn't just cost that early stage startup 5 grand I'd agree with you.

However regurgitating a platitude that everyone, including myself, learned when we tried getting our first business off the ground doesn't add much value here.

Had this been a Database with 10million rows it would have cost them 50k, and this is incredibly basic programming knowledge.

Basic proficiency is a far cry from worrying about best technical talent and not a particularly egregious ask.

Which really is a head scratcher. You'd figure especially as a startup seeing a 5k oopsie isn't really as acceptable. Mistakes do happen and I don't mean any shade to this particular person (they'll never make this mistake again) but as an industry the aggregate consequence of this is you have a lot of waste and stupid choices that then have to be cleaned up when more knowledgeable (read highly paid) people are introduced later on.

They'll have to clean up the mess which causes real business consequences that, and I've personally seen this, will directly impact bottom line and have no quick or easy solution to wiggle out of.

Maybe it's acceptable for products like this because the balance between good engineering and company health probably aren't as cut and clear but stuff like this always makes me sad because it's such low hanging fruit, it doesn't require any real effort, just basic curiosity around your job.

I feel like indexes are a pretty fundamental type of DB knowledge. In fact I'd say it's table stakes knowledge you should have if you're working with them. Further more, knowing that ForeignKeys typically apply an index to that column is also in my head basic knowledge. I'm sorry you got burnt, and congrats on learning a lesson, but you could have gotten the same knowledge by ever googling MySql ForeignKeys and saved yourself a headache.

In fact it's like a big bullet point near the top of the docs page.

"MySQL requires indexes on foreign keys and referenced keys so that foreign key checks can be fast and not require a table scan. In the referencing table, there must be an index where the foreign key columns are listed as the first columns in the same order. Such an index is created on the referencing table automatically if it does not exist. This index might be silently dropped later if you create another index that can be used to enforce the foreign key constraint. index_name, if given, is used as described previously."

I'm not entirely sure why buzz around "developer learns basic knowledge" has this on the front page.

I wound up not taking it because I had something that at the time seemed easier for me, but I negotiated a $100/hr contract on UpWork just fine looking for some part time work to hold me over on something. This was in the Data Engineering space fwiw.

Mmm I’ll go with that, to be honest I saw that LinkedIn had initially made a complaint under DMCA (which HiQ then got an injunction for) as well and given how the case played out I was uncertain to the extent the case was signaling that you may be waving certain rights by making some content publicly available with no gating like a sign up.

Ah yup, certainly. So this is the one I was mainly thinking of -- https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn

No but those are substantively different situations such that this exact thing is being argued in the highest courts of the US. It's not quite the cut and clear case you seem to believe it to be.

No? If you place information publicly on a website it's pretty much free game, no copyright violation, especially regarding user generated information. That's my take, but legally it's a gray area and it's still going back and forth in the courts (at least in the US) but for a while before a decision was vacated by the supreme court scraping publicly available information on a site was legally protected and seemingly inline with my thoughts on it.

Every time this comes up (like every 3 months on here) people just kind of digest it as fact without critically reviewing this. There are multiple posts in the wild that dispute Cal's claims and raise various issues with his benchmarks. Like always consider your work load, does it need a level of optimization? Yes? Then compare various performance options.

How many people spend a lot of time on social media? Billions, this is just providing a more immersive interaction for that. VR is only a tiny portion of that and I'm expecting AR and haptics and a whole bunch of other tech that enable more diverse sensory experiences to play a part here.

I don't really want "Meta" to own all of that largely because what made the internet successful was having no gatekeeper and a very low barrier to entry. Not that that's true anymore, but having no initial gatekeeper let people do crazy things for 30 years while the current gatekeepers were establishing themselves.

Organizational and as a part of that access control. One of the current critical issues I see in my own work is that we have a significant amount of downstream consumers of our data and each one needs access at a different layer.

This means someone may need access to raw data, data after it's been cleaned and put in the data lake, data in an intermediate state at some point in the transformation chain, pristine fully joined master records for some data in the warehouse. Some teams shouldn't have access to certain types of data, or data earlier and later in the stack.

There's not a central way to do access control at a fine grained (row in some cases, files in others etc) level across layers and multiple services/dbs in the data stack. At least no plug and play way from what I can see. I'll be in the midst of building that layer for an entire enterprise organization soon and I expect it to be enlightening.

I'm at the same time putting in some leg work to explore how common this problem is across other companies right now for obvious reasons.