Hacker News Reader

Grimm1on Avoiding bot detection: How to scrape the web without getting blocked?

And even bigger growth hack for a lot of companies would be having scraping protected by law.

Grimm1on Avoiding bot detection: How to scrape the web without getting blocked?

https://en.m.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn

That’s one of the bigger ones. Unfortunately recent events means scraping is still a gray area.

Well that's one of the more interesting things I think I'll see today! Neat.

Grimm1on Meta buys Within, the studio behind VR fitness app Supernatural

My quest 2 was like 299 and weighs 1lb and doesn't require a computer at all. The workouts especially HIIT and Boxing routines are more engaging to me in VR with my limited on hand equipment and I don't like being in a gym with other people, largely because I want to monopolize the equipment. I think you may be a bit outdated on what VR these days is like.

Grimm1on Meta buys Within, the studio behind VR fitness app Supernatural

I do a 30-40 minute HIIT or Boxing Routine 2-3 times a week in my quest 2. I keep meaning to get the silicone cover but it hasn't stopped me. The real issue is the foam does get sweaty and if I stop working out then decide I want to play a VR game that is not something you want to put back on after you've taken it off without drying it.

Grimm1on Grand jury subpoena for Signal user data, Central District of California

Even better then!

Grimm1on Grand jury subpoena for Signal user data, Central District of California

I mean I do. EFF, Wikimedia, ACLU

(Only EFF is really for this particular issue though)

Grimm1on GitHub Copilot available for JetBrains and Neovim

Yeah but if malicious intent is a concern you can just spin up a sandboxed instance to run the code to check first.

Really the thing is there's not way to ascribe correctness to a piece of code right, like humans fail at this even. The only "correct" code is like rote algorithmic code that has a well defined method of operation. And there's likely a lot more correct examples of that, like way more than you'd ever be able to poison.

You may be able to be misleading though by using names that say one thing but do another, but again you'd be fighting against the tide of correctly named things.

Grimm1on GitHub Copilot available for JetBrains and Neovim

In any training with code I've done, we've written a parser that validates against tree sitter grammars to make sure it's at least syntactically valid against some known subset of languages we're training on.

Grimm1on Ask HN: Senior Devs, what career have you moved on to avoid impeding doom?

I don’t intend to have any, wife I expect would be in a similar economic position to me

Grimm1on Microsoft has detected nation-state activity associated with NOBELIUM

I just got off the Destiny subreddit, why must you drag me back in!

They really do though, some RASPUTIN level AI just naming things.

Grimm1on Building a recommendation engine inside Postgres with Python and Pandas (2020)

Yikes, please tell me they were hired as a junior?

Obviously in your case some trivial SQL would have sufficed, which I feel like should have been caught in an interview but could be excused for a junior eng.

Also Pandas is not an amazing choice for data engineering tasks in general IMO, type conversion wonkiness is one reason I have questions about people who decide to use Pandas for legitimate data engineering tasks (as opposed to exploratory work or analysis).

Grimm1on Lambda School leaked documents show poor performance over the last two years

Man, in college I designed the curriculum for, led 7 other TAs, graded and lectured the introductory CS course with 300+ students for two semesters. Clearly I should have told them I was the teacher (Ha!) and should have been paid more than 23/hr but nope my title was Head TA while the professor, who admittedly I hold in high regard supervised to make sure things didn't go off the rails. Honestly it was a great learning experience but yeah sometimes I think about that and laugh to myself.

Grimm1on Ask HN: Senior Devs, what career have you moved on to avoid impeding doom?

2.4million is pretty easy to retire on (stock is expected to appreciate as well to account for taxes etc).

I could retire at something like 1.8m to be honest. My current style of living costs about 50k a year. Throw that into index funds live off a 4% withdrawal and I'm out and I move to a LCOL. That's 72k a year which is 20k some odd higher than median earning in the US. In most non metro areas that'd be a fantastic existence in which I no longer have to care about 99% of the mundane crap I do now.

I love working so I'd continue to do things but solely at my discretion so there'd still be some months where I bring in more external cash probably on contract.

Grimm1on Ask HN: Senior Devs, what career have you moved on to avoid impeding doom?

I moved from "regular" engineering to data engineering in the last few years which is closer to ML which I find very interesting but comprised of all the stuff that actually needs to be done before ML happens and moved up to "higher end" engineering from having worked entirely in startups to my first big corporate job where I make enough money to retire in 6 years if I want. Bonus is this company isn't a FAANG and morally I get to have a neutral to positive impact on the world but they pay roughly L5 Google level total comp.

If things go well, then if this whole industry goes south I'll be insulated.

Also not joking though blunt it may be, don't be in a European country where wages don't even approach our lowest end dev in the US.

BUT

I don't see the industry going south -- I see it being bifurcated where lower end devs who glue APIs together see their wages driven wayyyyyy down. Find a difficult niche that pays well and try to keep skills towards the top like 60th percentile and higher and your life won't change.

Grimm1on 20Y study finds little evidence religiosity leads to greater life satisfaction

Yes, hope and purpose are important. I like existentialism and cling hard to thought of being able to define that belief system myself and what purpose I have from it. That said as of late, after years I seem to be going through a bit of a dark night of the soul with that chosen system, to borrow more from religious terminology.

I guess one of the downsides of forming my own belief system versus religion is it's bedrock is just a wee bit younger and potentially less well formed as I've been learning. Back to the drawing board :P

Grimm1on 20Y study finds little evidence religiosity leads to greater life satisfaction

Honestly, and completely anecdotally, having been raised Catholic I would expect it to result in quite the opposite when so much of the doctrine is based on guilt. That said before anyone jumps on me, also anecdotally, I haven't found any greater life satisfaction by not being religious either. You do you.

The article says the study suggests there's not a really strong causal relationship in either way which surprises me a little but yeah I can see it given the experience.

And If I were to find religion suddenly again now I imagine it may only increase life satisfaction based on the fact I have no real community at the moment and that I think is probably a more interesting study.

How much has external community (i.e. not close friends, partners and families) breakdown occurred over the last 40 years and what is the effect on our life satisfaction? My grandfather was a part of a bunch of men's clubs etc last I checked most of those are floundering now.

Grimm1on What do people want in a co-founder?

Edit: I'll make a quick edit because the below sounded too aggressive to me after reading it again like two minutes later. Your point that not everyone takes unsolicited advice well because it feels like it undermines their own skill and knowledge is noted and I have run into people who think similarly to you, but I think it's helpful more than not to a wider group of people and it helped me so I will consider the thought in a given situation but I will likely continue to give the advice.

----

Right which is why I qualified it with "If you have a small set of useable features". Honestly, feeling shamed is your hang up, same with feeling demeaned. I've had that said to me and it motivated me and was helpful, because I had been holding back.

I guess we'll agree to disagree because I don't know you and I don't want to like armchair psychology here. But, I'm certainly not going to tiptoe around every word though just because someone somewhere may be offended, especially in this case where I believe most people would find it helpful.

Grimm1on Bid to unionize Amazon workers in New York nears milestone

As an alternative place workers on the board. Have them be elected by the workers. A lot of places in Europe apparently do this. There's your non union workers representation and brings parity to negotiating power as well.

Grimm1on What do people want in a co-founder?

First, at least in regards to my comment -- that's entirely reductionist. Second, I commented because it's one of the hardest things, in my opinion as an engineer myself, to overcome. You're interpretation of people wanting to seem like someone knows something they don't is at best uncharitable. My comment comes from a place of hopefully being helpful and giving someone a nudge for a thing that is definitely uncomfortable (showing your baby to the world) and showing some sense of camaraderie that a lot of us have been there.

Grimm1on PyTorch 1.10

Your mileage will certainly vary but I was able to eek out a lot more inference performance by exporting to ONNX, using a highspeed serving framework in that ecosystem and also relying on some computation graph optimizations that you can apply to the ONNX version using available community tools. Versus serving from PyTorch directly.

We were doing millions of inferences and we had a specific target of a couple thousand a second so a specific case for sure but that's my two cents.

Grimm1on What do people want in a co-founder?

In my experience that's a trap, if you have a small set of useable features, release it now and get feedback from real people.

Grimm1on Decreasing service fees on subscriptions to 15%

Suggesting that App distribution is even part of the consideration for why Linux didn’t catch on is kind of hilarious to be honest. Their app distribution methods being good or not had nothing to do with their lack of adoption and it’s kind of a non sequitur.

If I dropped their app distribution onto Windows it would also succeed there with the general mainstream public, in fact even more so, because of the garbage state for app management there.

Grimm1on Uber and Lyft don't save money, time, or the planet

Lyft saves me plenty of time so I'm not sure I can agree here. The alternative would be relying on regular cabs or the public infrastructure here in NY which has on multiple occasions proven to be less reliable for me. I'll gladly pay a little extra to not arrive at my location 40minutes to an hour late because of a service disruption in the subway. If we actually took all the money invested in transportation in the city and made things better, instead of I assume pocketing it, maybe I would have different thoughts, but for now these services make my life significantly easier.

Grimm1on A Meta-Scientific Perspective on Thinking: Fast and Slow (2020)

This is the issue I (and no doubt many others) have when I read pop science books from fields I'm not an expert in is the ability to discern whether their claims are based on solid foundations.

I read "Thinking: Fast and Slow" something like 7 years ago now and while I wouldn't be able to give you a direct account of the ideas I've taken away from it at this point, I wonder how much partially or completely incorrect ideas on thinking I internalized from it.

Obviously I can now adjust my mental model but pop sci still feels a bit like a mine field for the quality of thought I may be taking away from it. I just started "The Selfish Gene" and this post has me wary of the quality of it even though just like Fast And Slow it's fairly critically acclaimed.

I think if anything in the last few years I've found myself being highly skeptical of most claims and ideas (which is also a balancing act, but largely good) which wasn't necessarily the case when I was reading Fast and Slow instead committing a common sin and taking the fact the author was a well regarded practitioner in their field at face value.

I guess the point of my comment here is I find that without taking (or really having the) the time to follow every idea to it's foundations and given there are an incredible number of ideas being thrown around openly and with little barrier to entry having to rely on some type of illogical filter and tendency towards your own biases is required just to filter down to a smaller set of ideas that at least seem plausible. The issue with that as we see on a day to day basis is that most people's filter (including my own given the critique of this book) sucks and we wind up with a bunch of plausible but incorrect ideas being treated as fact and propagated throughout the world. It would be nice if finding some kind of objective truth didn't require turning seeking that truth into a life mission.

It's also something to be said that most people operate on these faulty models throughout life to great success by conventional standards so it's not like knowing some set of objective truths, if they're there in the first place (I'm not entirely convinced everything has an objective truth), is a pre-requisite to a happy, successful or fulfilling life -- they may even be the opposite. I just have always found that unpalatable and hope we find ways to think better.

Grimm1on More Americans are considering retirement

I mean I can generally negotiate higher than 20% bump every two years or so going to a new employer, not sure my incentive to stick around longer. My understanding is that’s prevalent throughout the industry

Grimm1on Women earned the majority of doctoral degrees in 2020 for the 12th straight year

US education fails boys significantly starting all the way back in elementary school. I knew multiple teachers in high school who talked about how the system fails boys and doesn't know how to engage them and extends especially to minority boys. I think it's a pretty poorly kept open secret. This article basically covers what I've been told for years by educator friends.

http://www.cpreview.org/blog/2021/6/why-are-we-failing-our-s...

Grimm1on Ethereum Services Are Centralized

Hmm yeah that is certainly interesting in the sense that it was easier to receive payment than it otherwise may have been in current financial systems. Easier to trace and such too I imagine if it was needed.

To your point about transparency and trust for general transactions, I can see how that will be useful but don't you lose the ability that most trusted entities have now which is to roll back the transaction or correct a situation if fraud has occurred, given what I understand to be the finality of a contract? That said, I'm personally not really worried about making a transaction over say VISA's network but I am concerned about the end party I'm interacting with and I feel like crypto in general doesn't do well to address that, which I almost feel like is the real issue. I don't think I understand why people feel like it's an issue if they have to transact over a bigger central authority, perhaps I'm just too far removed from the group of people not served by classical finance.

Grimm1on Ethereum Services Are Centralized

> I will grant you "simple", and maybe even "common"... but only because most people don't really understand decentralization and many of the ones who do don't care about it.

So assuming that this is mainly an issue of people not understanding versus understanding and not caring (though you said there’s some of that) how is the Ethereum community going to get the word out so that this becomes a mainstream success versus mainly for speculation as it is now?

A common issue and one that I personally feel crypto in general suffers from is that neat tech isnt a a criteria on which most people flock to use something so what are the reasons end users need to know to be like ah yup I will drop my regular finance interactions for this or heavily augment them with this?

Grimm1on ETL Pipelines with Airflow: The Good, the Bad and the Ugly

I think we've worked in very different jobs, in my case the analysts had a good idea of the underlying application data and often worked closely with both data engineering and regular engineering to understand it so they can make better analyses. They were quite competent in their own right, otherwise I wouldn't have given them control over the T which only made a net benefit to my life as reduced work.