subsaharancoder

A friend of mine wanted an ETL (SQL Server to BQ for analysis and dashboarding) set up and I ended up stumbling across Airflow. I spun up two VMs on GCP, one for Airflow and the other for the Postgres DB to store the metadata.

- A few things I've noticed is Airflow generates a tonne load of logs that will fill up your disk quite fast. I started with 100GB and I'm now at 500GB, granted disk space isn't expensive, but still even with a few DAGs i'm surprised at how quickly. Apparently you need a DAG to run to clear those logs but I was too lazy so I just purge the logs using a cron job.

- The SQL Server Operator is buggy, I filed an issue with the Airflow team but I had to do some hacky stuff to get it to work.

- Even with a few DAGs, Airflow will spike the CPU utilization of the VM to 100% for X minutes (in my case about 15 minutes) which is quite interesting. My tasks basically query SQL Server -> dump to CSV (stored on GCS) -> import to BQ.

- My DAGs execute every hour, and if Airflow is down for X hours and I resolve the issue, it will try to run all the tasks for the hours it was down which isn't ideal because it will take hours to catch up. So I've had to delete tasks and only run the most recent ones.

Granted my set up is pretty simple and YMMV, but Airflow has done what it needs to do albeit with some pain.

Intersectionality at play here, your son is part of the "oppressor" class and is essentially viewed as beneficiary of privilege by virtue of his melanin content, therefore he is not allowed to say, do or question certain things. It's a sad state of affairs.

FYI I'm an immigrant from Africa with 3 daughters but I've instilled in them the values I was raised with, that everyone is deserving of respect, dignity and are precious in the eyes of God.

One CNN is bad enough, CNN+ is just traaash on steroids!

>We've been killing each other for thousands of years for far more trivial reasons than getting 18 years of financial obligations.

Yes we have, and the repercussions of those acts are still being felt to this day with calls for justice, reducing human worth to purely a "financial obligation" to be avoided at all costs makes us lower than beasts roaming the Serengeti

> All scientific evidence suggests the nervous system is not developed enough to experience anything resembling consciousness or to feel pain until 20 weeks at the earliest.

Bold claim with no data to back it up and yet a cursory search on Google shows how false your claim is:

1. https://flo.health/pregnancy/pregnancy-health/fetal-developm.... 2. https://www.whattoexpect.com/pregnancy/fetal-development/fet...

Is feeling pain or being conscience the standard upon which we determine the worth of someones life? If yes then please head over to your nearest hospital and pull the plug on all the comatose patients.

Basically let's address a past injustice by doing unjust acts in the name of "leveling the play field" and if anyone objects to it we'll just accuse them of what we are trying to do, it baffles the mind how this kind of thinking has bled into every sphere of society.

> Abortion as soon as you discover you're pregnant is just a pill and quite easy.

You mean the intentional killing of a human being is "easy", I read this and I know society is doomed.

How is this different from Pegasus https://www.saaspegasus.com/ which is a fully basked SaaS?

The Art of Eloquent Hypocrisy by Barack Obama

Getting started with Laravel on Mac required me to fire up Docker desktop which I don't particularly care for but I proceeded because I like to try out new things, so I followed the documentation https://laravel.com/docs/9.x#your-first-laravel-project:

- Fired up Docker desktop - ran curl -s "https://laravel.build/example-app" | bash and "cd example-app" - typed "./vendor/bin/sail up" and got an error "no such file or directory: ./vendor/bin/sail" - Deleted the example-app and ran curl -s "https://laravel.build/example-app" | bash and "cd example-app" - typed "./vendor/bin/sail up" and got another error "no configuration file provided: not found"

By this time my MBP fan sounds like an Airbus A380 getting ready to takeoff and I'm already frustrated with the framework, not sure what I'm missing!!

> Amazon uses rating and calibration mechanisms to learn who you think as a manager are your lowest performers.

Meta also uses calibrations where I'm told managers present their direct reports performance reviews and essentially "duke" it out with other managers.

Disclaimer: I work at Meta but I'm not a people manager

> Wow, so it's basically like you are a new hire when you transfer? Do you mind saying which FAANG or is the experience you had simply norm at all of them?

The Big G. Your past performance primarily only influences the decision on whether you have a good track record of delivering, but once you transfer you essentially reset the clock with the new team and manager and the expectations might be completely different for whichever reason, it could be personality or org structures etc. I don't know if my experience is the norm at all of them since I haven't worked at all the FAANGs, but as I said it's important to "read the tea leaves" early, and if the new team doesn't feel right make your opinions known early and plan an exit before you come a statistic.

Worked at a FAANG in one team for 3.5 years (got promoted and had good reviews), transferred to a new team and got PIP'd by new manager and eventually fired within a year of transferring. One of my workmates clearly told me that my chances of getting out of the PIP were next to zero but by then it was too late.

Best advice I can give someone who gets on a PIP is don't wait or work the process, just make an exit plan and act on it (interview quickly) and own your destiny. Also if you've been a stellar performer and you move to a team where things aren't working out within the first 3 months, transfer quickly (go to HR and claim mental and emotional wellbeing) before your performance reviews get trashed and then it's impossible to transfer.

So did Elizabeth Warren, what's your point?

I worked at AMZN and this perfectly captures my experience there with those weekly reviews. I once set a project I was managing as "Red" and had multiple SDMs excoriate me for apparently "throwing them under the bus" even though we had missed multiple timelines and were essentially not going to deliver anything of quality on time. I don't miss this aspect of AMZN!

I thought mice studies don't directly translate to humans..

>You can access the data in ClickHouse at https://github.demo.trial.altinity.cloud:8443 (user=demo, password=demo). This is a public instance running on Altinity.Cloud.

All I get is a white screen with the word "Ok"

> It creates a Hunger Games mentality within teams and makes doing anything that actually matters virtually impossible, generating an infinite sequence of half assed 6 months projects that get systematically abandoned as soon as the people responsible manage to get promoted or switch teams.

That's a bit of an over dramatization, PSC is just peer feedback, and is very similar to Perf reviews at Google as well as other large SV tech companies. Having done both I didn't experience this "Hunger Games mentality" you described.

Many people don't realize that with the 2020 lockdown and next to zero face to face transactions happening, platforms like FB Marketplace provided an opportunity for many people to set up businesses and generate income. I understand the angst people have with FB, but there's a bigger world out there beyond our keyboards.

A lot of people, many of them home based businesses, also rely on FB Marketplace as a primary source of income.

>How it would have hurt to get a vaccine?

Notice how easy it is to conveniently jettison the science + data and just demand that people "tow the line or else.."

>Yeah, and it didn't help when during the BLM protests, when social distancing was being trumpeted in all contexts, a number of doctors (can't remember if the CDC endorsed it or not) came out and said the massive crowds at the protests were acceptable because "race issues are health issues too" or something to that effect. That's obviously hypocritical to anyone with half a brain

Here's the letter https://drive.google.com/file/d/1Jyfn4Wd2i6bRi12ePghMHtX3ys1... This quote is mind boggling "Even so, we continue to support demonstrators who are tackling the paramount public health problem of pervasive racism."

> Young people (say under <20, those that actually don't use FB[1]) have no need for FB in the first place - they see their friends every day in school, in parascholar activities and social gatherings.

Yeah..2020 and the better part of 2021 might have something to say about that

To travel to some countries now you need a negative PCR COVID-19 certificate for a test conducted within 96 hours prior to travel. I'm hoping the amazon one will suffice.

I worked at AMZN for about 2 years and one thing that I found rather interesting and quite disturbing is how people regurgitate the leadership principles to drive points home and sometimes to bash someone into submission. It was not uncommon to hear someone literally use 2-3 principles in a sentence to emphasize something e.g. "Demonstrate your bias for action and earn the trust of the people you are working.." Funny thing is when I interview at other companies I always use the principles as a reference to answer the questions :-)

>retarded dead-end boondoggle of a device

Zune music player?

Context!!..Context!!..Context!!

"The art of the rewrite" by Uber Leadership could be an interesting book. In 2019 When AB5 was passed another rewrite was done in order to make the case that drivers were not employees but independent contractors and teams were assembled into one "zone" working like crazy to make some major changes for one market (the fear was that AB5 would be the start and all other states would follow with similar laws). The funny thing is during multiple meetings people would ask if Lyft was doing anything as radical as Uber and AFAIK the answer was no. FFWD to 2021 and with the passage of Prop 22 most if not all of the features implemented as part of the rewrite including DSTOP (Drivers Set Their Own Price), Price range instead of Fixed price etc are being stripped out.

> I'd start with something livable

Define "livable" in the context of wages

I used bitcoin.tax way back in 2018 and it was pretty smooth filing crypto related transactions