Full of contempt, she refused to talk to the engineers
respectfulleadership.substack.com
respectfulleadership.substack.com
At my first job (infrastructure monitoring, 24/7 shifts) the COO appeared one morning at 7:30 am in the office because he had seen a ticket opened a few minutes before from a particularly annoying but important client.
He proceeded to give us some additional context on that client, added some clues about their infra and we went on our way to handle the case. In all this he was knowledgeable and kind. I have a fond memories of that man.
Holger if you’re reading this, you’re great.
Hahaha, oof. Tale as old as time, and I mean specifically this exact thing: Rails, passing through the hands of multiple teams, some or all outsourced. Anyone who's seen this before knows exactly what that codebase was like to work on.
I do wonder if the marketing data from a 150-person org wouldn’t fit on a good old-fashioned database server. No need to get your feet wet.
AWS these days have 24TB RAM servers available on demand . 100 GB data in even dozen different formats require just regular database .
A data lake just for generating reports on 100 GB of data which are going to be used by very few and built by even fewer seems overkill .
I would say no for anything less then 50-100TB today.
We do have to keep in mind she was fired in 2018 and this was probably few years before that .
Keeping 2015 as the reference frame a bare of minimum of few TBs to consider any lakes or puddles
This reminds of the System Engineering Maxim "User rejection is the number one cause for system failure"
I don't know that that is 100% correct, but I think one of the strengths of iterative processes is regular customer input & fast feedback loops
We don't know so many things about the project that all the solutions I see in the comments could be wrong for one reason or another.
> Respectful leadership pays dividends in all circumstances. But it especially pays dividends when we are talking about the relationship between top leadership and those teams on whom they are utterly dependent. Respectful Leadership does not mean that you have to spend your time listening to everyone in the organization — in a large enough organization that is not even possible. But it does mean that when you are dependent on someone’s work, you commit to working with them in good faith, and you make time to hear their concerns and their suggestions.
So, yes, the SQL query. Perhaps one of the small and easy parts of the problem.
> Me: Sure, let me just:
> SELECT * FROM some_ideal_clean_and_pristine.table_that_you_think_exists
Credit: https://twitter.com/sethrosen/status/1252291581320757249
Regardless, “data lake” is a fucking stupid and nonsensical metaphor.
I would interpret the first conversation as a giant red flag and get the hell out of there. Unless you're getting paid stupid amount of consulting dollars, it isn't worth the time dealing with COOs with inflated egos.
Milk it if you need the money, but avoid putting your reputation or long-term goals on the line for people like that.
Full of contempt, he refused to open QuickBooks.
Amelia: Okay, but what does this project involve? I mean, really? Have you thought about the details? You want us to get all of your Facebook data, all of your Google Ad Words data, all of your Amazon data, all of your print data? And your entire catalog? And all of your sales data? That’s an enormous amount of data.
Me: Oh, I see. You need a place to put it.
Amelia: We need a place to put it. So we are going to build our own, internal data lake, and we will put your data there. But I’m thinking, wait, does that make sense? We build a private data lake full of your data? Won’t you want access to that data lake?
Me: Yes, of course. If it’s our data, then we would want access to the data lake.
Amelia: That’s what I’m thinking.
Me: Since you have to do it anyway.
Amelia: Exactly. There is no other way we can do it.
what I'm thinking is that there is another way to do it.
This should have been a rant about out-sourcing. If understanding what they are spending money on is not a core requirement for a capitalist organization, I'm not sure what could count as one.
The business needs a database, the database needs someone competent in SQL along with some basic accounting and finance education.