150 karma · joined September 23, 2022
EDIT: Figured it out - I stopped using Google's public DNS server and started using my ISP's DNS server, and now I can use the site.
He was emphasizing his opinion by voicing it as a question. The text immediately after the excerpt you posted says this makes a valid rhetorical question:
> ... or as a means of displaying or emphasize the speaker's or author's opinion on a topic.
That should be much easier and less expensive to do than eg, replacing lead pipes in the house or town/city that may be delivering poisoned drinking water.
It's a shame that IBM sold its PC business to Lenovo. Maybe it's just the model I have, but Lenovo's ThinkPads feel so much worse in quality than IBM's ThinkPads. I probably won't get another one when the time comes to replace mine.
> The thing is, this is just how they get work done. If you point it out, they will still do it because this is what works. And you can’t fix the world overnight. And they don’t know what you know. And you don’t know what they know. And you may be wrong about a bunch of this anyway!
Wow. These two paragraphs are verbatim the biggest issues I've been dealing with in my current workplace. It's like Dan has been monitoring me and taking notes. It's really encouraging to hear about his experience and lessons learned — especially because of how much of an emotional struggle it has been to establish an official data engineering presence on our very old-fashioned team. It helps me know that the things I've been struggling with are much more common than I realized.
Every news outlet and blog should have something like this. Seems relatively quick and easy to put together compared to their main sites, and I'm sure tons of people will use & appreciate it.
I think I'll celebrate it's birthday by wearing my Debian t-shirt today :)
Another space opera I felt the same way about is Peter Hamilton's Commonwealth Saga. There's also a trilogy and some other novels set in the same universe. 99% of the fiction I read is space opera, and the Commonwealth universe is definitely my favorite. So action-packed and enthralling, couldn't put the books down. As a matter of fact — it's been a few years since I've read them, and I'm almost done with Lords of Uncreation, so I think I may read it again :)
I've gone through periods where I smoke a relatively small amount of flower on a daily basis for months at a time, and then stop cold turkey, over a dozen or so times in my life. One thing I really enjoy about stopping cold turkey to take a tolerance break is that my dreams become profoundly vivid during the first week or so of withdrawal. I can often remember dreams vividly as if they occurred in waking life, for as long as two days after the night I dreamt, and with enough clarity to write them out in detail. I usually look forward to the experience.
Maybe I'm just a weirdo, but I actually enjoy the "come down" from individual smoke "sessions", and the "reset" I experience when stopping cold turkey. It feels refreshing in a way, not really sure how to describe it.
(1) getting datasets from historically siloed data management teams to join with each other cleanly and programmatically — which in my case is a political issue, not a technical one
(2) determining the best way to replace old tabular (Excel) reporting with modern interactive (Tableau) dashboards that refresh from a database — which in my case usually means getting a better understanding of many cross-team business processes, understanding how data management is involved in them, and how data is being used to make decisions and take action
(3) determining how to structure and present this data to them with the least possible friction, and in a way that's the most useful and impactful to them
This is all to say that DE roles seem to vary wildly depending on the domain you work in and the kind of business you work for. But at the end of the day the purpose of the role is to make clean and accurate data available to end users with the least friction and in the most impactful way. And most kinds of data businesses use can be represented by, transformed with, and interacted with using SQL.
There's tons of SQL resources, but I'd love to find some books that cover these deeper issues of politics and business process discovery from a data engineering perspective.
I've never been in a SWE role before, but am related to and have known a number of them, and have a general sense of what being a SWE entails. That disclaimer out of the way, it's my gut feeling that a DE typically does more "hacky" kind of coding than a SWE. Whereas SWEs have much more clearly established standards for how to do certain things.
My first modules were a hot nasty mess. I've been refactoring and refining them over the past 1.5 years so they're more effective, efficient, and easier to maintain. But they've always just worked, and that has been good enough for my employer.
I have one 1600 line module solely dedicated to validating a set of invoices from a single source. It took me months of trial and error to get that monster working reliably.
In addition to everything above, I'm thankful for these:
Pass (passwordstore.org)
Newsblur
Multitail