HNHacker News
TopNewBestAskShowJobs

msluyter

4,603 karma · joined March 5, 2008

Software Archaeologist. Lives in Orlando, FL. Unfortunately.
submissionscomments
msluyter··on Ask HN: Inherited the worst code and tech team I have ever seen. How to fix it?
Exactly. I often think of scenarios such as this in evolutionary terms. Imagine a doppelgänger competitor in this space, exactly the same, except they're pretty content with their crappy tech ecosystem and focus on delivering features (albeit, more slowly than they might with a more modern system.) And so...

1. You focus on the advice here, adding tests (which IMHO is actually pretty difficult on a legacy poorly architected/documented system), source control, refactoring, etc.... with any time remaining in the schedule devoted to adding features.

2. Your doppelgänger chugs along and releases several more features than you do, giving them a market edge.

What happens next I suppose depends upon what difference those extra features make. If the delta is small, you may be able to pull out ahead, in the long run. But if it's large, then your company may start to lose revenue / customers. Then, the screws will likely tighten even harder, and you'll be forced to sideline refactoring efforts and double down on delivering features. And then those refactoring/cleanup efforts will bit rot and you'll find yourself back roughly where you started, except now you're behind your competitor.

There's a quote I can't seem to find atm that summarizes this. It was something along the lines of "With php our product is already up and gaining market share meanwhile our competitor is still futzing with their rails configurations." (If you know the actual quote I'd love to see it.))

msluyter··on Software component names should be whimsical and cryptic
I have doubts about the author's overall point, but something to consider is how does a name help/hinder search? A name that's overly mundane, say, "Go", becomes problematic to google (hence the need to search for "golang" or add other terms).

I feel almost like there's an analogy between choosing names for packages/projects and bird calls. Birds need to make their calls stand out in a given ecosystem. If you have a dozen projects like, say, "object-mapper," "object-db-mapper," "object-mapping", "object-mapper-thingy", "object-mapping-service", "db-object-mapper", it's hard to remember wtf was the one you need. If in such an ecosystem your project was named "Orangutan," it might stand out and be noticed. OTOH, if all projects in a given ecosystem have weird names like "murano", "swift", "glance", "hotdog", "gorilla", "zaqar", "cthulhu", "funky-chicken", "Sauron" or whatever, it may be hard to remember what they all do/mean -- yes, I'm giving you the side eye, openstack. In such a space, "cloud-disk" or something might win, IMHO.

Slightly tangentially related is the use of odd words in log messages to aid searchability. I've been grateful for misspellings in the past, so I could easily grep for some critical log message containing `transation_id` or whatnot. (Good struct logging makes this less necessary, thankfully.)

msluyter··on Why is Snowflake so expensive
Some of these complaints seem fair to me, some not as much. tl;dr -- Snowflake requires a fair bit of knowledge/effort to use optimally.

I spent a number of months last year focused on lowering Snowflake spend. In the process I learned a ton about Snowflake and gained a fair amount of respect for the product. Respect as in "this is really great" as well as respect as in "I need to be on guard here or I'm going to get hurt."

I think my biggest misconception at the outset was thinking of Snowflake like it's a relational database. It's not. Or rather, it is with a large number of caveats. Snowflake doesn't have b-tree indexes -- rather it has "clustering keys," which are sort of like coarse grained indexes that colocate data in micropartions, allowing queries to do micropartition pruning. If you have a well clustered table and you're filtering on your clustering keys, things will be great. But if not, or, for example you have to do multi-table joins on non-clustered columns, you'll suffer. So unless you have search optimization enabled (which costs more!), you have to retrain yourself away from "oh, just add an index here or there to make things fast" type of thinking you may have had working with Postgres or whatnot.

Regarding the author's complaints about lack of observability, I generally found it pretty easy to analyze what was going on via the query_history table. And the built in query analyzer is quite helpful. We did add tags to our dbt runs, which was pretty easy, and I wrote a handful of queries to find like the most expensive dbt models. It wasn't really that hard.

That said, dbt in particular provides a number of foot guns wrt Snowflake. Subqueries, as the author mentions, is one. We created some custom dbt macros to do things like instead of `select * from foo where x in (select * from blah)` -- if blah was small -- do a query on blah and write the query using a literal list, like `select * from foo where x in ('a', 'b', 'c', 'etc...').

Another issue we discovered is that in dbt it's trivial to create views. But we found that if views get too deeply nested, Snowflake can't adequately do predicate pushdown. So big stacks of views on views are suboptimal.

Another interesting one was tests. Dbt makes it trivial to perform null or uniqueness checks against a column. We found we were spending a lot on those tests that simply were doing something like `select * from blah where col is null`. On non-cluster key columns or complex views, these were causing full table scans. We took a number of steps to mitigate those issues. (Combining queries; changing where we did these checks in the dag). The way tests are scheduled is problematic as well. One "long pole" test will keep your warehouse up and using credits even after the other 99.9% of the tests have completed. After some analysis we separated long pole tests from the others and put them on different warehouses.

I could go on and on, actually, but I think that provides a taste of some of the complexities involved. Like almost any tool, you have to really understand it to use it effectively. But it's all too easy for, say, analysts, who may be blissfully unaware of the issues above, to write really poorly performing SQL on Snowflake.

msluyter··on Wealthy Americans moving to Florida at 4 times the rate of other states
Everything is relative. We moved from Austin to Orlando last year for family reasons and our new house of roughly equivalent size/amenities was significantly less than our Austin house. And now we wouldn’t be able to afford our old Austin house.

Not saying it’s not bad here — it’s just less worse than some places.

msluyter··on Ask HN: What are the best programming tricks you know?
One that a professor of mine used when I was in grad school which I don't see that often these days is the use of sentinel values. For arrays/lists of unknown sizes, it's sometimes nice to do something like:

    while val != SENTINEL:
        # ...stuff...
        val = some_iterator()
You just have to be careful that you can never have real data equal the sentinel.

A variation on this can be used to stop queue processing in a multi-threaded/processing environment. You can put N sentinels (aka, poison) in the queue for N readers, and when a reader gets the poison, it quits.

msluyter··on Ask HN: What are the best programming tricks you know?
Not sure if this counts as a "programming trick", but for maneuvering around the shell: learning readline.

https://readline.kablamo.org/emacs.html

^This one cool trick will increase your productivity by 1000%!

I've had sr. engineers see me using ctrl-r and go "woah, how did you do that?"

msluyter··on Ask HN: CS grad who barely knows any CS
My bachelors was in music and I was midway through a masters in music when I decided to switch to CS. Needless to say it was a huge paradigm shift, and having had very little experience prior to that, and also as a result of a program that was highly theoretical, I came out with a MS in CS but with some massive gaps. I wasn't that comfortable with basic tools, like the unix command line, vi, or even DOS (it was a while ago), for example.

So, feeling somewhat unconfident, I took a job in software testing -- SDET, doing test automation mostly in Perl. I figured it'd be a gentler learning curve during which I'd beef up my skills, and after which I'd move over to software development. It worked out fairly well for me, though due to a variety of factors, I stayed in the testing/QA side longer than planned.

I think that's not a bad path, and better than starting in tech support imho because you'll be directly involved in the development process. Just make sure that it's a technically focused role with opportunities for writing code, and not just button pushing. And don't stay there too long, or you'll find yourself in the unfortunate niche of the rare and highly sought after testing/QA person who can also code.

msluyter··on Ask HN: What game do you wish existed?
"- Accelerate game play by automating score keeping and timekeeping"

Yes indeed, this is huge. I'm thinking of you, Through the Ages, with your incredibly fiddly and easy to make mistakes in upkeep rules. Our first few run throughs were pretty much ruined because someone made an early mistake in their favor that snowballed over time. Imagine if everyone had an iPad in place of their game board that would largely eliminate mistakes.

Similarly, back when Dominion came out, there were some online servers where you could play (I believe they were mostly shut down), and it was such a nicer experience because you didn't have to spend all of your time reshuffling your deck.

msluyter··on Billing systems are a nightmare for engineers
I rewrote an invoicing system (originally in .NET) in Python and yeah, I totally feel this. All the system did was generate a CSV file that could be input into another system (Quickbooks, iirc). The original .NET version was nonsensical at times, replete with bugs, and I spent a lot of time trying to figure out what the original intent was (there were no docs, of course, and finance often had conflicting/incorrect ideas about what the system should do.) Version 1 had to simply replicate the csv generated by the original system exactly, which required purposefully writing some bugs that I discovered along the way.

But yeah, there were all sorts of nasty complexities -- this was a usage based system, but there were (optional) tiers (pay less for higher tiers), as well as certain time segments that were discounted. And then there were cases where multiple accounts could share a set of data and get a discounted rate and/or only one of the accounts would be billed. And on and on....

One lesson I drew from this experience was that, when discussing things with finance to try to figure out what the requirements were, it really helped to have pictures. I would create these elaborate usage/time graphs, and color in portions to reflect different billing/pricing scenarios. These made it much easier to discuss the current/future behavior of the system. Because saying "really, this is simply the integral of a usage function over time" didn't fly.

msluyter··on NaturalSpeech: End-to-end text to speech synthesis with human-level quality
Totally tangential comment:

You can click play on any/all of the samples simultaneously, resulting in a neat sonic effect vaguely reminiscent of Steve Reich's famous "Come out." [1]

[1] https://www.youtube.com/watch?v=g0WVh1D0N50 (skip to like 7 minutes in to get the idea)

msluyter··on User Disengagement
I don't know anything about the company but I applaud the sentiment. But is it obvious to note that a lot of companies start with similar ideas... and then any number of forces -- the market, shareholders, the board, management changes, black swans -- cause the screws to tighten? And then the ideals begin to erode.

I guess I'm sort of implying (without any empirical proof) that there are darwinian reasons for state of things that are inherently difficult to avoid.

msluyter··on New links found between musical training and cognitive ability
Preface: I have a pretty deep musical background; bachelors & masters degrees in flute performance.

I have a 7 yr old. Her kindergarten required either piano or violin, and we opted for piano. At that time, we implemented a daily '5 minutes of piano' mini-lesson where I'd help her with her current lesson/song. We're doing homeschool now, but have kept up the '5 minutes of piano'. My daughter doesn't really like it that much, tbh, but I've made her stick with it because: a) she's actually quite talented -- great pitch/rhythm, and can already play by ear, for example, and I'd like to get her to a point where she can really enjoy that talent, and b) as a general lesson in perseverance/grit/skill mastery.

But I really struggle with the notion that the above is simply a rationalization and that I'm really trying make up for my own failure[1] to achieve my own musical goals vicariously through my daughter. And I've also struggled not to force it too much. I've bent over backwards to make it fun -- I make up games, do a variety of musical/rhythmic activities like drumming, dancing, or even playing pattycake, or do things like watching Fantasia or even listening to pop songs. And I've had to put the brakes on thoughts like "we're going to polish the crap out of this song so we can impress the relatives" and focus more on the overall process: is she having fun? On the other hand, I'm trying to not reward a "this requires some effort so I'm going to quit" type of mindset.

It's definitely been a tough balancing act.

[1] How I used to perceive it -- I no longer feel that I was a "failure" so much as I chose a suboptimal path.

msluyter··on Zq: An easier and faster alternative to jq
Whenever jq comes up I feel obligated to mention 'gron'[1]. If all you're doing is trying to grep some deeply nested field, it's way easier with gron, IMHO.

[1] https://github.com/tomnomnom/gron

msluyter··on Americans are drowning in spam
These conversations remind me of the "why your spam solution won't work" checklist that used to go around slashdot:

https://craphound.com/spamsolutions.txt

It's been a while though, so perhaps some of these are no longer relevant.

msluyter··on I'm a scam prevention expert and I got scammed
Am I alone in just never answering my phone? Like, ever, for _any_ unknown number. Seems like that alone solves a certain class of problems.
msluyter··on Developers spend most of their time figuring the system out
Aside from one's subjective take on pair programming -- and there are obviously many valid reasons why it doesn't work for some folks -- I'd have to guess that doing it 100% of the time is quite inefficient. Software development presents a wide variety of tasks of varying cognitive difficulty. I tend to believe that pair programming's sweet spot is for tasks of medium to medium-high levels of difficulty.

For the most challenging tasks, say, coming up with an original solution to an unsolved problem, I think it's better to alternate deep focused thinking alone with group brainstorming. But you really need that meditative alone time to gnaw at difficult problems -- for me it tends to be in the shower, or during a walk. That's where you have those awesome "Aha!" moments.

On the other end of the scale, certain mundane/repetitive/trivial tasks really don't warrant pair programming IMHO. You're spending two brains on something that really takes 1/4 of a brain.

The ideal case for pair programming, IMHO, are those tasks where you can visualize the general path forward, but where there are significant challenges wrt implementation such that another pair of eyes can really provide insight. For example, say you have to implement a module, and you have a general idea of what it needs to do, but the overall decomposition and abstractions involved may be inchoate. While I think a fair amount of "implement this new feature" type of stories fall into that category, in my experience that comprises only ~40-60% of the total workload.

The above mostly applies to pair programming with two fairly equivalent/experienced developers. I agree with others who have noted that it's a fantastic training mechanism for new/junior devs. And then, there are various special case scenarios that justify it, as in, you have a problem with X and need to pair with an expert in X to resolve it. IMHO, these are best done ad hoc rather than via a formal process.

msluyter··on Rejection Therapy (2011)
Similar to what the famous psychologist Albert Ellis did:

"As an adolescent, Albert was extremely shy around ladies. At age 19, he started showing signs of thinking like a cognitive-behavioral therapist by forcing himself to talk to about 100 women in the Bronx Botanical Gardens within a period of one month. This helped him get rid of his fear of rejection by women."[1]

"He found 130 women sitting alone that month on park benches. He sat next to all of them, whereupon 30 got up and walked away. He spoke to the remaining 100 — for the first time in his life — about the birds and the bees, the flowers, books, whatever came to mind.

Al later said, "If Fred Skinner, who was then teaching at Indiana University, had known about my exploits, he would have thought I would have got extinguished, because of the 100 women I made one date — and she didn't show up!

"But I prepared myself philosophically, even then, by seeing that nobody took out a stiletto and cut my balls off, nobody vomited and ran away, nobody called the cops. I had 100 pleasant conversations and with the second 100 I got good and made a few dates." [2]

"Nobody took out a stiletto and cut my balls off" -- I've always loved that expression.

[1] https://totallyhistory.com/albert-ellis/ [2] http://www.rebtnetwork.org/ask/may06.html

msluyter··on The Great Steepening
> At that level, it still makes sense to borrow all you can, buy the basket of goods or gold or anything that holds its value in dollar terms, sell it later on, and pay the loan back.

I'm guessing -- also not deeply versed in macroeconomics -- that "buy the basket of goods or gold or anything that holds its value in dollar terms" isn't actually that easy. For one, if everyone does that, you'd expect current prices of said goods to rise, and then fall when you try to sell them. Thus there's some risk that you'll essentially mis-time your transactions and buy high / sell low.

Two, there may be other reasons whatever you buy fails to hold its value. Someone might discover a giant gold deposit or whatnot, rocking the gold market.

Three, there are transaction costs. For property, you're paying realtors, home inspectors, closing costs, etc...

I'm sure this is an incomplete and naive analysis.

msluyter··on Google Search Is Dying
It seems like we're approaching what I call the "dismal equilibrium." This is the idea that any free site/service/app eventually will have to monetize itself in order to remain free, inevitably in a way that degrades the experience. Ads, typically, "pay to win" for games, or perhaps even calls for donations for public radio. An equilibrium is reached when further monetization isn't possible without driving away users; quality is just barely tolerable, hence, "dismal."
msluyter··on New Zealand's bid to become a dark sky nation
It disrupts insects (and thus, everything downstream of them in the food chain):

https://www.sciencedaily.com/releases/2021/03/210317141651.h...

msluyter··on Facebook loses users for the first time
Agree with most of this, but I'm curious where Facebook groups fit in here. We recently moved to Florida and have been scrambling to find non-religious home school friends for our daughter (homeschooling because of Covid). Facebook has been pretty much only resource available, and I've joined a number of local groups that have proved somewhat helpful. I'm wondering whether groups may provide a level of stickiness and user retention that the timeline may not?
msluyter··on The State of Web Scraping 2022
Re government websites: they're often terrible. I've occasionally contemplated a side project just to scrape and restructure some local/state websites into a usable forms with search and whatnot.
msluyter··on Low interest rates in advanced countries have pushed money into real estate
>>* remove the mortgage interest deduction

>So what's your goal here, given that historically this was the single biggest tax benefits to the (ever-declining) middle class.

Weird, I've mostly read that it disproportionally helps the rich. Examples:

[1] https://www.stlouisfed.org/open-vault/2018/may/why-economist...

[2] https://www.forbes.com/sites/artcarden/2019/06/28/should-we-...

[3] https://fivethirtyeight.com/features/the-tax-deductions-econ...

msluyter··on Launch HN: Inflow (YC S21) – Self-help app for people with ADHD
Re: "where did I put that" -- I got a set of Apple airtags recently and they've been a godsend. I also have an apple watch, so via the watch I can find my phone, and via the phone I can find my wallet/keys/etc... They're awesome.

Re: reminders, one thing I've noticed with any sort of reminder system based on alerts/notifications, is that while they work for a while, eventually my brain just starts ignoring the alerts. I honestly have no great solutions for that. This applies to any sort of organizational system I've tried, actually.

msluyter··on Launch HN: Inflow (YC S21) – Self-help app for people with ADHD
Isn't that crazy, that one could go 45+ years without knowing they had ADHD? It wasn't until I was... > 50yrs until someone I was talking to said "sounds like you have ADHD" and I was like "what???" But sure enough, after seeing a couple of specialists and reading up on it, turned out I had all the classic symptoms and was coping with it in a number of suboptimal ways. Knowing about it and treating it has improved my life immensely.
msluyter··on Thoughts on chess improvement, after gaining 600 points in 6 months
By "Woodpecker method" do you mean this:

https://www.amazon.com/Woodpecker-Method-Axel-Smith-Tikkanen...

?

This is why HN is great -- appears to be an amazing book.

msluyter··on Faster Python with Guido van Rossum
I agree with this, but when it comes to scaling, making things faster at the language level is only one option. And while, as you suggest, the Python/static_lang gulf has narrowed, I would guess the gulf between "what needs to be done in the language itself" vs. "what can be done by some external system/database/process" has also narrowed.

What I'm getting at is, let's say you have some system where you need highly performant real time processing of some data. Perhaps 10 years ago, you might have needed a complex multi-threaded java/c++ app to handle it. But perhaps now, you use, say, DynamoDB, kinesis, kafka, lambdas, or some other collection of services that you're basically gluing together with... python.

I'm not saying you're wrong. I'm just wondering, if you could truly factor in the costs in both developer time -- developing multi-threaded apps is usually pretty tricky after all -- and infrastructure costs and whatnot, where the boundary between "python is adequate" vs. "we really need go/java/c++" lies, and in which direction it's moving.

In retrospect, I guess it's sort of a silly question, since at the core, the services we'd be gluing together are no doubt written in faster languages. Perhaps a better way of framing it what % of problems can be adequately solved (cheaply) by just using language X, and how has that changed over time?

msluyter··on The History of OS/2 (2001)
Random OS/2 stories.

My first real technical job was doing OS/2 support as a co-op (sorta like an intern). Our team supported larger clients, who'd send us questions via IBM's mainframe system (RETAIN, or PROFS, iirc). I'd say the answer to half the questions was roughly "install the latest driver from $HARDWARE_VENDOR, and/or, contact $HARDWARE_VENDOR to see if they have a driver that supports OS/2." The other half were OS/2 / Windows compatibility issues. Because, while in theory you could run Windows apps in OS/2, it was typically pretty flaky.

We had to type our answers into this antiquated mainframe editor, where if you typed past the character limit, you'd end up typing into like a command buffer and end up garbling your message, so you couldn't paste into the mainframe editor! So if you wanted to simply quote an existing solution, you had to retype it.

I spent my spare time writing a little app in VX Rexx that provided a native text box (like the one I'm typing in now) and that would then splat the message (with correct line breaks) into the mainframe editor via something called HLLAPI, iirc. Made life incredibly easier because you could cut&paste and whatnot. Still pretty proud of that one.

The types of questions we got made it pretty clear to me that OS/2 was doomed to failure. Not even IBM's own laptops had drivers to support it adequately! And while preemptive multitasking is nice, most people didn't need 10 terminal windows open at a time. They just needed to run Excel and Word or -- and I think this was a critical factor -- play games.

msluyter··on Problems with Oracle SQL
Tom Kyte books were great and beyond Oracle, I learned a lot about databases generally from them.
msluyter··on Miller CLI – Like Awk, sed, cut, join, and sort for CSV, TSV and JSON
Some other useful tools for csv files, in particular:

csvkit, a set of command line tools for manipulating csvs: https://csvkit.readthedocs.io/en/latest/

visdata, a quite terminal based csv explorer: https://www.visidata.org/

← PreviousPage 3 of 33Next →