44,386 karma · joined August 30, 2007
http://dbmarkham.com/
daniel@danielbmarkham.com
https://leanpub.com/info-ops/c/LeanpubWeeklySale2024Jan19
Note: Book 2 is more code-centric, with active strategies to minimize solution complexity. This book is all about how to minimize to-do list complexity and tracking (which arguably is more important)
Think of how you already interact with pure functions: bash piping, SQL statements, OS math libraries, etc. If you're expecting an int, you get an int. Now, it may be the _wrong_ int, but that's on you, not the coder. Implementation complexity goes away.
Contrast that to how you might interact with a favorite JS library or framework. You do a thing that you think is right and the wrong answer or outcome occurs. 99% of the time your'e now researching SO and digging through somebody else's code trying to figure out whether it's you or them. Most of the time it's you, the code is good, but in either case you've mentally assumed responsibility for all this _new_ complexity that, by definition, is not related to the value you want to provide; it's just supportive.
Imagine if every time we used these pure functional things if we had to grok around the web and dive into code. What would using spreadsheet by like? How would word processors help us? Can you imagine an accounting project that spent three years designing their own spreadsheet for this particular customer, or decided to use a dozen new accounting tools that had never been put together exactly like this? Yet this seems to be the most common way programmers do their work. It's completely screwed up, we're hurting both our industry and our customers by continuing to work this way, and we should be ashamed of ourselves frankly. There are plenty of examples of complex programming solutions being provided that aren't created and maintained this way. (rant continues)
I used to feel the same way. I wrote and blogged about it, then did a bunch of research. I feel that this journey of ideas I took was the most important of my programming career. There's some extremely interesting stuff in how this guy is wrong.
In short, they are opposite things. In pure functional programming, the two are exact opposites. Mutable imperative programming gets fuzzy, which is why it tends to be so verbose. If you think about it, pure functional code without OO doesn't need variable names. The programmer's intent has been completely extricated from the implementation. We start adding a bunch of words to convey our intent to ourselves and future coders. Eventually this naively begins to seem as if we're actually communicating in a human language. Make the code readable, I used to say. It's a solution not a mystery novel. Comments are always good/bad.
I miss those innocent halcyon days of yore. Happy that I grew, but it was also fun confusing these two entirely different epistemological domains and believing there was some value in that confusion. There was not.
https://time.com/6301288/the-ai-jokes-that-give-me-nightmare...
Taking this a step further, and being facetious, I'm thinking of starting my own network! It's all me, all the time. Sometimes I'll be a nature show, sometimes I'll be in a spy movie, sometimes I'll have the guys from Mystery Science Theater 3000 cracking jokes at how dumb I am. Can't be that hard to do what this guy did, only in twenty-four different genres, one for each hour of the day. Turn it on, let it play.
I think it would be fun. Having said that, can you imagine somebody with mental illness, perhaps that makes them paranoid, having an AI interweave them into a spy thriller or murder mystery?
People already took computers too seriously before ChatGPT and the like. Having some official-sounding narrator, somebody you know from previous appearances, helping you out in your fantasy life?
As much fun as this is going to be, it also bothers me quite a bit.
I counted four topics in the first few paragraphs that the author defined in a poor, self-serving way. Any one of these topics and associated definitions would be interesting to talk about. Put them all together and it's just too much to clean up (for folks taking any kind of issue at all with the thesis or conclusion.)
It was well-structured and cogent, though. Kudos to the author for that. That puts them well above other essays of this type.
Then I read about the billions(!) being dropped into new productions of classics like "The Wheel of Time" or a spinoff of "Lord of the Rings"
This was a great feeling!
Then all of those billions started delivering. If you were _lucky_, season one was put together with tons of care and picked over. It was also based on a novel series where the author had done the same. If you were unlucky, the TV became like the radio used to be: just background stuff while you do something else. You don't even pay attention to it.
No matter what, season two almost always had the dramatic quality of a baked potato.
And heaven forbid the algorithmic numbers don't work. Then the thing is a dead series walking no matter what the quality.
You can't make quality art by throwing money at it. Very strangely, it actually has the opposite effect: people stop wanting to take risks. You can make a somewhat mathematically-sound four-quadrant marketing pitch hoping to hit the right taste clusters with a guess at whether it'll work or not, but you're not looking at emotional impact over time. Hell no. Hence the current tidal wave of edgy mediocrity.
I leave the TV on in the afternoons when I'm not working and trying to decompress. I have seen a few good series. They are all lower-budget attempts by high-quality artists using a preexisting literary work. Too often, without a solid canon of literary work, season one is great and the rest of it is just doubling down on whatever stimulus was in season one that the algorithm said people liked.
My opinion is that people are going to like "classic" TV the same way they like classic rock. There'll be something about it that they'll argue about endlessly, but there'll also be a lot of derivative work trying to recapture the lightning of actually knowing what the hell you're doing and being lucky.
Definitions matter. To me, art is emotional impact over time. Good art can impact you emotionally from hundreds of years ago. Bad art doesn't even hold your attention all the way through the presentation. (The impact does not have to be positive or negative; some of the best art can really bum you out)
I was watching an interview with an animator a while back. What did he think of all the new cool animation tools?
He gushed over the awesome tech and the wonderful talent that's out today. It's amazing and incredible compared to just a few decades ago.
Then he said something very interesting. After making sure folks realized he wasn't criticizing all modern animation, he said that he thought it all kind of looked the same. It was all perfect, and all perfect in the same way.
We have seen this happen to fiction over the last 50-70 years. Even when it's good, it's all kinda the same anymore. It's different and excellent, but in the same ways all fiction is. It's not unusual when experiencing a book or movie to get all hyped up by what you expect the experience to be, to gush while experiencing it how awesome it was, and to have no recollection at all even a month later what was so good about it.
If you want to feel sad for art, drop in on a reddit post where somebody asks "What's the best X?" where X can be a book, movie, podcast, etc.
It's all stuff over the past ten years, and it always is. We are in a state of perpetual forgetting and recycling canned emotions. The classics aren't the classics because they're better. They're the classics because they encourage us to create the same level of quality material. Sometimes they do so over our lifetime.
This is all marketing/commoditization at work, of course, and I'm not negative long-term. I think most artists see this. But there's a hella lot of denial going on, mostly from folks looking to pump up the next plasticine marketing special effects kabuki dance.
We always end up back with cluster/cohort analysis, then definitions. I think a big part of the problem is that we've already convinced ourselves of the definitions of the terms. These definitions prevent actual progress. We end up defining ourselves into our own locked room.
Reproducibility trumps causal models. If you call the psychic hotline and get the same results time-after-time, then keep using the psychic hotline. Leave the modeling to others.
When developing new science, stats lag behind reproducibility. First you see something that correlates, then somebody is able to make A always infer B, then you start doing some statistical surveying in an effort to build up a larger model.
Observation first, then Abduction (in the Peircian sense), then measurement, then modeling, and finally Deductive/Inductive reasoning.
We keep trying to do it backwards where we have the model for everything, then bang actual data into various parts of the model. The saddest thing in this process is that if we're only doing incremental, normal science (Kuhn), none's the wiser. This is why reproducibility tests are so important. We lie to ourselves. We're quite good at it.
BTW, I agree that with you general conclusion about case analysis and such, but only in the sense that it drives Abduction, not as a goal in itself.
As I understand your argument, because the net has solid endpoints we can identify and isolate, we should ignore that fact. Instead we should create more and more complex systems to work around bad actors?
Bad actor takes control of grandma's computer. We should do all sorts of things except stop talking to grandma's computer? The thing, I would suspect, that most people would expect?
Businesses suffer from too much transparency. Got that part. They buy things that don't work and sometimes hurt people, even if they don't intend to do this. So far, so good. Where is the part where new businesses models are supposed to exist because some people made bad choices and the current models don't work? Why don't we just publicize the bad choices and let things work themselves out?
Sorry. Missing it.
If anybody is suggesting permanent bans of IPs, it's not me, at least not at a public level. I may very well choose privately to do that.
To clarify, I, personally chooses a blacklist policy. Not some other org. I think if you offload this onto any kind of external structure, it breaks again.
ADD: We make publicly-available, second-by-second, how the internet is broken and invite all comers, including me and my blocklist, to help fix it.
There's a huge commerical interest in NOT fixing the problem of random crap showing up, from dancing cats selling things to targeted inserted ads. I get it. We saw this same thing happen with adblockers. It's now going on with "free" VPNs. Can't fight that perverse incentive, so don't fight it.
And yes, we can block entire subnets. You own the IP addresses, you're responsible for stuff coming out of them, at least to the degree that it's not maliscious to the web as a whole. (but not the content itself, of course)
I'm calling bullshit on these assumptions. The internet is a communications tool. If it's not communicating, it's broken. If you provide dynamic IPs to clients that attack people, you're breaking it. It's not my problem or something I should ever be expected to pay for.
To be clear, my point is that we're suggesting yet another layer of commercial, paid crap on top of a broken system in order to fix it. It'd be phenomenally better just to publicly identify place and methods where it's broken and let other folks with more vested interests than information consumers worry about it. Hell, I'm not interested in paying for the current busload of bytes I'm currently consuming for every one sentence of value I receive.
If your computer is infected, I don't want to talk to you for a month. If it continues to be infected, I might up that to a year, or permanently ban you.
It's your problem. Go fix it.
Looked at this way, this project is a thing of beauty. Just like the human mind, it's not going to explain how the LLM came up with an answer, it's going to create plausible arguments why it did so. How would we know? Do you actually think it'll make a difference whether the AI is correct about any random oddball question, especially when it can make a pretty coherent case about why the answer was correct all along?
It will not.
We are performing a very strange experiment with our fellow humans. Should be fascinating to watch it play out.
As I learn more, I realize how much we teach kids younger in their lives is oversimplified, bowlderized BS, no matter the topic.
I think that's unavoidable. You have to learn some comic book version of, say, physics before you're ready to talk about the Kuhn and paradigm changes and so forth. But there's also a real danger here: whatever your dumbed-down version of a topic is, it had better be a positive one, one that engages and challenges students to learn more.
Negative stories and narratives have the self-destruction of passionate learning built into them. It's poor-quality pedagogy. If I teach a class of fourth graders that clowns are evil, have always been evil, circuses are the work of the devil, and so forth? I can guarantee you that nobody in that classroom will ever become an expert on the fascinating history of circuses, animal shows, clowns, and so forth. It's extremely tough to get students wound up enough to spend a huge amount of time diving in somewhere, but it might take ten seconds to turn them off to an entire area of future study.
And frankly, those simplistic good-vs-evil stories are not only bad history, they're boring. They make consumers dumb. They make for idiotic public discussion. (Like I said, I tend to rant. I feel that we are cheating an entire generation out of the deep and beautiful vista of the humanities in our endless search to sell stuff to one another)
I get quite angry when people get reductionist about history, usually because of story or narrative (pre-templated story patterns re-applied over and over again)
The beauty of history is that the same event can have multiple, conflicting, powerful narratives associated with it. Looking at these narratives challenges our understanding of our own humanity. If we're smart, we end up realizing as Solzhenitsyn did, that "...If only there were evil people somewhere insidiously committing evil deeds, and it were necessary only to separate them from the rest of us and destroy them. But the line dividing good and evil cuts through the heart of every human being. And who is willing to destroy a piece of his own heart?..."
And then we _really_ begin our study of history, humanity, wisdom, and the rest of it.
It's far too powerful to buy into a story and get blinders on. I think it's probably a great thing when beginning your journey, but you should never stay there. And if you're a historian stuck in a version of a story I'd argue you don't know your own profession.
1. Predicting the long-term future is a bizarre and fruitless game. One might even say idiotic. If you're a prepper, and I'm not, prepare to live for a month or three. I do. Whatever the heck happens, within 90 days there will be some new normal. You're not trying to live forever. You're just trying to outlive temporary chaos. At that point, you're going to have to reinvent some kind of life. Who knows what that will be? Not me. Not this guy either.
2. He mentions wi-fi a lot. Gotta have that internet, gotta have that wi-fi, gotta have that bluetooth. I kinda doubt it. Can you go a day without internet? How about power? Just turn all of your stuff off. Most folks should be able to learn how to live like normal humans have for eons. There are various hobbies and occupations where you spend a lot of time without power or plumbing: scouts, infantry, etc. These people know what it's like. There are enough of them that the species itself will do just fine no matter what sort of apocalypse you can dream up.
It seems the problem with this essay, like so many pieces of media I consume daily, is the desire for _certainty_. Here's problem A. Here's exactly what it means, what causes it, and so on. And you have to do B to fix it. (Or, in this case, there is no B)
Disasters are chaotic and uncertain. Assuming climate disasters, moreso. But that both means that things are going to be tougher _and_ you can't have-wave your way around predicting how it all will turn out. Are many people who think themselves preppers disillusioned about their actual skills and their ability to adapt? Absolutely! But that's a one sentence essay. What's the fun in writing and commenting about that?
Apologies to the author. I believe there is a good one sentence thesis in there somewhere. But this sucked. I wish you better in the future.
Yes, low level and high level error handling can coexist, but this style of programming works much better (in my opinion) as a blackboard exercise or a do-it-once-then-done situation.
Love me some DDD, but you gotta have some kind of reasonable maintenance cycle that a moron programmer like myself can manage. That was the beauty of TDD in mutable coding: it scoped down the cognitive load for maintenance.
I can see railway programming carrying a lot of context through a lot of transforms and hell if I'd want to be at the end of the railway getting a big trainload of business and system context I'm unprepared to handle.
DDD and onion architecture for the win. Best of both worlds, and you end up with a railway anyway; it's just outside the micro-apps, not stuck inside them. In my mind, you want railway issues both explicit and a bit cumbersome to code. Most of the time, if done well most railway coding decisions involve business decisions that you should never be using clever coding constructs to avoid in the first place.
ADD: To clarify, monadic programming is great but there's a temptation to use it to avoid necessary business decisions. Sticking ambiguity into a type system can lead you to some difficult or impossible situations later on. I'm not saying never do it. I'm saying most ways I've seen it done are not so good.
From the outside, it appears that these megachurches are offering sort of an AWS for religion, facilitating small-group worship that happens in all kinds of ways. So you get the Agile Open Spaces idea, where folks can self-select in-or-out and topics to dive into, combined with the small group worship concept.
That's just a guess. I don't do churches. But the concept of an overall "thing" that people align to, then a very large group of people in another kind of "thing", and finally some sort of small-scale high-value interaction (for all three levels) is fascinating and not, in my opinion, unique to religion.
The only thing I'd add is that raising those issues and trying to find resolution is probably even more important than the code. People can see whether the app is working or not. They can't see when these conflicts happen unless somebody helps them.
Another way of stating that is that all professions (aside from math-based ones) are inconsistent at some level of detail and have various workarounds for that.
Even accountants are paid to give "accounting opinions" about the various legalities and appropriate nature of the books. Looking across a bunch of domains, once you get people involved, around the edges of the problem domain accumulates a lot of fluff that for throughout history didn't matter. It still might not matter. Or maybe it does. We're the first to spot it.
Because there's this whole school of thought that you can code around inconsistencies using various techniques, higher levels of abstraction, rule-based code, and so on.
These things are all possible, so whatever example I give I'm completely sure that you'll be able to "solve" it in code.
The issue here, however is that when we fix these inconsistencies in code we're taking what should be a business or analysis decision and sticking it in the solution framework. That works for a while until you create some monster nobody knows or can maintain.
I want to help you, but this is a business event, not a coding one, so it's dependent on depth-of-knowledge and general intelligence of your business partners. I'll give you something trivial, with the caveat that there's not a coding response to this that's appropriate. That's the point of the essay.
Let's say you have three customers. For various reasons they can't be in the room at the same time, so you're stuck listening to them sequentially describe some new CRM system you're building. Customer 1 tells you that the system should maintain an estimate of the total value of the new customer in dollars. You add in a decimal field to the system. Customer 2 comes by later and says nope, we need international currency support, so perhaps you add a currency type (insert lots of various possible solutions). Later still, Customer 3 comes by and gee, what was wrong with those guys! We actually deploy in a lot of places where barter is used, so the future value obviously needs to be expressed in farm animals. Maybe you switch to a string, screw 'em. Or maybe you have fun doing lots of coding stuff. We coders can solve anything given enough code.
But that's not the point. The point is that these bozos have completely different ideas of what the heck the app is going to do and us coding around that isn't doing anybody any favors. In fact, we're hurting them. We're making a mess at the same time. This disagreement is a business one and they need to figure it out, not us. No amount of fancy coding is going to fix the fact that these guys are walking around with different mental models.
Now, imagine that scenario with 50 symbols instead of just a few. We see logical inconsistencies as coders that would never appear in a conversation if we didn't bring it up. That's a precious gift that we've never had before, mainly because until 50 years or so ago we've never had hundreds of thousands of coders diving into tens of thousands of domains and finding these problems at this level of detail.
As a programmer, maybe it's not important. Maybe it is. Not my call. Things I might think trivial are critically important and things I feel important may be trivial. I'm not the lawyer telling those guys something about the law. I'm a programmer. I make logically consistent stuff in great depth. I'm just a kind of mirror, a type of mirror that is completely new to our species.
Yes, all code/abstractions are broken, ie, all models are broken but some are more useful than others. That, however, is not the point. The point is that these models are inconsistent in an invisible way even to experienced practitioners in whatever domain the code is in. It's not the the code or abstraction is broken. As you say, that's a non-event. It's that the code can show where the people are broken in ways that they themselves do not understand. That's the new part. That's where the line about the code talking back to the people comes from in the essay.
An abstraction is a person generalizing and codifying something. This is about code correcting abstractions, not abstractions being more or less useful than one another. I believe your directional arrow here is backward from the intent of the essay.
Humans can't create logically-consistent models in-depth, no matter what the domain. Programmers, at least old-school domain-model-driven programmers, live in a world of constantly reminding people of this.
Put differently, "logically-impossible" is a relative term. It's relative to human intelligence, not domain area.
This is much akin to chess, where really smart people may be able to track a dozen or so symbols in a mostly-consistent way while others may just track a few. But nobody tracks 50, at least without some sort of logically-consistent computational domain model.
Not very much. It's not unusual to load a humongous site with tons of libraries to get one or two sentences of information you wanted, usually after the obligatory 3 paragraphs of bullshit AI text.
My guess is that the actual information, in text, that you consume in various forms is extremely small. The ratio keeps getting worse and worse for the consumer.
Apologies. I overstated. How about "I study a lot of stuff"
And now the reply, if I've accurately modeled it, is "So you've studied a lot of stuff, what makes you an expert on any of it?"
Programming at its heart has always been cross-discipline, so there's really no judgment of ability aside from solving problems involving those disciplines you're working for. I find myself with a lack of adjectives that will keep our conversation short.
The main one in regards to John's tweet is this: desired behavior has to "force" out code and architecture. Typically teams and individual developers carry a lot of presumptions and assumptions into their solution space. They probably pick this kind of thing up from reading HN, Twitter, and the like. We all do this.
It's these "things you do but don't really think about" that are so pernicious in technology development. Guess what? Looks like AI is going to master that kind of thing since with each social media post we continue to train it on "How things are usually done"
By taking an extreme "force me to write code" approach, you end up developing a wide range of skills: DevSecOps, analysis, database design, and so on. In fact, you don't really know what you're going to develop when you start working on it.
That's the point. That's the job. Always has been.