385 karma · joined February 28, 2022
...yep, in general one of the root causes of having to do this kind of nasty work was someone (maybe even your past self) having failed badly at decoupling a distributed system ...add some "documenting it was supposed to be milestone 3" but then priorities got restructured and it never happened, then people left and others came, and voila!
The KILLER extra topping for me is when you lack any kind of quick-iterative-feedback-loop too. Eg. maybe it's cloud infra done in a braindead way and any way to debug is to do a tens-of-minute-length rerun/redeploy etc.
I'm OK working incrementaly, and even with incremental ambiguity, but I need a FAST feedback loop to iterate.
This 100%. I've learnt the hard way that the only way to get unstuck at this kind of hellish work is random action - I'm biased to over-analyzing the problem so get stuck because I can't pick an option. A bias towards action, ANY/RANDOM action if you can't decide is what seems to help here best!
"Easy but ambigous" sounds paradoxical, maybe the terms are not correct, but it is a very very different thing totally different than the other two.
Maybe there's a better way to explain this though: there's work that would be easy if it was clarified but it's not, and at the same time you can't just think hard and discuss and clarify and make it clear befor starting to work on it, so that you can set yourself to doing it, either in robot mode (while thinking of other things), or in regular problems solving mode; you just can't, that fog and ambiguity and unclarity is just "sticky", stays there until the problem is solved and sometimes even after, making it hard to even be sure that you've really solved it; there's no reward in tackling the hard problem of fully clarifying things because you can't succeed at it, but paradoxically there's no frustration either because you don't know it's a doable tasks either; you just have to muddle through, at every step of the way doubting yourself that you've chosen the right path since there's no clear way to measure stuff and to quantify your step; and you constantly have to interrupt yourself, going in "fishing trips" for information that is stuck in god knows whos brain because it was never documented, and never getting sure enough of what you discover to have trust that by documenting it yourself you make someone's life easier; and when you get to a "solution" you still keep the bitter taste in your mouth becase the systems have no adecquate testing tools and you can never be truly sure your solution will really 100% work; and if it seems to work for know, it's never sure it will keep working in the future, or that what it does will satisfy its operators...
To take your hiking analogy: it's basically a one-foot-after-another multi-day hike, but there's fog (and fog is a meteoreological phenomenon so you can't make go away), you're not really sure which of the destinations is the right one (or if many could qualify as adequate destinations), and there's a couple landmines hidden around, with very low probability of triggering them (1-5%) but yu still know they're there, and you have no water with you and you know that the water you'll find is 90% likely to give you disentery, and you might need to ask people for directions but it's hard to know whether they'll be right.
> I frequently tell my manager/team that we have two kinds of problems
...when enough managers/leads/architects/etc. mess things up in a row, you do end up with problems of the third kind :)
There's also the 3rd category of the hell of work that is easy, ambigous, but just uses up a ton of mental space to work through that mushy ambiguity. I'd solve difficult problems all day, can stomach doing hard work when its requirements are clear, but god keep me away from the hell of "death by a thousand papercuts" in the marshes of easy-but-mindspace/time-sucking quai-ambiguous problems...
Clarifying ambiguous requirements at least is rewarding when you achieve that clarity and help people understand what they actually need. But there's hellish work that is just unclarifiable before doing this, you just have to crawl to that mud of trivial but not-so-trivial-that-you-can-think-about-something-else-while-doing-them... 100% better to work a garden or serve in bar than that!
Problem is I fee either afraid I'm doing the exercises wrong so I stop doing them, or I find others impractical (eg. swimming or freediving).
Probably I should research into anaerobic that are actually considered helpful or somethig, since my brain seems to dislike the least these low-oxigen-effort types of exercises. Dunno...
It's frustrating because all the exercise advice I find seems targeted at people that are practically a whole different species from me...
It's also the getting used with the little monster, can state that from experience... to me at least alchohol was a must-have drug to reduce & control rage and aggresiveness induced by the little one and all the people that had to be around and be interact with... not a peoples person and my natural reaction to demanding people is to want to make them go away one way or another, with a great temptation to do so violently and permanently. (to clarify, was never angry or enraged when alone with the baby himself, taken alone the little thing was sweet af... only with all the people that seemed to suddenly go crazy and become extra "caring", "attentive", "demanding" or what not... I like babies, but I find other peoples' excessive reactions to them utterly disgusting... "baby brain" is a real thing apparently and it makes people uttely unsufferabe...)
I'm not less hungry, but I'm less hungry for nasty stuff, so from my perspective there seems to be a lot of truth in this!
Context: I'm natively extremely lazy (phisically, not intellectually), I find any phisical activity a pain an a nuisance, I derive zero pleasure from any kind of sport and phyisical exercise (except hiking and skidiving, but not the phisical effort aspects of them), the release-endorphines-on-effort circuit is 100% broken for me or I just lack the receptors for those endophins... eg. 100%-lazy-cat except not fat because genetically I come from a line of people slim-no-matter-what-we-eat. (Probably more average or atletic people's bodies just don't work like mine...)
Tbh, lots of non-overpriced products that I really like tend to disappear or be hard to find, so I just learned to settle for (a) a healthy enough cheap enough alternative that is never close to being as enjoyable but "whatever, it works", or (b) an X times more expensive quasi-luxury-class equivalent that has similar qualities and is from a common enough quasi-luxury-class producer with big market presence.
But there's a second avenue of though to explore: some people care for the qualities of the product itself, and some might argue that these kinds of people (we?) are a minority, most follow trends, either marketing phabricated or grass-roots word-of-mouth trends, so if a good quality products fails to be trendy, it will have horrible economical performance at least compared to the expectations set for such a product (R&D spent and all), so has higher chances to get pulled off than a bad performing low end products, that will stick around for more, and might by sheer luck "catch a trend" at some point.
Personalized production is what will save us in the end I guess, but when it reaches mainstream will also coincide with most people becoming jobless, so I don't know how the quality of goods and life will look like overall then, can't find my crystal ball :) (I guess the utopian scenario would be that anything mass-produced even decent quality is dirt cheap by then and affordable on UBI, and all the made-to-order-by-robots will be the various degrees of luxury - white sneakers? 0.01$ same model but factory made with dotted texture on violet gradient and printed with the face of your cat? 1000.00$, and hand-customized post-factory might be the new mid-class as human labour could become cheap again?)
Don't assume your gut feeling correlates with reality in such a quantitative problem. There might be low hanging fruit here - and it might save your freakin life man! Biochemical testing is not intuitive and the details are best lead to quantitative biochemists and such. Heck, even what Theranos embarked on doing could've lead to breakthrough improvements for some tests if they had been honest and focused the money on the research avenues that could've produced results...
you basically want an UI for inputing and validating some schema-enforced JSON, but that's overwhelming and overkill for most users and more apps, so you script into existence a simplified version of that (aking to "everyone who doesn't learn lisp is cursed to reinvent it" or smth like that but for frontend devs).
And realistically speaking, we'll never be able to standardize on inputing lists/tables/trees/graphs-of-text-cells anytime soon (ever?), so yeah, scripting is the only way.
This is why the web evolved to what it is, because the problem is always another different and more complicated problem than we thought it was, and then people never agree on solutions, so we have to say "anything goes", so it's scripting and web apps for anything about creating or inputing content.
Sure, you can have nicer standard ways for displaying context, but they'd alway need to be decoupled from the producing of content and be outside the control of the content producer ...because if you couple them you just get the heavy scripting spill out to the display side too, and that's how you get a horrbly slow and heavy javascript-only SPA frontend.
Accept heavy scripting on the content creation / input side, leave the display separate and accept that you cannot control it if you want it to stay nice and clean.
P.S. Confluence had some upgrades recently, and they're newish version is quite goo, especially table editing is dream imo :P (compared to the horror of their prev version) ...it's not Notion, but it's productively usable now :)
and in general, whatever bacteria can grow in a medum rich enough in salt + whatever acid (lactic if you're fermenting, citric or acetic otherwise) is probably not gonna hurt you too bad (salmonella and all the nasty stuff require protein + mostly towards neutral or alcaline environment).
sure, if you're into pickled eggs or weird stuff like that, watch out, but any sour pickled vegetables are safe and even if they go off will not make you seriously ill, they'll just start tasting like s and mess your stomach a tid bit.
A bulleted list is [add new element] > [buleted list] -> a container with text fields for each bullet (+ add/remove buttons) appears.
Same for table, a proper grid with separate inputs for all cells (see modern Confluence).
it's the application that should decide how it's to be displayed in different contexts
you're fighting the web if you're trying to do it otherwise, but then again, everybody is nowadays...
sure, the "semantic web" failed, but that's the history
you were not supposed to be able to control how content looks like on the web, this was supposed to be the choice of the browser customized by the user (eg. a user could set his browser to render sci-fi topics web pages text with font X and financial news with font Y min-size Z etc.)
"web design" is something that should't have ever existed or had any reason to even be a concept...
it all went off-the rails in the real world ofc, but this was the real context and dream of the web and its tehnologies as they were designed and standardized
> a single disorganized pile
Nobody proposes a "single disorganized pile". Sure, it's hard to eg. put bite-sized examples + condesed 'why' explanations into API docs and to maintain such docs, but it's doable - in the ML community some people use notebooks for docs wiriting and sometimes even for development by generating the code from them (sure, this is utterly extremist and I don't like it myself, but it works for some), some kind of renaissance of "literate programming".
I don't say we revive literate programming, but going haf-way between that and "a disorganized pile" is what I meant.
There's no personas, developers don't have three different brains they keep in the fridge and swap the ones in their head with.
Just admit that sure, interweaving information to get good quality docs is exhausting. And that we invented this "write for an audience" crap to simplify the work of the writers - "we're lazy or we have limited resources for writing docs, that's why we're doying it this way".
It's the same for writers and journalist, the "write for a specific audience" trick is way to lower the effort needed to get reviews / publishers' attention / clicks / views / shares etc. You do it when you're getting started if you ever want to get started I guess. But large scale it produces shallow literature, hard to fact check or contextualize journalism etc.
You do it for yourself (the writer) or to save resources for your companies (sure, interwoven/mixed docs are hard to maintain - I'm not even sure how eg. Pytorch's team manages to do that, but probably being the leading ML frameworks attracts top talent from the whole planet to one open-source project so it's doable for them...). But don't sell the bs that you're doing the reader/consumer any service, keept it hones to yourself and others!
We want multi-level docs combined/spliced/interwoven together so we can get our work done without wasting gandalfing the docs.
> While redesigning the Cloudflare developer docs
...oh, so this is the reason for CF's docs totally uncomprehensible and confusing mess. They've made getting to anythin so comfusing that AWS seems a breath of fresh air in comparison.
I dunno, maybe other peoples' brains are wired totally different than mine, but boy, these new fever of "writing better docs" seems to produce the exact opposite effects whenever applied.
The few enojoyable docs nowadays seem to be written by old-school unixy geeks and/or people that are not programmers or worse, "documentation developers": physicists, mathematicians, biologists, quants, data analysts etc. seem to do a nice job when they have the time.
In general, after you get to a piece of code you need to have thre questions answered:
- what problem does this solve? - this is actually the problem how-to guides solve, in practice your particular use case is NOT even close to the example use cases, so you'd just read the how-to guides as the no-bs higher level explanations because you know anything higher level is often bs
- how do I get something done with this? - API docs combined with dense explanations combined with dense usage docs (I don't want to have to piece them together myself through my own effort, I want them all in one place)
- what is the reasoning for it being implemented/architected/etc. this way? - design docs + "philosophy" section of docs + architecture diagrams
Further it's explained that this is where the "understanding" bit comes in. Ugh, so this school of thought is where some of the terrible modern documentation comes from...
I want the explanations/understanding first COMBINED with a minimal how-to guide serving as illustration for the explanation.
Then a quick link to an API reference that INCLUDES snippets of explanations and usage examples (that you'd find in tutorials or how-to guides).
Separating thinking about documentation this way leads to horrible documentation where you start with dumbed-down tutorials missing lots of crucial information, and then you have to put in tons of unnecessary effort putting disparate things toget ther in your head. FFS, programmers are rarely students since most of what we learn is learning-on-the-job and never chefs or theoreticians. Tutorials always waste your time so you'd prefer to start with something much denser... only that the only thing denser you easily find are the API docs, and you lack the explanations to understand them and usage exmaples so you have to hunt them for yourself... yuck.
As an example of GREAT documentation that combines things well (the API docs contain concise math theory / formular / explanations AND usage examples), take a look at some of the Pytorch docs: https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.ht...
What would be really interesting instead would be to admit that there's lots of knowledge and questions that are not encompassed by any particular discipline but instead lay at the interfaces between domains, that mondels like those oh philosophy no longer address well (and religion no longer provides useful psychological tools for), and re-image a new kind of philosophy build from the ground up on the idea of computation, and in this new discipline re-define +/- break apart fuzzy concepts like consciousness so that we finally have something to talk and think about!
---
...generally a chemical that doesn't degrade is chemically inert so it has less chances than any other more degradable compounds to cause harm (btw, how much research did we put into the multiple pathways of degradation for the currently fashinable "biodegradable" stuff? how many of those possible intermediary degradation compounds are not in fact quite toxical). (Bio)degradable also means reactive, so things are more complicated than it seems.
But yeah, any non-degradable fluid is probably a very bad idea because it's gonna spill and get everywhere, so even minor probability of toxic effect could have insanely large consequences...
Sure using "matter" like this without defining it is equally mystical. That's why we'd take 'computation' to be "fundamental" or 'the wave function'. "Matter" is just a higher level emergent property of what we perceive.
> I happen to take the view that consciousness exists, and is not an emergent phenomenon
OK, now I see why my viewpoint would sound so "off" to you, we're probably on different extremes of the thinking spectrum :) I don't discard you're viewpoint, it's just so so so so far from mine:
I take most of what we label "reality" to be emergent properties from some kind of fundamental computation (and by "computation" I don't imply a "computer" like we know, just "math that `can run`") whose math is probably too complicated and strange for us to intuitively understand (we can just hope to get to calculate better approximations of it). I don't just see consciousness as an emergent phenomenon. I see matter as an emergent phenomenon. And also space-time itself as an emergent phenomena inside our emergent consciusnesses. The "fundamental" could just very well be something like the simplest celular automaton with non-periodict behavior or maybe something too strange for our ape-minds to ever be able to comprehend (or too simple to understand how it could be the basis - there could be enough complexity/structure in just 'the distribution of all the prime numbers' or 'the digits of pi' to contain our entire universe with its infinite past and future "inside of it").
And to clarify, "simulation" is also quite generalizable - a simulation is just something that runs as an informational phenomenon on top of some physical substrate, but is fully independent of that susbstrate to the extent that it could be "ported" from that substrate to another completely different substrate that just happens to also preserve the subset of mathematical laws required for the computation. Eg. "software" that can be "ported" so it's independent of the actual nature of the hardware. Digital/discrete (as opposed to analogical) computation gives you this magical "divorce" of computation from substrate. It doesn't have to be something like our current day software running on something like our computers - any kind of "portable discrete/digital computation" is "a simulation".
China should invest seriously into making itself more a more livable / enjoyable country and into securing its agriculture instead of their grand space, AI and social engineering projects, if they want the slightest chance of competing in the new game...
This is just rephrasing of the symulation hypothesis in mystical terms and is just as useless as the symulation hypothesis itself (regardless if it's "true" or not).
There's some valuable deeper stuff in hindu and buddhist philosophies, but this set of ideas isn't it (neither valuable nor actually deep, just "exotic" sounding to an extent).
- the real end-user would quite likely care about some details because they'd know it will directly affect their lives (but you'll likely not get the actual end-user in the room because they're kind of never the stakeholder in AI/ML products)
- managers on AI/ML products are often quite technical
So technically they're right, it just their deifnition of "non-data specialists" does not include actual stakeholders. And ofc. you're right too.
That's often the problem with current AI/ML development, the owner/client is neither the stakeholder nor the actual end-user (at "best" the end-user is their direct "prey/victim")... and this leads to much worse outcomes than simply failing to explain/understand how/why a solution works :( I was having some hopes that the much maligned "web 3" push towards decentralization would make a dent here pulling at least a degree towards a better course... but nope.