While researching / reading up / debugging, you stumble upon something interesting. Upon looking into that, yet another subject catches your attention.
You know how this goes. So... (see title). Bonus questions: what intermediate steps did you pass along the way? What stuck in your mind the most?
Linux proved interesting enough that I kept finding all sorts of cool new rabbit holes to go down - shell scripting, filesystems, Python, databases. It was side-quests within side-quests! Plus, having kicked my gaming habit, I had plenty of time to explore these.
Anyway, to cut a long story short, that was 23 years ago. I ended up getting a career in tech, relocated, got married, had kids, lived the American Dream... The "life" rabbit hole kind of got in the way of my plans, so I can't wait to finally get back on track and play GTA III on a decent box.
It got me to learn C, graphics programming, operating systems, networks and firewall, literally everything I wanted to do required a couple of days deep in arch linux wiki learning about all kinds of inter-connected systems.
Same here.
Life happened.
What I told my dad would never happen (not using Linux From Scratch or Gentoo - like how can he use Windows and like just get things done instead of digging deep and solving problems all the time?) did happen.
I now use Debian derived but more up-to-date and easy to use for mainstream people distros (like shudder Ubuntu) and they're relegated to server and TV duty and I hope nothing breaks when I upgrade from LTS to LTS version coz "I don't have time for this!" and I play a few games here and there on my Windows laptop.
But I think it's important to have gone through the "rabbit hole" in the middle. All the digging and understanding I did I still do all the time at work. I just no longer spend the other half of my life on it. I spend it digging into other things.
(Digression: did you know libpng, the one everyone uses, is not supposed to be an optimized production library—rather it's a reference implementation? It's almost completely unoptimized, no really, take a look, anywhere in the codebase. Critical hot loops are 15 year old C that doesn't autovectorize. I easily got a 200% speedup with a 30-line patch, on something I cared about (their decoding of 1-bit bilevel to RGBA). I'm using that modified libpng right now. I know of nowhere to submit this patch. Why the heck is everyone using libpng?)
The worst offender (so far) is the JBIG2 format (several major libraries, including jbig2dec), a very popular format that gets EXTREMELY high compression ratios on bilevel images of types typical to scanned pdfs. But: it's also a format that's pretty slow to decompress—not something you want in a UI loop, like a PDF reader is! And, there's no way around that—if you look at the hot loop, which is arithmetic coding, it's a mess of highly branchy code that's purely serial and cannot be thread- nor SIMD- parallelized. (Standardized in 2000, so it wasn't an obvious downside then). I want to try to deep-dive into this one (as best as my limited skill allows), but I think it's unlikely there's any low-hanging optimization fruit, like there's so much of in libpng. It's all wrong that everyone's using this slow, non-optimizable compression format in PDF's today, but, no one really cares. Everyone's doing things wrong and there is no way to stop them.
Another observation: lots of people create PDF's at print-quality pixel density that's useless for screens, and greatly increases rendering latency. Does JBIG2 support interlacing or progressive decoding, to sidestep this challenge? Of course it doesn't.
Everyone's doing PDF things wrong and there is no way under the blue sky to make them stop.
Looking at the jbig2dec code, there appears to be some room for improvement. If my observations are correct, each segment has its own arithmetic decoder state, and thus can be decoded in its own thread. The main reader loop[1] is basically a state machine which attempts to load each segment in sequence[2], but it should not need to. The file has segment headers which contains the segments offsets and sizes. It should be possible to first decode the header and populate the segment headers, then spawn N-threads to decode N-segments in parallel. Obviously, you don't want the threads competing for the file resource, so you could load each segment into its own buffer first, or mmap the whole file into memory.
[1]:https://github.com/ArtifexSoftware/jbig2dec/blob/master/jbig...
[2]:https://github.com/ArtifexSoftware/jbig2dec/blob/master/jbig...
Funny you say that, https://en.wikipedia.org/wiki/JBIG2#Character_substitution_e...
> Another observation: lots of people create PDF's at print-quality pixel density that's useless for screens, and greatly increases rendering latency.
Is this relevant to text in the PDF? I would assume text is vectorized, meaning resolution is not relevant until you _actually_ print it?
Or is it just relevant to rasterized content like embedded images?
How about the folks listed as "Authors":
* http://www.libpng.org/pub/png/libpng.html
> Why the heck is everyone using libpng?
What is the alternative(s)?
The thing about diy'ing audio (primarily speakers but also amps, DACs etc) is that you can get top of the line performance for a fraction of the market price. A $50,000 speaker setup that would bring tears to your eyes could be made for perhaps $5000. A DIY $500 kit can perform similar to a $2-3000 set of speakers. Open source amps with gerber files on github are amazing.
The biggest reason it's so easy to get amazing value is because that $600 speaker only has $150 of materials. Upgrading its $25 woofer to a $80 one would help a lot, but no company would do that and not sell it now for $1000 if they could.
However the biggest allure for me is not beating commercial systems on cost, but making what I want. A small speaker with deep base? Easy. Speakers with quasi-active noise cancellation behind them? Sure, why not. Speakers that'll make the most overpowered/fancy beach-boombox sound like a crappy toy? Simple.
The only limit is your imagination and time/money.
I'd very much recommend diyaudio.com, but be warned, parts of this field are mature while others are still in effective infancy. Also, being an engineer (electrical/mechanical) helps a lot, there's a ton of signals processing and electrical/mech oscillation.
I will admit that stuff like "speakers with quasi-active noise cancellation behind them" sounds intriguing. That's probably a good reason to get into this rabbit hole!
Latest at https://fourays.lon.dev
I've just got to the point where I think I know what the module is going to be, but last night found out that PCB manufacture puts additional constraints on the PCB design, so I have to go back and re-do a lot of it, including probably dropping some features to make it simpler. The learning never ends.
I would now put learning CAD in the same category of mandatory life skill as learning to code. The ability to translate what you see in your mind to something that can be repeatably fabricated is an incredible power move, akin to learning how to communicate complex ideas with empathetic language.
My advice is to start by following this tutorial step-by-step. It's a 90 minute video that took me ten days to get through. Step two is to take an existing project and change it in a significant way. Step three is to create something from scratch which solves a problem that you have.
Learning G code allows you to start using CNC machines - even a simple 2 axis plasma cutter can do interesting things. 3 axis machine centers can make things that are quite remarkable. I am now stepping into 5 axis.
Here I wonder the same thing. Not that everything joyful must be productive. But if there was a way to apply this to something that was neat in the real world I think I’d be far more motivated to learn the skill. And enjoy it more.
However, I would recommend the open source https://solvespace.com! It hits a sweet spot between features vs complexity/learning effort. (And as a programmer I dig the terminal aesthetics)
I'm now looking at going down the Blender / render rabbit hole as Fusion can only get you so far.
Curious how you best learned how to communicate complex ideas with empathetic language?
I started out when I got a new phone, and didn't know what to do with the old one. One of the ideas I had was a homeserver. Turns out it's not trivial to run Docker even on rooted Android phones, and you need a lot of kernel patching, tweaking and more and it still had issues after that.
The next step was when I figured out I could install postmarketOS on it, and I managed to flash it, SSH into it and set up Nextcloud for our photos and unbound as a recursive DNS for my home network. I thoroughly recommend postmarketOS, and the contributors are amazing as well.
I was however running out of storage, so I ordered an 256GB SD card, and set up mergerFS between it and local storage, worked fine.
After some time however, I got paranoid about having and old device with a LiPo battery constantly being charged in my home, so I decided to get a mini PC from Aliexpress and chucked a 2TB SSD in.
In the meantime, I discovered Immich, which turned out to be much better for photos than Nextcloud, and fell in love with it.
The final thing I added was a miniDLNA service to play my local movies and shows on my LG TV without having to bother with Plex/Jellyfin and reencode anything. Unfortunately, it kept disappearing after roughly 2 days of operation, so I just added a cron job to restart it at 5 AM.
For the time being, I don't need anything more and am turning my attention to other things.
The challenge is that in an aquarium, plants grow reasonably fast but not fast enough to sell regularly for a decent income. You need ways to produce more plants faster, more reliably, and without taking up too much space. That’s where tissue culturing comes in.
It has reaaaally sucked me in. I’m culturing everything I can find. I’m also propagating aquatic plants through more typical means, and that’s fun too. I’m out of space though.
Tissue culturing is a really fascinating science and practice. I love keeping track of the media recipes, results, growth rates, etc. I’m too early to have had meaningful results, but I look forward to tracking those as well.
I've always really wanted to get into this but really don't know where to start.
Also, "Murashige and Skoog" is such a silly name; I love it.
The first attempt was over a decade ago, and I never really quite got it.
The second attempt was about 6 years ago, and while the fundamentals clicked and I improved dramatically as a musician/composer, I still only ever learned some small portion of the basics.
This time, I'm taking it a few steps further. I'm going back to the basics first to recheck my existing understanding of things, and really trying to take my time and understand each new concept before moving on.
I'm currently working my way through this online textbook: https://musictheory.pugetsound.edu/mt21c/frontmatter.html
I've also picked up a simple little budget keyboard (a 61-key Casio CT-S1) that does just what I need at a great price.
Having a lot of fun so far and learning a ton! :)
I realized 15 years after finishing music school how chord progressions would speed up my learning new songs on piano and learn playing jazz. So Im looking for any kind of literature that would improve my knowledge there.
Whenever I get spammed, I check the email headers. Turns out they were using an email with the owner’s initials in the unsubscribe header. From it, I was able to easily guess their actual name. I found them on LinkedIn! I used that, plus the list of all the domains they used for marketing and sending mails, to build a pretty comprehensive map of their operations.
I thought for a while about what I could do with this info… but in the end, reporting them to my country’s consumer rights authority for spam did the trick. No reason to get in trouble myself, as fun as it could be.
So the lesson is: look at email headers! There’s fun stuff in there!
https://ghostinfluence.com/the-ultimate-retaliation-pranking...
So I wrote https://github.com/arcuru/chaz to connect any LLM to Matrix so that I could use any Matrix client with any LLM.
That got me writing a Matrix bot account in Rust, and I couldn't find a simple bot framework so I split out a bot framework in https://github.com/arcuru/headjack
But _then_ I realized a really easy bot I could write that would also be very useful for me, and I wrote https://github.com/arcuru/pokem which is sort of a clone of ntfy.sh but using Matrix. Send yourself (or others, or a group) a ping on Matrix using an HTTP request or a simple CLI app.
[0]: https://matrix.to/#/%23thisweekinmatrix%3Amatrix.org?via=mat...
It's all sitting on my desk, first flight will likely be in May.
What sticks with me most through this experience is the brilliance of the open-source community (and a special satisfaction with now being able to say "I use nix btw")
Turns out getting particulates out of a solution is a massive, massive industry with a large body of science, literature, and engineering practice behind it.
EDIT: Here's a few wiki entries I found as OK overviews. ChatGPT was handy for figuring out what relevant literature in the field was and terminology I could use to find more pertinent resources:
1. https://en.wikipedia.org/wiki/Food_engineering
2. https://en.wikipedia.org/wiki/Ultrafiltration
3. Food Chemistry: https://www.amazon.com/Fennemas-Food-Chemistry-Srinivasan-Da...
4. Introduction to Food Engineering: https://www.sciencedirect.com/book/9780123985309/introductio...
5. Handbook of Food Engineering Practice: https://www.routledge.com/Handbook-of-Food-Engineering-Pract...
This took me down a rabbit hole on current methods to detect seizure onset... I came across a very interesting journal article on applying ML in an implantable that can detect seizures within 3 seconds, which spurred my current research on less invasive detection methods. Like any good rabbit hole, I've strayed from the original mission.
Seizures seem scary and I don't want to give them to people, but the causes of their onset seem to be too nuanced and patient-specific to build with any guarantees. The best I can do is avoid the obvious and hope the cutting edge detection and mitigation research bears fruit.
1. https://developer.mozilla.org/en-US/docs/Web/Accessibility/S...
https://github.com/apple/VideoFlashingReduction
https://developer.apple.com/documentation/mediaaccessibility...
Apple users can dim flashing lights in settings
---
Learning to skate is amazing. Once the motions start to “click” and you can start flying around the ice it is so much fun. Wearing full hockey pads makes the learning process a little easier on the body. Falls on the ice hurt!
Spring is a Java dependency injection system that uses XML-based configuration. Recognising that XML sucked, they later added 2 additional ways to specify configuration, which they call annotation-based configuration and Java-based configuration. Both kinds use annotations. Both kinds use Java.
Spring Boot is a layer on top of regular Spring that tries to make things simpler by automatically guessing what you're trying to do and configuring it for you, with something it calls auto-configuration.
Just trying to understand what makes what happen in a (very simple) Spring Boot app sucked weeks from my life.
I first discovered that there are tables (and tables and tables) of preflop card combinations that tell you whether to raise/call/fold based off of where you are in the dealing order and how many players are active.
Then I learn that's basically derived from monte carlo simulations, to calculate your chance of winning at any given moment (equity). It seems that it's probably more accurate to make your decision based off of equity and pot odds, i.e. Kelly Criterion. If I can get fast enough at estimating that in my head.
Also having fun trying to find open source libraries to do those simulations so I can create my own drilling exercises. Honestly, I'm having more fun doing that stuff than playing it.
The biggest dumbest problem is when blinds accelerate to the point that the minimum bet is greater than what Kelly would recommend. Some online poker sites are aggressive about this, to the point that you are forced to make irrational choices pretty quickly or get blinded out. A way to juice the house advantage I guess. It's almost enough to make me give up and find the next project.
The house does not have an advantage with poker. With player games, they earn money from hosting by taking vig(orish). But rapidly rising blinds in tournament poker is a way for them to get more games played, hence introducing turbos in tournament poker and Zone-type cash games.
Better understanding common betting patterns is more useful than hand simulators or whatever. Being able to accurately update your opponent's range throughout the hand is essentially how to play well.
I've been on a journey for a while to understand how to layout diagrams / graphs in an "aesthetically pleasing but structured" way. Long story short, DOT[0] is the best language I've found for defining graphs (compared to doing something with Mermaid.js or any other markup language), but rendering with the DOT engine in GraphViz fails the "aesthetic" test for me.
Did a bit of a literature review[1] to understand better the different approaches, and to understand the scope of the field. This book does great job of defining and providing the keywords for the different levels of requirements, starting with "principles" that are provable in the academic sense, to "conventions" that are like principles, but cannot be necessarily computed (eg NP hard, so requiring heuristics or simulations to achieve), and ending with actual "aesthetics" where things get very subjective.
Ultimately got pretty deep writing my own force-directed graph simulation in Rust and visualizing with egui[2] (needed an excuse to work on UIs and I've always wanted to write less Python), but I'm taking a break to use what I've learned writing Rust to shore up the REST API testing suite for my dayjob.
[0]: https://graphviz.org/doc/info/lang.html [1]: https://www.amazon.com/Graph-Drawing-Algorithms-Visualizatio... [2]: https://docs.rs/egui/latest/egui/
JellyFin. Amazing. A little clunky here and there, so some automation is needed. Then the weird bugs - gotta get a debug instance going to see what is going on. Once you start thinking about it as a "personal Netflix" you start building out a larger collection which needs organizing. Then your friends want an account. Then you realize your satellite tuner has a streaming function so you start reading through the plugin code for other streaming boxes...
Next one was the Japanese PS2 DVR combo unit. Sold as junk (which mostly they are - none can read discs anymore basically) but very interesting in that they are beautiful and cool and weird. The English OSD translations will often break your unit in subtle ways - so I joined the discord and identified that this was actually a configuration management problem - you have to only ever use a translation which was done to a set of OSD files which match your firmware revision. So I started writing a framework to auto-translate the XML file strings. Then the guys on the community mentioned the images with strings, as well as detecting when your translated string is too long for the UI element... I am still working on the framework. No dealbreakers yet! I will probably buy a few more to generate all my own OSD translations. My wife is going to kill me - I have 2 PSX DVR's now, and probably will buy more on our trip to Japan soon...
It's interesting to discover that a lot of 'facts' about public transport people take for granted just aren't true. The names and liveries by which vehicles go often don't actually correspond with their actual operators and owners. The company that's named on your ticket might not actually get any money from your purchase - or they might make money from passengers who don't pay at all, due to a myriad of subsidy schemes run by different levels of government.
Waves of privatization and re-nationalization with political motivations at every turn have produced a system which is amazingly efficient in some ways, and appallingly wasteful in others. Workforce strikes are obvious to the general public, but what's not obvious is who the negotiating parties even are, with various trade unions (and unions of unions) competing against various management groups (and groups of groups).
Some things are pleasantly surprising. Without any fanfare, digital systems for vehicle tracking have been introduced with remarkable efficiency. Then, for me, there's the astonishment of discovering that not only is every timetable published in a consistent, nation-wide data format, but one that has been utilised in production for twenty years!
It all makes me realise how limited the public discourse about public transport and 'green' mobility policies are in my region. It is simply impossible to grasp the true consequences of any given proposal in the meagre columns that they're given in the newspapers and the two-minute reports in which they feature on the television. Diving into this rabbit hole has led me to respect the complexity of the field much more than I did before, and fills me with both hope and despair on topics which I had hitherto scarcely lent a thought.
Wife and I live on a floating house in Oregon outside of Portland. Taking my paddle board to various beaches on the river to pick up trash has become a beloved hobby of mine. Fantastic way to get exercise, get out in nature, and clean up my community.
I kept finding old, antique bottles during outings: Coke bottles from the 40s and 50s, little medicine bottles from who knows when, old beer bottles from brands no longer in business. I eventually learned that there's a whole hobby around this called "bottle digging", even an active subreddit (r/bottledigging).
I've since bought a nice snorkel and mask and have been diving for old bottles on clear days, when river visibility allows it. Thinking about taking it to the next level and getting scuba certified.
If you like finding things, you might like bottle digging.
I've also learned how to make pendants out of them.
Although, if I were you, I'd be looking for old necklaces, watches, etc. in that river. It excites me to imagine finding some old gold coins or something.
We frequented a subdivision that was being built on land that was a former city dump.
Turning a thousand-page book - PAIP - into a stack of Markdown files in a git repo, readable online. The print book received more editing and revisions than the ebook. I converted an ebook's ... odd formatting ... into Markdown, remade diagrams, generated new ePub and pdf files, and had the spine cut off a print copy to make a fresh scan. Working on that scan, I made Scantailor, an X program, easier to access from a Mac, via Docker. I tried different OCR engines, and pored over the diffs, incorporating dozens (hundreds?) of improvements. I got to find so many differences between Markdown engines. I have ideas on how to make Pandoc links between chapters. There's still a lot to do!
My current WIP: Lars Wirzenius posted about file systems with a billion (empty) files. I started exploring because I was curious, if I was remembering correctly, how well a mostly empty image file would recompress - like, drive_image.gz.gz. Lars offered a Rust program; I was curious about how other methods compared. Like, how about nested shell loops, tar, and touch? And, hey, how well can we archive and compress them? I've gotten to see some issues, bottlenecks, and outright failure modes with SMR hard drives, Samba re: sparse files, and parallel gzip compression. I've accumulated some shell script boilerplate to make it easier to go back and verify my processes, and harder to accidentally wipe out past work if I rerun it.
Did some duolingo for french just recently and already did for latin two years ago. It’s difficult to read them but broad context is enough for an overview understanding and I feel a detailed understanding increase by the chapter from exposure alone.
Apart from the actual books I enjoy bidding, similar to but healthier than gambling on stocks, and “winners curse” of overpaying for stuff I really like eh I’m aware of it but the dopamine is worth it too. Some arbitrage opportunities as well, as in finds on local auctions and selling online internationally, but usually international shipping for heavier stuff like entire œuvres and just the time investment to pack and ship something deters me from doing that. Wouldnt want to damage 1600s books in shipping I suppose as well, they look quite cheap locally but fetch more internationally.
I love how many absolutely insanely interesting books from that period aren’t affordable. I have a copy of the book in which Bishop Ussher put forward his theory that the earth was made on "the entrance of the night preceding the 23rd day of October... the year before Christ 4004"; that is, around 6 pm on 22 October 4004 BC.
I use as a cautionary example of data driven analysis. It was all from lineages in the bible and ages mentioned there. Seemed like a solid methodology at the time!
- I’ve ditched VSCode and gone all-in on NeoVim. I’ve spent a bunch of time watching Primeagen, etc., tweaking my vid config and learning how to navigate as efficiently as possible.
- Switched from QWERTY to Colemak-DH to hopefully reduce RSI. I’m at about 70wpm with decent accuracy after 4 weeks. My QWERTY skills are gone. I like Colemak, but we’ll see how I feel in another month or two.
- Finished my custom hot swappable Sofle keyboard, and spent many hours customizing the layout. I think I’m pretty close to feeling comfortable. I’m using home row mods, which I love. Currently using Kailh box whites (clicky). Might switch to Gateron Brown Pros.
- Been going through a “Build your own git” course, to understand git as deeply as possible.
Something is incredibly beautiful to me about classifying the kinds of symmetry things can have.
I’m trying to understand where the sporadic simple groups come from. Starting with the Matthieu groups. So far it seems to be due to some anomaly in Pascal’s triangle, but I’m still trying to put it together. “Another Roof” on YouTube has a good video about this.
I wanted to be able to do this cross platform, so I re-implemented ELF patching and Mach-O patching and adhoc signing in Python, and wrapped them into a tool called repairwheel: https://github.com/jvolkman/repairwheel
That guy was busy with other tasks, so iterating on firmware was too slow. So I decided to dive in. I mean I knew C a bit.
So I had to learn STM32 arm, I had to learn low level C, I had to learn assembly, I had to get some understanding of those electronics things to get some sense of it, I had to read tons of manuals and datasheets.
Long story short, I rewrote this PoC firmware into something I could bear. It's so nice to control all software from the start to the end.
Now our company wants to rework this device into "smart", add display with touchscreen and stuff. So I'm digging into embedded Linux programming, LoL.
I'm generally consider myself full stack developer, so I can write frontend, backend, kubernetes, setup servers, deal with cloud stuff. However digging that deep feels like testing my limits.
please don't do that. Add some kind of interface, eg bluetooth, and an open-source app, or open-source/documented protocol, to control it.
0: https://ajxs.me/blog/Introduction_to_Reverse-Engineering_Vin...
1: https://archive.org/details/Hitachi-DotMarixLiquidCrystalDis...
I've been attempting to add a oauth2 device code flow to a Tacacs server with the goal of extending Azure accounts to access network device management planes. Pretty neat, I can get a "enter this device at URI" from the router/switch and let Azure do it's 2fa/compliance etc. Currently trying to get token validation working on the tacacs server =).
Ultimate goal is have a reverse proxy web front end kind of like Apache Guacamole that does the Oauth for the user and when they click on a network device, the JWT is passed through to the network device over SSH and thus the tacacs server which is relatively local to the network device which will validate it and let the user into the network device.
Playing around with GPT4/Opus a lot lately and man... I have feelings. They've been a great learning tool to learn the basics of Go though so I'm thankful.
It's going swimingly /s but I seem to be making progress. Slowly, I'll bake this into my bigger network management tool if it an be secure and make sense to do so...
Add to the fact that you have to manually reload the unpacked chrome extension to apply your new changes, so I’ve hooked solution 1 up with a web socket server and a custom chrome extension that watches all your other extensions and talks to the web socket server to auto reload an unpacked extension anytime the build step completes
It’s a nightmare hack, but I may just be the worlds most productive chrome extension builder as a result! I’ve released 5 extensions in the last 5 months :)
About to release a big one, but it’s also probably the words most complicated chrome extension (LLM + Firebase store + Stripe + Auth + Serverless functions)
This weekend I was able to reach my home node from a state park 8.2 km away and have been giddy since.
[1] https://learn.microsoft.com/en-us/sql/tools/bcp-utility?view... [2] https://babelfishpg.org/
Fast forward into my first job, there was a Windows machine with a crashed HDD and my task was to recover as much data as I could. Windows tools all sucked hard, even the ones that we had at the company that cost thousands of bucks per month. That's when the power of the Linux ecosystem hit me.
Went on to being an IT forensics guy, then went into pentesting, then into blueteaming and now I am having my own startup that builds a better EDR software.
I still have to think about that coincidence. Literally nerd-sniped my life, otherwise I probably would have still been a sysadmin or something.
On the way I learned a ton of new things, from programming languages and compiler bugs to exploit techniques and kernel development and even hardware design. If you go deep enough in hardware pentesting, the whole phreaking scene is amazingly welcoming. The CCC chapters are also amazing, and there's just so many opportunities to grow your knowledge and experience in the field. It never gets boring!
- 3D-printable parts storage solutions (via: I found some part storage bins in the discard pile at a local hackerspace)
- MITM proxy to snoop on Github Copilot API requests (via: we're building an jupyter AI assistant thing and got curious how other players do it).
- DIY robot arms (via: I'm making several for a nested 'you pass butter' joke, via a casual conversation about robotics being accessible now. YouTube is amazing at surfacing smaller makers once you start watching a few videos on a given topic)
- Learning about Oauth and JWT (via: 'why is auth still a pain?')
- Invertebrate UV fluorescence (via: that millipede is glowing under my UV torch!)
(a small subset of these end up documented https://johnowhitaker.dev/all.html eventually if you're curious to see a longer historical list)
I like rabbit holes where following the curiosity gradient to a satisfying conclusion is possible. "How does X work" leads eventually to code that does X. I'm less happy when they lead into a tangle of complexity, like digging into a library only to find weird abstractions 6 layers deep or trying to compare 18 different alternatives in a field I don't know very well.
OP I'd also like to hear yours!
Today I gave some thought to what would be a fitting name for my boat (if I were to rename it).
One option: the glider pattern from Conway's Game of Life. Instantly recognizable by true hackers, just a weird symbol to others.
Of course a quick check on Wikipedia. Know that I'm always interested in things small / simple / computing, so... cellular atomata. Which led me to varieties used to simulate or help understand biological systems ("systems biology" - if only that field had even existed back when I left high school).
From there on: artificial life, Core Wars & co, self-replicating machinery, and... Astro-chicken (deserves a HN post of its own, imho).
Btw. it's amazing to see how many big, open questions there still are, related to the origins of (biological) life, and evolution. Eg. full simulation of a single cell organism: never been done (too complex).
Next up: a cup of hot chocolate.
Fast forward a few years... I just wrote a book on the timing topic, called Why Now: How Good Timing Makes Great Products.
I got into it through wanting some cheap radios to keep in my house, so I wouldn't have to go all the way upstairs when I needed to communicate with my daughter.
Well let me tell you Baofeng radios are extremely cheap but really flexible. I got these things for the simplest possible use case but after realizing their potential I just had to learn more about the space. You can adjust their configuration with a tool called Chirp and you're off to the races!
I attended a local severe weather awareness event where I met some hams who were part of an emergency response network. It's really cool to learn about how these communities operate. It's legal to receive even without a license - you only need the license to transmit.
I plan to take the technician test soon and get my license so I can help out at a nearby bike event. The area is incredibly rural so there's no cell coverage and the ham operators are really helpful in coordinating things.
Anyway, I feel like the hobby is a bit of a dying art, but it's something that seems like it would have a lot of appeal to the programmer crowd.
If you have a repeater in your area there may be a net where other licensed operators check in every week. I've heard people check in from over 60 miles before. You can check this website for repeaters in your area: https://mygmrs.com/repeaters. Good luck on your technician test!
One broken engine and one non operational one and turning them into a single good motor. American thin-wall cast V8 engines are fairly similar, but different enough that if you don't get them built you have to do a bit of puzzle solving (especially in the timing case). Plenty of youtube videos and forum posts on the Cleveland and it's been fun piecing it back together and learning about new things like installing cam bearings.
Trying to compare good query plans with bad ones, and then work out what changes we need to make to the slow queries is ... interesting.
Ostensibly I wanted to be able to code on the production server like a miscreant with the same tools as my laptop.
However I just wanted to regain command of my dev environment after years not coding.
I also reorganized the furniture in my office and got weirder lighting to make it hacker friendly. I bought a new desk to solder electronics.
Most people know me as a partnerships marketer or product manager but I am a compsci at heart. This made me happy.
I researched what topics are typically included in a 4-year math major university program and what textbooks are used to teach those topics at MIT. Then started grinding all the way through from beginning to end.
It was so awesome that upon finishing, I promptly started all over again... but with physics instead.
DOI (Digital Object Identifiers) are used by many modern research papers as sort of a UUID for papers, run by doi.org.
But they're discipline-specific. So they're used widely by certain disciplines. But others use different databases.
So for biology-related papers, NIH's PubMed ID. Or for Astronomers, Bibcode.
All are "global" identifiers and each has some kind of consortium that's trying to make theirs the One ID. DOI seems to be the closest.
There are several registries, crossref is the big one in the west but it's not the only one. They have probably the best access to the data out of all of the larger registries though.
Dois are pretty good, though not persistent and there's no versioning built in so people have their own formats.
I spent a lot of time working with these as part of https://dimensions.ai for a decade or so. Happy to chat if you want to delve in more.
I saw my grandparents go through 10 years of doing all the retirement things(beach, hobbies, social hour etc) to keep them more than busy. Then they kind of did it all and just started having social hour earlier and earlier every year into their 80s. Cocktail our started about 11:30 am in the end.
I'm starting my retirement and have so many interest/subjects I want to learn so feel like I can stay pretty entertained for more than 10 years but maybe not.
A while back I had bought a domain for my email, and I thought “I should write a blog about creating a blog”. At first I hosted it in GitHub Pages, but then I realized I have a perfectly good Raspberry Pi. It’s not like I’m ever gonna get a lot of traffic… so why not self-host?
That sent me into a very deep rabbit hole. How do I make sure my website doesn’t go down if my IP address changes (no static IPs for me, sadly)? How do I create and automatically renew a certificate? How do I achieve high availability?
A few years passed, and now I have a cluster of a few Raspberry Pis running Docker Swarm, managed by Portainer, with stacks running multiple websites and services I self-host. I’ve learned a lot!
My next move is going to be a full overhaul: Docker Swarm is blocking me from setting up some things the way I want to, so I want to build a new cluster using Kubernetes. I’ll use the opportunity to overhaul the network layout as well.
The funniest part is that I haven’t written a single blog post in 3 years. I wanted to add responsive images so I could add diagrams and photos. Somewhere along the way I realized I shaved too many yaks.
Cloudflare tunnel would solve the issue of static IPs and you also get DDoS mitigation and caching. Caching on the edge would be especially beneficial for something like a blog which is likely to be fully static or SSG at most. Although it won't help you with the writing part (;
https://www.youtube.com/watch?v=5mmISldi060
I realise now that I've spent most of my career just shaving yaks.
My current rabbit hole is tuning my home's boiler to be more efficient in its use of gas. It is an interesting engineering problem because the lack of feedback loop since the thermostat is dumb and specific home variables makes smart thermostats useless as well as boiler sizing more complex than most installers understand. My goal is to add some features specific to my home to reduce gas consumption based on more variables - outside temp, minimum outside temp, sun/clouds and home variables like brick wall temp, fireplace heat, boiler controller settings as well as the wifes 'I'm cold' variable.
I am using a microcontroller and custom current switch as well as IOTSTACK to send inlet/outlet temp and gas valve & circulator state (on/off) to influxdb/grafana so I can see what is happening between thermostat and boiler controller. I have identified a few freebies in terms of consumption and inefficiencies. I have added a relay to delay the gas valve once the boiler starts cycling to reduce "short cycling" which is a waste of gas on startup and a mini explosion every time gas lights. I have managed to reduce cycles in half which helps with wear and tear as well as the number of boom sounds coming from my boiler room :)
I would love to go down the simulink rabbit hole but I think I will not.
What do you mean my launch.json file is missing? It was there yesterday?
Wait, I can set up custom launch settings in my launch.json? What else?
Ok, so I've got seven different launch settings in there, and now to see if I can have one used for markdown for my markdown word editor.
Oh, neat, lots of extensions for markdown.
Wait, you can install vim?
An hour later, and I've completely re-broken my VSC and am reinstalling from scratch.
For those who are curious, it is not one single law that makes this the case, which is why many people will tell you that it's still allowed. However, with the combination of privacy laws forbidding the use of aerial cameras, UAV regulations limiting the locations you can fly, limitations on certain radio bands as well as specific rules about maintaining line-of-sight, it is not usually possible. How could it work? Perhaps if you live in a rural area with no nearby air traffic, have permission and access to use a large patch of land (such as a farm) from which to take off, a willing partner to maintain line of sight (which of course precludes many of the stunts often associated with competitive flying) and you have done the paperwork. Not really hobby territory for most people at that point :(
Is it possible to start super cheap just to see if I like it and then upgrade?
Spent WAY too much time adapting a Buddhist sutra into a heavy metal banger: https://www.youtube.com/watch?v=H-5Y9Z7DK4s
Now I'm trying to abandon 30 years of muscle memory and typing at 4 wpm while I learn Colemak-DH. Maybe what I should really do is build a custom 34-key board...
[0] https://github.com/benhamad/blog/blob/main/2024-04-12-dramal...
I'm building a paperweight inspired by vintage brass table lamps to hold the papers in place on a wooden platform.
The REAL rabbit hole is the astounding amount, and quality, of AUv3 plugins for iOS. Sounds, effects, looping tools, MIDI things, just... wow. And almost all of them are under $20, and many are free! I've spent less on a dozen software toys than on the first two guitar pedals I got. And infinitely more powerful.
Check out this video of someone doing the looping thing way way way better than I'll ever be able to (but it's fun to work towards a goal). Software she's using is called Loopy Pro, another amazing thing:
Its almost certainly a rabbit hole but at least I’ve forewarned myself.
Electric Bikes: Hub Motors and Mid drives are a really great spring / summer rabbit hole to go down. So many form factors of ride and you can also kill two birds with one stone by going for 60V lawn care equipment (There's adapters on eBay to connect them to your bike).
- Designed/built a small USB controlled pan/tilt camera head to control the mirrorless I use as a webcam (couple of servos, gears, belts), and then designed/built a custom ortholinear keyboard with a joystick to control the camera (custom PCB, CNC'd aluminum case, etc)
- I'm a pretty big runner, built my own web based calendar UI that integrates with Google Calendar where I can type in workouts like "1 mile warmup @z2 + 5x(30 seconds @ 6:00/mile + 0.5 miles recovery) + 1 mile cooldown" and this gets parsed/total weekly mileage gets tallied. The next step down this rabbit hole is building a small iOS app to automatically generate Apple Watch Workouts using WorkoutKit.
The reason why I want to learn more about it is I feel it is like a base building block of distributed systems and may be easier to grok and even write a toy version than a bigger thing like kubernetes or a leaderless distributed datastore. I would also learn some go and know how a critical piece of kubernetes works.
What led me there is practicing for a damn system design interview. As much as this whole topic is controversial on HN the grinding has really got me curious about the tech that runs at larger scales and how it works under the hood.
I just got the box finished up over the weekend and it's working really nicely so far. I went with an MSI MEG x670e "Godlike" motherboard, AMD Ryzen 9 7950X CPU, MSI Gaming Radeon RX 7900 XTX GPU, and 64GB of DDR5 RAM. The whole thing is running Ubuntu 22.04 (I was going to use Alma Linux, but the Alma installer wouldn't even start), ROCm 6.1.0 and Ollama. So far I've mostly been working with the LLM stuff using Ollama and the llama3 model.
Now I've started using Spring AI to interface with the Ollama API. Next steps: figure out "function calling" with Ollama (which doesn't seem to be supported by Spring AI yet, boo) and some "agentic workflows" and multi-agent stuff...
It started with an article about the hypothesis that planet nine may be a primordial black hole with 3-6 Earth masses.
What’s a primordial black hole? It’s one that formed in the first seconds after the Big Bang. We don’t know for sure they exist but many theories and simulations predict them.
They’re an excellent dark matter candidate. Could it be that simple? Could at least a lot of the missing mass be tied up in little baseball sized embers from the birth of the universe that rarely interact with anything so we don’t see them? They’d be small, would rarely interact, and unless they are sucking in mass (causing a hot accretion disk) would be dark.
Then I got onto Hawking radiation and whether micro black holes could exist. Along the way I read about loop quantum gravity (LQG) which looks to me like a decent stab at unifying QM and GR that’s much less baroque and more testable than string theory.
That then led to the LQG “bounce” hypothesis for black holes. See LQG does away with true infinite mass singularities. Instead a black hole would be matter packed to its theoretical maximum density (which is still insane). From there it would quickly “bounce” and become a white hole.
So wait… how do black holes persist then? Time dilation! From a the hole’s frame of reference it collapses and then instantly bounces and goes kaboom. From our frame of reference though all that gravity slows it to such a crawl that the black hole phase at or near max density looks like it’s stable. The bounce takes billions to even trillions of years!
Last but not least I learned about the black hole starship idea. It’s a set of ideas about how far future intelligences could use black holes as mass energy converters to reach relativistic velocities. Might be somewhat easier (for crazy sci-fi values of “easy”) than handling antimatter. This also gives SETI yet another wild extreme technosignature to look for.
… and back to the beginning I found a post about how if planet nine were a PBH we could use it to yeet probes to the stars at meaningful fractions of c… at least if we could make them able to survive insane g forces. Unlike the black hole starship this would be feasible today. It’d just be a gravity assist off a ludicrous gravity well.
Here I thought black holes were dull. Turns out they’re the most extreme objects in the universe and a whole lot of the most amazing physics intersects around them. If there is any way we could tap into the phenomenon we could potentially access sci-fi levels of energy too.
It's been a while since I've worked on a CRUD app so I'm finding the whole thing quite interesting. The purpose of the app is to solve a scheduling problem.
I've written my own CDCL SAT solver (now just using google or tools), and on the app side I've jumped from Phoenix (elixir) -> Dream (ocaml) -> axum (rust) -> Django. I feel like Phoenix probably perfectly suits what I'd like to do with this app (long running tasks and collaborative editing) but I'm at the point where I want to support this app long term and I don't see me not being familiar with python anytime soon.
Bookbinding has fascinating details.
Crazy how familiar and yet different things are.
That's exactly the kind of topics they explore.
It's fun to reach a point where things just run forever at zero expense.
In parallel, I work hard on developer experience just for myself. I finally get the greybears and their keyboard incantations. No UI can beat scripts and muscle reflex.
an amusing AI generated origin that entertains the humans sufficiently well, while solving some frictions the humans encountered
basically somebody launched a crypto token and rugged it for some quick cash. it was a hit because their code, coaxed on and perhaps entirely generated by ChatGPT, did fun things:
- issues an NFT from a collection whenever you buy a whole token, but not when buying fractions
- transferring the NFT transfers the whole token
- while selling a non-whole unit of the token irreparable burns the NFT
community said “no that was awesome lets keep doing that” but original dev could only think about doing more rugs, so some in the community made Pandora token as the new reference implementation and pushing this draft standard’s development along
in reality it solves some frictions with token + Nft launches, any NFT collection launch, and multiseries management within 1 contract. Something ERC1155 tried to achieve in a more complex way. now people can launch NFT collections directly from an AMM’s liquidity pool like Uniswap. No direct boutique websites or NFT marketplaces needed to trade the NFT or create liquidity. And has a fun incentive for holding on to the tokens.
so far in practice, communities make separate markets for the NFT's with them trading at a premium to the token, as the NFTs in the limited size collection gets burned over time by sellers and has its own scarcity attributes while the fungible token trades at a different lower price, allowing for roundtrip arbitrage. Automatic community with the arbitrage features. ie. a Pandora NFT trades at $7,000 and comes with 1 token. While the token market alone trades at $5,800.
Guardrails: No longer do we have to use the AI to fight AI, AWS has presets for blocking inappropriate content going into or out of the AI block.
Evaluation: Basically just give it a JSONL file of all the inputs and expected outputs, and compare two models or judge the quality of a model. This can be done by a human (it even manages the access of this team of humans). But more interestingly, it can be automated or done by AWS workers.
Knowledge base and agents: Seems like it's only for Claude, but damn, Claude has been impressive for a good price. Claude Sonnet (the mid-range) has been about as good as GPT-4 for half the price. The weakness has been things like knowledge base and AWS lets us just quickly set up vector DBs and embeddings. Then the agents feature lets us connect with it and combine it with all of the above.
I first wrote it the dumbest way possible, one big array with padding at the back. Worked fine actually for most modern use cases, but as this is also a learning experience, I want it to be best in class in performance.
I think I'll settle on gap buffer because the performance is great and it doesn't hurt my head.
An online game I play includes an optional two player Russian Roulette type feature (non-fatal). I got to wondering if there was an optimal betting percentage to use, if you set aside some money as a betting seed. So I spent time coding up a really ugly brute force "just run lots of games and see".
Pretty much the answer is you'll lose more often than you win, looks like your best bets are around 2% of whatever money you have left of your betting money.
If you play 75 games, at 2% of your betting pool, you'll come out ahead only about 49.8% of the time.
There's more efficient ways of working that out than I bothered to do, which was to create a basic abstraction for a gun. For example, your odds of winning is essentially 50%, given two players. For every "game" I simulated, I could have just picked a random integer between 0 and 1 instead. Faster and the same effect.
As best as I could find, there are no good betting strategies on a coin toss (which is what this really is)
This is usually called https://en.wikipedia.org/wiki/Monte_Carlo_method
More than just using some formal language/tool i had wanted to learn about the Mathematics/Ideas behind formal methods and how they are embodied in TLA/Z language/B method/etc. After a survey of available books i zeroed in on Understanding Formal Methods by Jean François Monin hoping to get an overall idea of how everything comes together. But what i got was a fire-hose/mishmash of so many different sub-fields/notations/abstractions used in the field that it is quite a struggle to get a good grasp on anything. The author's style of writing is obtuse/challenging and the contents are more of a survey/introduction than detailed explanations.
The result is that i am now interested in figuring out a whole lot of mathematical/logical sub-fields which i suspect is going to occupy a lot of my time in the future.
Intermediate step - I feel pretty confident with the gory details of the kernel code now. can possibly build a custom kernel, boot qemu with both a simple C+assembly bare metal kernel, or the self compiled kernel. I feels like the clouds have cleared and I can see the sun. - Incidentally the kernel source code is pretty well documented, but one thing which is missing is a much smaller list of files which are most important. true pareto here. 20% files carry the weight. You also need to know the subsystem you want to touch. Chances are that subsystem is much lesser number of files.
Finally - Got to reading about kernel packet handling. at the L2/L3/L7 level. from nic hardware to userspace. Turns out that eBPF [hello old friend!] has a networking avataar called XDP which is pretty recent [<5 years] way of doing high performance networking on the linux kernel. Along the way, got to know about network performance optimizations specially in modern multicore systems in the kernel like RPS/RSS/aRFS, DPDK/fd.io/VPP.
And now I feel the itch to apply this to some of our networks. Particularly, baremetal servers on equinix metal + aws ec2 + azure can be peered with either VPP/Bird to make a p2p connection which is a factor more performant than the vpc interconnect/gateways which are provided off the shelf.
I might extend the holiday by a few days. and I would love to talk to people who have hands on experience with any of this. Its hard to contain my excitement tbh.
I play with io_uring and multithreading. I am looking at event loops.
My one, at the moment is precision time keeping. Time nuts. I have a pile of oven controlled crystal oscillators, GPSs (even the full Ublox time specific LEA M8).
I got a BG7TBL counter and multiple cheap GPSDOs to test my own.
I have a DAC1220 20 bit DAC on an ESP32 disciplining a TCXO from an old phone base station by counting the 10Mhz using the PCNT and gated off the GPS 1PPS.
Meantime, I learnt the esp-idf so I could have more control over things. Everything done with the esp-idf is way way more stable than using the Arduino wrapper, no idea why, maybe later versions?
The disciplining/tracking parameters are exposed by http and mqtt and put into influx.
I have a 5.5 digit multiple meter (I repaired the classic HP 3478A). Maybe I need another digit, there goes another rabbit hole. Voltnuts.
https://www.sweetwater.com/c953--Synchronizers
https://en.wikipedia.org/wiki/Black_and_burst
https://www.sweetwater.com/insync/black-burst/
https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2334524.m...
Might be interesting to convert an audio precision word clock into a precision computer clock. Audio atomic clocks are available ($7000):
1. Looked at what it would take to turn it into a sort of "pubnix" for some friends
2. ...which got me looking into how to set up Postfix to manage local emails (allegedly this works out of the box, but I must have screwed something up since I never did get my test messages from one user to the other)
3. Then on to looking at BBS systems, starting from Enigma 1/2. Didn't get too far into that since the theme customization scared me away (and not enough of my RL friends are nerdy enough to get into it)
4. Finally backing away from the pubnix thing again because of insufficiently nerdy friends (although one is humoring me in experimenting with SSB), I then instead set up a Synapse server to have my own identity in the Matrix ecosystem.
It's more fun than I thought because whipping up every cocktail you've ever heard of is actually quite simple. There's also some cool generalizations to uncover so that you can Pareto principle your way to knowing how to do almost everything with a few tricks and ~10 liquids. I could have someone knowing more than 2/3 of bartenders (most are just a job / basics) in an evening. Then once you've got a good handle on all the basics, there's an endless adventure into variations and history. There are a lot of flavors out there you wouldn't believe existed; many concoctions of herbs and botanicals. It's a great social activity too.
I've developed my own film in the past but knowing so little about chemistry myself, it's still pretty much magic to me even after digesting all of the info from the series.
I've been looking at digital backs for old cameras such as the blad I have taking dust. Sadly, either they're completely impractical or they're way above my budget. Hopeful that some day, something better than a polaroid back can be used to resurrect my old hassleblads.
I wanted to be able to easily distinguish between the 60+ figurines of the Zombicide board game, so I figured I could paint them "quick and dirty".
Well 8 months later, I'm not finished because I "had to" learn about paint, color theory, paint mixing, human vision, brush types...
Being colorblind I gave no attention to colors around me, but I have since discovered I can see more shades than I was aware, and I'm having a blast just looking at the foliage... Which does not speed up the painting!
Otherwise, ThePrimagen pisses me off always talking about Neovim and Vim motions, so I am right now in the painstaking process of learning Vim motions and I want my life to end because of the learning curve.
Ended up getting a simple rust function built, with some slight miscomplilations via wasm + wasm2c. Now I'm going to try to get graphics working.
There is a surprising amount of public code containing calls to sony's licensed SDK, if you know what to search for (Not to mention the SDK, which was, "obtained" dubiously). Fascinating stuff
I was playing with w2c2 last week to try and get some stuff running on Mac OS X Leopard, but I couldn't quite get it working :(.
Watched a couple videos on making AI to play video games, looked into localai and gptpilot, failed to get it to write a QT application. Played around with tts and stt, and now I'm stuck in prompt hell with diffusers/dreamshaper.
(It's mostly there, but I feel like it's screwing with me because it always finds a way to screw up the picture.)
I don't currently use it in any serious projects aside from tinkering about with it, but it has been a lot of fun to learn and study.
Between the Nix package manager, the associated language, etc., there has been a lot to learn about, and it's been good fun. I have nixOS on my spare Thinkpad for toying with, and I have Nix on my main Debian systems, if I want to pull something from nixpkgs.
* Turns out the work is not 'weird' or 'Gnostic' but is directly addressing details from Lucretius, including paraphrasing his view of evolution and atomism, but refuting the claim there's no afterlife by basically appealing to the idea we're in a simulated copy of an original physical world where the spirit doesn't actually depend on a body, because there is no actual body.
* As I dug more into the various mystery religions the followers of the work claimed as informing their views, I saw a number of those were associated with figures various Greek historians were saying came from the same Exodus from Egypt as Moses.
* Turns out a lot of the ahistorical details in the Biblical Exodus narrative better fit the joint sea peoples and Libyan resistance who end up forcibly resettled into the Southern Levant latter on. In the past decade we've also started finding early Iron Age evidence of Aegean and Anatolian settlement and trade previously unknown in the area, including in supposed Israelite settlements like Tel Dan, lending support to the theory that Dan were the Denyen sea peoples.
* Also turns out that in just the past few years a number of Ashkenazi users have been puzzled by their genomic similarity to ancient DNA samples, where the closest overall match in a DNA bank was 3,500 year old Minoan graves sequenced in 2017 or that they have such a high amount of Neolithic Anatolian (which the 2017 study found was effectively identical to Minoan).
* The G2019S LRRK2 mutation that's almost only found among the Libyan Berbers and the Ashkenazi appears to have originated with the former but appeared in the ancestry of the latter ~4,500 plus/minus 1k years. Which is a window that predates the emergence of the Israelites in the first place, but is on the cusp of the sea peoples/Libyan alliance.
* There's also been discovery of endogamy among some of the Minoan populations. Did the Ashkenazi endogamy evidenced from their emergence in Europe and the bottleneck in the first millennium CE actually go back much further than we've been thinking? Maybe Tacitus wasn't so off base when he talked about how some claimed the Exodus involved people from Crete hiding out in Libya.
Anyways, that's a very rough summary of some of the rabbit holes I was going down.
Bonus: Herodotus's description of Helen of Troy spending the whole time in Egypt has two datable markers to the 18th dynasty, which is when Nefertiti, "beautiful woman who arrived" is around during a complete change to Egyptian art and religion while she's the only woman in history to be depicted in the smiting pose, with her only noted relatives being a sister and wetnurse.
Sounds a bit gnostic no?
Despite leaving a bug ticket with my library, I received no response after a month. I struggled to debug this library within a library. One day I realized the rabbit hole I was in and switched over to Tornado and no problems since.
Using semantic-router for dynamic conversation routing, and LiteLMM for model providers.
It was lots of fun to learn and build. I will be adding function calling support (tools use) for the models to have it more capable, like an agent, in the future.
Thanks in advance.
One person brake fluid replacement / bleeding procedure after replacing a brake master cylinder. For my car, I can now do this without taking the wheels off. I'll be done within a couple of days and after that, I'll be kind of an expert.
I've actually proved it several times...except for the insignificant detail that I glossed over that didn't seem important but tanks the proof.
Someday I'll have to publish my "book of lemmas that don't prove the collatz conjecture."
Some history podcasts had me digging into the Napoleonic Wars and Israel/Palestine.
Also a recent interest in human health and diseases has basically sent me down the path of self-study equivalent to a Kinesiology/Exercise Science/Sports Physiology degree.
I knew about most of them a little or fair bit already, but there's always something to learn the deeper you go :)
Been tinkering with that for an absurdly long time vs just throwing some python onto a $5 dollar VPS but it’s been fun & learned a decent bit about over engineering pitfalls along the way
Somehow and unintentionally as the search began from some random article I’d read, this seemingly unrelated subject ended up uncovering some insights into a problem with deduping database rows I’d been working on for another project.
Margaritas -> Jello Shots -> Chimoy/Tajin rim/topper -> Pop Rocks -> History of Pop Rocks
ChatGPT gave some nice code, but it wouldn't work at all, good ideas though
I then set up a VAX 11/780 running OpenVMS 7.3, which I can telnet into from outside. ;-)
Even today, I'm seeing conflicting reporting of quotes. For example the CBC has one article that quotes the police chief:
> Outside court, however, Toronto Police Chief Myron Demkiw struck a different tone. "While we respect the judicial process and appreciate the work of everyone involved in this difficult case, we were hoping for a different outcome," he said.
https://www.cbc.ca/news/canada/toronto/umar-zameer-acquittal....
However, the video of the Chief's statement is slightly different (and quoted correctly in another article):
> "While we respect the judicial process and appreciate the work of the 12 citizens who sat on a very difficult case, I share the feelings of our members who were hoping for a different outcome," Demkiw said.
https://www.cbc.ca/news/canada/toronto/umar-zameer-verdict-1...
The first quote cuts out essential information and does not note that the quote was not verbatim (as presented). It's especially problematic as the quotes might have subtle but different interpretations now that his suggestion of "hoping for a different outcome" is being addressed. (There's a subtle difference between 'sharing feelings with someone that hopes for a certain outcome' and 'hoping for a different outcome' -- and this could be major if it's seen as the police services suggesting the wanted the man, who was declared innocent, to have been convicted.)
(One other possibility is that the Chief did in fact mention both lines, and that they are so similar because it was based on his official public statement).
I later ported the game to the ZX Spectrum, because that was a fun challenge, and I only needed a few basic I/O operations - "write to screen", "read a line of input", etc, etc.
It occurred to me that I could reimplement the very few CP/M BIOS functions and combine those implementatiosn with a Z80 emulator to run it "natively". So I did that, then I wondered what it would take to run Zork and other games.
Slowly I've been reimplementing the necessary CP/M BDOS functions so that I can run more and more applications. I'm not going to go crazy, anything with sectors/disks is out of scope, but adding the file-based I/O functions takes me pretty far.
At the moment I've got an annoying bug where the Aztec C-compiler doesn't quite work under my emulator and I'm trying to track it down. The C-compiler produces an assembly file which is 100% identical to that produced on my real hardware, but for some reason the assembler output from compiling that file is broken - I suspect I've got something wrong with my file-based I/O, but I've not yet resolved the problem.
TLDR; writing a CP/M emulator in golang, and getting more and more software running on it - https://github.com/skx/cpmulator
I could not derive a single piece of solid science in any of it.
It was remarkable how much content there was on this subject with little to no actual information - enjoy:
In case you want to have unlimited fun yourself, ask yourself: "What is the purpose of X?" and then "how can you measure/assess the fit-for-purpose of it?"
Possible Side-effects: #1 you might get disgusted and even angry with the self-declared "experts" who have not even understood the basic concepts.
#2 you might learn how little you understand yourself and how deep the rabbit hole goles.
Example in Software Development: Understand the quality dimensions for a "definition of ready" and what impact a good/used DoR has compared to a bad/not used DoR for the efficiency and effectiveness of a software development process.
Long story short, I was inspired by the Super Mario 64 and REDRIVER2 decompilation projects and wanted to do one. I picked a PlayStation video game from my childhood, started Ghidra and then I quickly realized that the game code's a complete mess. It's bad enough that I don't see myself ever finishing this project unless I can somehow divide-and-conquer this problem into manageable pieces. But you can't exactly break a program into pieces... can you?
So I've started to think for a bit and remembered the basic toolchain workflow: source files are compiled into assembly files, which are assembled into object files, which are all linked together into a program. The last bit stood out to me and I wondered: what if I could undo the work of the linker? I'd get a bunch of object files, dividing the original reverse-engineering problem into smaller pieces.
I searched online and found absolutely nothing on the topic. That should've tipped me off, but instead I started scribbling on a piece of paper. Object files are made up of sections (named arrays of bytes), symbols (named offsets within these sections) and relocations (spots to patch with a symbol's address). The linker lays out the sections, computes the addresses of the symbols and then patches the relocation spots to produce the program. I can't just take the program bytes and stuff them into object files because of these applied relocations, but if I could somehow undo them...
The good idea fairy struck, and the fairy struck hard.
I'm writing scripts in Jython and after a couple hundred lines I get results on sample test cases. I try them on the game and it takes forever due to algorithmic complexity. I rewrote a new implementation in Java, forking Ghidra in the process. I rewrote it a couple more times because my analyzer kept hitting edge cases. I built an elaborate and exhaustive test harness because I keep introducing hard to track down regressions. I submitted a couple of pull requests to Ghidra to solve some painful points and reduce the size of the diff, which spanned thousands of lines. I reply to the questions from the Ghidra team with walls of texts trying to explain my use-case, but the PRs get rejected because they don't fit the current design of Ghidra well.
When the Ghidra team rejected my stuff probably because at this point I was probably speaking in the native language of Cthulhu, I really should've taken the hint.
Instead, I spin off my fork as a Ghidra extension to alleviate the maintenance burden, which by now was getting closer to ten thousand lines. I keep rewriting my MIPS relocation analyzers again and again to improve their correctness, always hitting a new edge case. I've decided to start a blog, because I'm tired of trying to explain this stuff from basic principles to people since there's no literature on this topic. I get side-tracked writing a complete series of articles on the basics of reverse-engineering to introduce the topic. I get side-tracked again writing a series of articles on the applications of delinking related to software ports, with a case study on a x86 program that requires me to write relocation analyzers for this architecture and perform refactorings to support multiple ISAs and object file formats.
I'm finally back on reverse-engineering the video game that started all of this and get side-tracked once more because I'm documenting the process in another series of articles. By sheer luck I stumble upon a SYM debugging symbols file, but I don't have the matching executable for it, so I build a placeholder one that matches its shape, then import the placeholder into Ghidra, then write about a thousand lines of Java to import this data on top of the placeholder, then write a bunch of scripts and my own correlators to version track it onto a executable I do have because Ghidra doesn't know what to do with a source executable that doesn't have a single initialized byte to its name. I've tried to engage with the Ghidra community about this latest problem, but no answer. I assume they're probably busy trying to find an exorcist, so I carry on regardless.
Two years. Two years I've spent digging this rabbit hole that's probably worth a thesis or two. I know enough about delinking now that I could probably write a book that would read like a Lovecraftian horror story to people that develop linkers for a living. I've automated this stuff down to making a selection inside Ghidra and clicking on "File > Export Program...", but there's only so much you can do to make accessible or even understandable a technology that allows you to literally rip out code from a Linux program and shove into a Windows program or from a PlayStation game into a Linux program and have it work, in spite of ABIs or common sense.
TL;DR I've developed a reverse-engineering technique by accident that would give professors teaching Computer Sciences 101 an existential crisis.