Hacker News Reader

Comment by css | Hacker News Reader

The California Poppy release [0] of `imessage-exporter` is the biggest update yet. Most notably, it implements a novel `typedstream` deserializer [1], allowing it to understand the data stored in `NSAttributedString` binary data that is stored in the iMessage database.

I've spent a long time reverse engineering nearly every aspect of Apple's iMessage SQLite tables to build this program. As far as I know, there are no other tools that support the full corpus of iMessage features [2], including edited messages, app messages, reactions, text range formatting, and threads.

[0]: https://github.com/ReagentX/imessage-exporter/releases/tag/2...

[1]: https://docs.rs/imessage-database/latest/imessage_database/u...

[2]: https://docs.rs/imessage-database/2.0.1/imessage_database/me...

View on HN

FWIW, I have built something like this--including safely reimplementing the deserialization of that old "streamtyped" format, though not necessarily dealing with "the full corpus of iMessage features", as I thankfully don't experience people using newer formatting ;P)--and, to me, the hardest part wasn't/isn't dealing with a single database, or adding file format features like that deserialization: it is that the user almost certainly has a handful of these database files collected from a hodgepodge of devices and backups that are all slightly out of sync (as always happens for various reasons, both reasonable and unreasonable) and were last touched by different versions of macOS/iOS, and you have to merge all of their data into a single unified export. I found this particularly hard when going back and dealing with ancient database files from early versions of iPhoneOS, before "iMessage" even existed, as well as weird corner cases where old database files were on devices which happened to be upgraded to newer versions of iOS, as the migrations are sometimes a bit more awkward than a de novo message. Like, to me, "table stakes" for this kind of program is that it takes a list of database files, not a single one, or it is simply never going to work well. (I would be happy to help test such with my database files--though probably a tad slowly right now as I've been overly-booked recently--as well as, if it would somehow be at all helpful, to show you the somewhat disorganized code I developed, particularly so if you would be willing to support an SQLite or JSON output format... it would be nice to never have to maintain my version of this stuff again, as I only wrote what I have so far for my own personal use cases, and mostly in a frantic panic over a few days in 2021 when dealing with discovery for a lawsuit against Apple, not to distribute to random users, though I otherwise might some day if this use case is never fully served.)

Reply on news.ycombinator.com