Show HN: imessage-exporter, a CLI app and library | Hacker News Reader

Show HN: imessage-exporter, a CLI app and library

github.com

161 pointscss43 comments

This is quite nice.

iMessage doesn't keep your whole message history on-device. You can scroll back and it will fetch a bit more each time, but that's it.

Is there anyway to pull back to the beginning of time into the local database? I'd like to retrieve some specific old messages from a friend who has passed away. If I could get them out I could search (I know what I am looking for) but it would involve an unfeasible amount of "pull to fetch more"

Wouldn't the tool we are commenting on solve your problem nicely or what are you missing? Have you checked out the sqlite database where they are all stored? Does the iMessage search not work for this use case?

https://arctype.com/blog/search-imessage/

I've been tinkering with this recently and there are conversations where the last content in the sqlite database is a month and a half old. Oddly in some cases there used to be newer content in there that had since disappeared.

If you are talking about data missing from the `text` column, for some reason it disappears after you read a message. The content is stored in a binary blob in a different column, which I parse like this: https://github.com/ReagentX/imessage-exporter/blob/c73bc4d66...

oooooh that's exactly what it is! thank you!

> iMessage doesn't keep your whole message history on-device

Unless you’ve changed the “message history” setting, yes, it does keep all history on-device. It’s just incredibly frustrating to access any of it because Messages.app is awful at history.

> iMessage doesn't keep your whole message history on-device. You can scroll back and it will fetch a bit more each time, but that's it.

This is confidently incorrect.

The entire database is stored on device in a SQLite database. It absolutely has a terrible UI for fetching messages, but they’re all there.

I previously kept copies of this SQLite database on hand when backing up my phone (it stores to a specific folder every time), but since encrypted backups were introduced a few iOS versions ago, I found it too much of a hassle.

If OP wants to find this file, a good search term is "3d0d7e5fb2ce288813306e4d4636395e047a3d28" which apparently is a sum of the string "HomeDomain-Library/SMS/sms.db".

One thing I miss from Android was the simplicity of getting data in and out of the phone. I could just dump everything to CSV and be done with it.

They might mean attachments too, which I count as whole message history.

Messages seems to be somewhat random about keeping/removing attachments from your devices regardless of how much storage you have free or whether you have 'Optimise Storage' off.

*note this is specifically when using Messages with iCloud

If you don't sync imessage to iCloud, I thought it quite literally only stored on device in that case, besides when it in transit to be delivered. Referencing this official apple article, messages that are waiting to be delivered to devices can sit in the message queue for up to 30 days. https://support.apple.com/guide/security/how-imessage-sends-...

The image on the readme that talks about what the product does is a really nice way to do that - and in fact it's better than talking about what the product does because it's an example of an actual export - thanks for making this and for having a great readme

Thanks! I figured it was a concise way to show what the output looks like while also explaining some of the features.

I've been using TouchCopy for years, it supports encrypted backups as well. Not as nice as an open source tool, but it does work and it seems to get all the messages, attachments, everything. It does cost a little money, but it's been the only reliable thing that continues to work.

For anyone wondering, you can search your iMessages on iPhone by going to the message app, pull down and type what you want in the search bar on top.

Oddly this doesn't always give results for me, for things I KNOW are in my messages. All my iMessage history is on-device (I don't use and will never use iCloud backup). Pretty disappointing :\

Yeah I don’t use iCloud either, but that method always worked for me, where it does fail for you?! What’s the scenario where it doesn’t return what you were looking for?

Oh it's just like some arbitrary limitation, like the search function only goes so far back, or something. No clue, I haven't narrowed down the exact factors that determine whether my search will get a result or not

Supposedly iMessage search is getting way better in iOS 17. Fingers crossed.

This doesn't return all messages unfortunately, and the way the algorithm works, it will return a few results at a time, forcing you to re-pull for more results. I wanted all the messages I had received from my dad, and it was impossible to do this way.

It's downright criminal that apple doesn't provide tools to backup and archive imessages. I have like 40gb of messages on my phone that I do not want to delete, but I don't need them on my phone.

Enable automatic deletion after 30 days and free yourself from the mental shackles of historical message hoarding.

>that I do not want to delete

They explicitly said what they don't want to do it.

many friends have lost a good chunk of their life by losing their phones.

one friend lost many of his baby pictures

another lost all text messages from his (deceased) parent

many family members just break a phone and move on.

it is heart-crushingly bad that apple doesn't allow easy access and export from their proprietary message database.

and no, I don't mean running sqlite3

I wonder why people keep/rely on a messaging app as a source/storage/backup (?) for things like pictures. Isn’t that kinda inadequate…?

Most people don’t think about data backup. At all.

I know I should just look this up myself to confirm, but are these not backed up to icloud? It's not that expensive, and if you don't want to pay the apple tax for services you don't buy apple hardware...

People like to moan about "the cloud" or the "apple tax", then moan about how they lost all their irreplaceable images when they lost their phone.

Most of the time the complaint is that apple forces you to use icloud and back up everything.

There's tools like Wondershare transfer. It's not free but does a surprisingly good job of converting between android and iOS. Somehow they figured out the secret sauce of Apple's backups and charge only $40 or so for the privilege (no affiliation btw). For me, it was worth it

Agreed, but also this is a terrible spot to keep your only copy of something important. Reminds me of XKCD 1172[1].

1. https://xkcd.com/1172/

Oh this looks great! I’ve crudely been querying the iMessage’s chat.db to export convos to txt files so this is a much more elegant option

Do you have any pointers on how that is done? I assume/hope it is just a sqlite database so the only trick is yanking the file from the phone?

Yea I saw iPhone usage was mentioned in the docs and I have no idea how that’s done. I’ve only tried through my laptop with iMessage sync by issuing

$ sqlite3 ~/Library/Messages/chat.db

and then performing some sql commands

The database file is the same across both platforms, you just need to choose the right path: https://github.com/ReagentX/imessage-exporter/blob/c73bc4d66...

You can get this file from an unencrypted iOS backup: https://github.com/ReagentX/imessage-exporter/blob/c73bc4d66...

How do I make a backup without the Cloud on Linux?

Instructions I found from Apple (https://support.apple.com/en-us/HT203977) are to use the Cloud, a Mac, or Windows +iTunes.

Maybe something like https://github.com/libimobiledevice/libimobiledevice? I'm not sure, I haven't tried to do this.

libimobiledevice definitely works for this, I've done it before.

iMessage... why they refuse to permit users to file, tag, archive, print & share messages is a mystery to me. More and more of my clients want to communicate via text message.

Dealing with iMessage threads during Discovery was a nightmare. I wish I knew about this tool when I needed it

Hey can this merge two iMessage databases? I have one from an old machine and TimeMachine biffed something, so now I have two copies I'd like to combine.

I've used this several times - it's really very good.

Nice, but it needs continuous integration!

Something that runs in docker container on my home server. If my phone gets stolen, I have 1 hour old backup....

But iMessages are already stored in iCloud and will be available on your computer or any new phone when you log into iCloud again if enabled. Wouldn't that already solve that use case?

Reply on news.ycombinator.com