Signal Desktop is corrupting its database
github.com
github.com
(In theory it should be possible to recover from this and I can still access my sqlcipher database manually but Electron and the stateful Signal protocol make it extremely difficult so I gave up. Multiple backups of the whole ~/.config/Signal directory didn't help either.)
(See: https://github.com/signalapp/Signal-Desktop/issues/4513#issu... )
Why Signal doesn't support this is beyond me. I can basically only use it for scenarios where I don't want any chat history, which just isn't that common.
Does my fucking head in. Yes Signal is about security but that doesn't mean it can't have a functional backup feature! Honestly it needs to be priority number one imho.
This bug shows just how important backups and importing conversations are. Whenever I setup Signal on a new system I hate when I see the message "For your security, conversation history isn't transferred to new linked devices."
Signal doesn't (or at least shouldn't) get to decide what happens here. If I am having to re-register because of a Signal failure my security is not effected by importing the conversation history as it was there two minutes ago before they corrupted the database. At the very least they should make it an option even if it is disabled by default.
Apologies for the ranty nature of this comment but I am fed up with this absolutely pathetic denial that backups are important because it is "just a messaging app". AHHH!
Edit: See this reply from when I complained about the lack of backup a month ago https://news.ycombinator.com/item?id=25687851
I understand it's frustrating when a (open source, forkable) application doesn't work the way you want it to, but you have to be able to see it from the other side as well.
Signal is signalling that chats are ephemeral and treating them as something else introduces security and privacy issues _in the way they are thinking about it_. It's very possible that's not an issue for you, but that doesn't make Signal wrong, is just make the two of you misaligned on why Signal exists. Signal is not for everyone and I think the team themselves make that pretty clear. Signal is for private and secure communications, and make no compromises that would lower either the privacy or the security, they rather compromise on other ends.
These kind of comments show the entitled parts of the internet, where open source software HAS to work a particular way otherwise ITS BROKEN AND HORRIBLE.
There are tradeoffs to everything. In this case, Signal made tradeoffs that you feel are wrong. You have a couple of choices, none of them include screaming that Signal should change their priorities based on your opionion: 1) continue using it anyways, start treating chats as ephemeral, 2) try to fork Signal and show us you can do it better or 3) dump Signal for a messenger that works the way you want it to.
In the end, there are multiple chat apps, use the right app for the right use case. Signal is definitely not a app for one-size-fits-them-all and it's pretty clear they are not trying to either.
If you have a choice between Whatsapp (metadata not private, but has backup) versus Signal (metadata private but no backup), if you need backup (or anyone you communicate with does), you lose privacy because you're forced to use Whatsapp. Arguably because of Signal's refusal to "compromise" on privacy by allowing backups you have lost privacy anyway.
Can one fork the Signal iOS app, add features then publish to the App Store while using Signals infrastructure?
If backups are a weak point/risk don't enable the backups then? How is denying it for all a good solution?
> Signal is not for everyone and I think the team themselves make that pretty clear.
If Signal don't want to be a WhatsApp competitor they need to stop acting like one and jumping on every opportunity to point out how they are not Facebook/are better than WhatsApp.
All I have seen from Signal on social media for the past month is how they are the perfect alternative to WhatsApp.
Signal has disappearing messages so if a sender doesn't want their messages captured in a backup they can easily get that. Of course that doesn't stop the recipient exporting that one message or even just taking a screenshot so it isn't perfect but better than nothing.
> These kind of comments show the entitled parts of the internet
Comments like this piss me off. It is not "entitled" to want to protect my data. I have been telling people to protect their data for the twenty years I have worked in IT. But now for chat conversations apparently backups are pointless and we shouldn't be doing them??? Come on. This flies in the face of literally two plus decades of the tech community pushing for "normal people" to do backups!
Why enable something that could introduce additional security risks when it's both cheaper and safer not to develop that feature in the first place?
> All I have seen from Signal on social media for the past month is how they are the perfect alternative to WhatsApp.
Huh, I've not gotten that vibe at all after speaking with some of the people at Signal. They know that their product is not for absolutely everyone and quick glance at their Twitter doesn't seem to paint the picture you're seeing either. You have any specific examples you could point me to?
> It is not "entitled" to want to protect my data
Of course not and that was also not my point... The entitlement comes from you declaring that Signal is wrong here while not actually understanding the other perspective that they are operating from (which is different from yours, obviously).
> But now for chat conversations apparently backups are pointless and we shouldn't be doing them???
Again, if you are subscribing to the same worldview as the team of Signal is subscribed to, then yes, you shouldn't backup your ephemeral chat conversations. If you're instead interested in persisting your conversations, Telegram/Whatsapp/whatever probably fits your use case better and you feel free to use those instead.
Either you use a chat application that makes absolutely zero compromises on security and privacy but might have worse UX, or you chose a chat application that does compromise on those things but have a better UI. Your entitlement is that you think you can change Signals whole mission because you actually should be using a different chat application.
4/5 of their last few tweets all seem to be pitching themselves as an alternative to WhatsApp / Facebook / targeted advertising? Realistically though their current position is you have to choose privacy or backups.
"We believe in targeted admirizing, not targeted advertising. Happy Valentine's Day from all of us at Signal."
"We've discovered that @tecnomobile devices (some of the most popular phones in Africa) enable notifications for @Facebook apps like @WhatsApp, but block Signal notifications by default. Privacy should be the default."
"Facebook, Instagram, and WhatsApp trying to collect your data when you're on Signal" (confused guy gif)
"If you see the shadow of looming advertisements in your current messaging app, make an early Spring to something better."
"January just ended, but if one of your New Year's Resolutions was to break up with Facebook there is still plenty of time."
https://i.imgur.com/bdUhqth.png
https://twitter.com/signalapp/status/1358903379515199488
Literally the first thing on their Twitter feed!
Sorry but if you are unable to see that Signal are clearly trying to position themselves as a WhatsApp alternative there is no point in continuing this conversation.
I can plug those into Syncthing and have them head off to my server or whatever. Just document the format so I can decrypt them with some tool to recover them later.
Kind of makes it worse in a way as it clearly shows they do care about backups just not doing it consistently.
IOS has had filesystem access for going on 2 years now, there's literally no excuse beyond laziness for not letting us backup and restore the signal messages. I don't even care if it gets dumped into an insecure format if that's the only way it works. I'm far more concerned with someone intercepting my messages in-flight than any other vector. If someone has a backdoor on the phone itself, one signal backup is the least of my worries. Ignoring the fact that backup would be optional.
Incorporate SMS (with a red background or something obvious that the chat is insecure), incorporate backup, and just fully embrace being a messaging app. If they do that they'll make the world universally more secure because it will increase adoption 100 fold. I'll have a lot easier time convincing my dad to use signal if I don't have to try to walk him through "well when you're texting grandma, use this app, and when you're texting me, use this app".
You can already use SMS from within the Signal app on mobile. On desktop it makes sense that there is no SMS feature, unless you want to use the desktop client to send SMS from your phone (like Android messaging), or you want Signal to operate an SMS gateway?
https://support.signal.org/hc/en-us/articles/360007321171-Ca...
Sure, iOS doesn't allow you to intercept SMS but others have already figured out how to work around this by assigning a virtual phone number.
I would even be satisfied if i can regularly backup messages to plaintext.
I like to keep my messages and be able to look up stuff later. If I don't want to keep the log i switch on self deleting messages, which is quite cool.
Here's issue #905 from 2015, a request for transferring messages when moving phones, closed the same day: https://github.com/signalapp/Signal-iOS/issues/905
Similarly, #967 https://github.com/signalapp/Signal-iOS/issues/967
And #2542 (2017, when I was first looking for transferring my Signal to a new phone) https://github.com/signalapp/Signal-iOS/issues/2542
That issue was locked and I was told to create a thread on their forum instead: https://whispersystems.discoursehosting.net/t/ios-backup-kee... - 3.5 years and 400+ messages later, backup functionality still doesn't exist for iOS, and the best solution for archiving your old messages requires jailbreaking and extracting encryption keys from your keychain: https://cight.co/backup-signal-ios-jailbreak/
NOTE: last year, Signal introduced functionality for transferring messages to a new device, but that won't help you from data corruption or losing your device since backups and export are still not supported: https://signal.org/blog/ios-device-transfer/
Not to dismiss the annoyance involved. I'm just trying to understand what this bug is really about.
While Signal's greatest strength is its privacy features I think that at some point they are going to have to meet their customers/prospects/users' other needs.
The biggest one being the availability of chat history anywhere, anytime at their own risk if needed.
We won't educate people to use messaging app in the way we want them to (for privacy sensible conversations only) because the vast majority don't use messaging app like that.
It's either give up on that idea or heavily advertise that Signal is not a Whatsapp/Telegram/Viber/Messenger/Whatever replacement: it's a tool to use when you want to have private conversations.
Maybe it'd be better to leave the Signal messaging app lives its life and allows a third party chat history viewer to emerge. You can already export your encrypted backup to a readable CSV file. https://github.com/xeals/signal-back edit: which obviously doesn't work with a corrupted database :p. Signal backup as a service startup ?
Point in case: mom complained the other day that Signal Desktop took some time to launch because it was “syncing things”. I told here that this syncing from her phone to the computer is the proof her messages only exist in the application. Desktop or smartphone, that's why it needs to sync, facebook and others don't work like that (I grossly simplified).
note: Thread with comments like is how I remember HN is now a site with a huge audience.
What I need is just to persist the groups, really. So I easily can continue chatting there and not hope someone else sends a message first.
When you send someone a message you lose control of it, regardless of how it "feels".
Their design causes it to be incredibly slow, to miss messages, to have them arrive out of order - and that's not even acknowledging the usability downfalls.
They really need to do better. I hope they can figure out some way to shift more resources to it, a good desktop client is essential to modern messengers.
Electron is decidedly not why Signal Desktop is terrible, it's just a contributing factor.
That defeats purpose of Electron, though.
> To figure out why ethics, moral obligations, and skills cannot be easily separable in real life, consider the following. When you tell someone in a position of responsibility, say your bookkeeper, "I trust you," do you mean that 1) you trust his ethics (he will not divert money to Panama), 2) you trust his accounting precision, or 3) both? The entire point of the book is that in the real world it is hard to disentangle ethics on one hand from knowledge and competence on the other.
So far moxie appears to be ethical, but his and his team's work has been rather sloppy. Meanwhile, many are still questioning the integrity of telegram, but durov's team is definitely the most competent one.
It had a few problems but it mostly did the job. But lately, they disabled it with the latest onboarding update, without mentioning anything in the change log. I had to look at the git history to see that, I also found a "won't fix" ticket about the problem.
I think they are ignoring an very important feature but anyways, make a decision. Either you officially disable the feature, write it in the changelog and remove the dead code. Or you keep it and hopefully fix it. The way they are doing it is sloppy.
I didn't look too much into it but dead code is definitely a code smell and your experience seems to match mine. It is not terrible, but a bit underwhelming considered it is a highly regarded, security sensitive app.
Do you have any clue how to make things better?
Recently I have moved to Signal because, after the WhatsApp opportunity, I had to move my non-tech savvy family members to something better, and I suspect Telegram isn't it (I can't understand how it is funded, it is too "magic").
But it is rough, specially compared to Telegram.
For example: there's no way I'll share my phone number to chat with strangers, whilst on Telegram I have an anonymous username I can use.
But even forgetting about that, it is the small things, like it can't record and send a video (you can record it out of the app, and then send it from Signal; at least on Android), or the atrocious desktop app.
I'm happy it exists and I'll stick with it because they're supposed to be "the good guys", but I'm hopping it improves before I have to admit it was a mistake and I should have trusted Telegram.
Do you have any idea how we could practically make Signal a better app in the future?
The video call issue is that if I rotate my phone from portrait to landscape, the recipient will still receive a portrait video stream from me. I'd expect the recipient to receive a landscape aspect ratio stream instead.
I must add that I am incredibly thankful for the effort you guys have put into Signal. If users like me come across as a bit sour, at least in my case, I am that way because there seem to be a lot of low hanging fruit that users willing to use the app seem to hit on the head. I'd love for signal to be more successful.
Not sure how reliable and resilient SQLCipher is but that might (significantly?) increase the risk for a bug/corruption to occur. And the encryption certainly makes the analysis more difficult (while, at least on GNU/Linux, I don't see any advantage as the encryption key is stored unencrypted in ~/.config/Signal/config.json - not sure if other Desktop platforms support secure keystores like on Android and iOS). I briefly tried to analyze my corrupted DB but quickly gave up as I'm not familiar with SQLCipher and basically only got a generic "Error: file is not a database" error message when trying to decrypt it (and there's no plaintext header IIRC so it looks just like random data).
I also had multiple backups of the SQLCipher DB that I could successfully access manually but I was unable to use them for Signal-Desktop (not sure if this was due to some other Electron DBs/state, the stateful Signal protocol, or something else - IIRC the only hint was the "Database startup error: Error: SQLITE_NOTADB: file is not a database" message that didn't really help much).
SQLCipher has open-source communitiy edition.
As popular as it is, it's still underrated.
But the thread says SQLCipher is used, so not sure.
They have plenty of time and money to fix this issue quickly before it gets out of hand.
They seem to strictly follow their own agenda. If they don't think something is important, it won't be taken care of.
Is there a citation for this claim? I tried searching on Twitter for tweets regarding Signal errors, and I didn't see anything except for retweets of the link to this HN post.
It's also just the local database, so the phone that their desktop app is linked to will still have their chat history, thus these alleged journalists would not have lost their sources.
This is very serious for everyone depending on signal, that I could just lose all my secure chat history without warning!
[1]: https://support.signal.org/hc/en-us/articles/360007059752-Ba...
Do you think developers willingly write bugs into their applications, then simply release them hoping they can mess up somebody's day for the fun of it?
"Allowed to roll out into production" as if there's a manager looking at the list of newly created bugs, grinning and going "This bug... I like this bug! Roll it out!"
Was there any tests for this sort of thing? surely if you are storing secure chat history to a database this should be tested to death.
Had they tested more of this functionality this serious bug would have been caught, and now that I recommended this to people, I pretty much now regret doing so for secure messaging.
It’s a (totally fair) attack on Signal’s lack of QA/testing in their development + release cycle.
Do they implement peer reviews on PR? Multiple reviews on changes touching mission critical code (ie. data migrations)? Do their test suites provide adequate test coverage? Do they have a manual Q/A process that involves real people testing new releases?
Considering Signal’s funding, I would hope the answer to all of those questions is yes.
But if it’s possible to release code that completely corrupts the app with no known fix, I suspect their test coverage and Q/A processes aren’t as robust as they need to be.
Microsoft got rightfully roasted for last years data loss release. This is arguably the most serious class of bug, and the most preventable.
This right here was the final serious nail in the coffin that your chat history is corrupted due to this bug in production.
$60M in funding and they still can't fix these issues or handle these many users. I liked the Signal name and its friendliness to the end user, but I think the true hard-hitting reality is, it is just not ready yet for serious use. What a shame.
EDIT - if you downvote, please explain why. If you disagree it's not a reason to downvote.
Concerning your edit: I thought it was a pretty established part of the etiquette on HN that a downvote is used also for simple disagreement. It means "I do not agree", not "you are wrong and should be shamed".
When substantive comments get unfairly downvoted, it's good to give them a corrective upvote. People mostly tend to do that and that mostly fixes the problem. Not entirely—but close enough that there's no globally better solution that we're aware of.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
https://news.ycombinator.com/newsguidelines.html
Near the bottom
That's not set in stone; reportedly pg said downvoting for disagreement was fine, but I don't have the quote handy nor a source.
Signal did as well as anybody. They bounced back after about 48 hours, and usability seems as good as FB Messenger. Not nearly the roller coaster of Parler.
Because something has potential caveats, problematic corner cases, or philosophical nits, open source projects will often end up sticking with a worse solution out of pure bullheadedness.
Iterative development is a thing for a reason.
Also, in regards to your edit:
> EDIT - if you downvote, please explain why. If you disagree it's not a reason to downvote.
https://news.ycombinator.com/newsguidelines.html grep "voting"
also: "Please don't comment about the voting on comments. It never does any good, and it makes boring reading."
I usually took a downvote with no comment as an information dead end. There are many reasons one would downvote, maybe factual error, in which case the correct answer is greatly appreciated.
On the HN guidelines page the word "down" is not mentioned. Just curious, do you all feel that up and downvotes are not different beasts? Or is that page just for users who cannot downvote?
Those are pretty terrifying bugs, the fail-open can be disastrous in the wrong situation, and the others easily result in catastrophic data loss. Because that's what encryption is supposed to do in some situations.
Both Element/Matrix and Signal don't come close to WhatsApp, sadly.
That is the unfortunate reality-hitting hard truth right there. Element and Signal are just not ready yet for serious widespread general use to compete with WhatsApp.
Signal is still immature and lacks tons of functionality compared to WhatsApp. Element suffers from usability and onboarding issues which frustrate the user. The fact we have to keep mentioning the protocol 'Matrix' next to the client name 'Element' creates further confusion to the user; leaving them to ignore it altogether. That's before they get confused and lost in the settings page.
Telegram on the other hand has a better chance to compete against WhatsApp.
It's important to realize that Signal is aiming to be iMessage replacement Whatsapp/Telegram/Discord are more like "chatservers" with rooms. Signal right now focuses on simple 1to1 messaging with sending pictures/media etc. That's what they do really well.
Btw iMessage also have super basic desktop client. It works even worse than Signal desktop (random logouts and messages sent as different account with only "email"). It might not be so easy going from 1v1 fully encrypted chat to manytomany fully encrypted chatroom megaserver. Apple seems to also struggle with it.
At what exactly? Elaborate and enlighten us a bit.
> and doesn't require phone numbers?
That's true, but you know it's still not enough for John and Jane Doe to use it. Elaborate us on more reasons otherwise John and Jane Doe will use 'this Signal crap' instead or even will go back to WhatsApp. (Again)
And then I am looking at telegram - everything is polished, many smart and useful functionality is included. The design is outstanding, the updates seem to add useful stuff.
Can someone explain to me why Signal is so very bad compared to telegram?
Other apps may provide higher anonymity, but none provide privacy guarantees higher than (or even close to) Signal.
So, being "best" at privacy is meaningless unless you are in the very lucky position that your local jurisdiction fits Signal's threat model. There aren't many in the west even...
Signal has taken measures to limit its ability to know who I'm sending and receiving messages from. Though it's still possible by mapping my IP address to me. So yes there is still a centralized metadata problem, but I'm not letting perfect stand in the way of better.
Not in most cases, for most people, most of the time. Everyone knows you communicate with your family, friends and business associates. Very few people communicate out of those groups and even if they do they rarely have to worry about anyone finding out about it.
- Using phone numbers means that Signal can constantly check against your contacts to let you know if any of your contacts are now using Signal. As such, using phone numbers encourages use of Signal.
- The usage of phone numbers instead of email or usernames also helps combat temporary/fake account creation abuse, as phone numbers are KYC if you ban VoIP numbers
That said, in the end it is a compromise on privacy and anonymity to increase the popularity and stability of the app.
I'm more concerned about every day privacy, and Telegram is far better on them. Do you realized that you sent by mistake some data to the wrong people? In Telegram you can delete or edit every messages you want whenever you want.
Don't you want to share your phone number in a group? In Telegram your phone number is hidden by default, meaning that other group members doesn't see you number unless you share it with them.
Also Signal is open source but not really open. In Telegram you can use whatever client you want, in Signal you are forbidden to use anything else than the official client. And the official client is open source, but it depends on proprietary services, like the Google Play services on Android, so you really can't use it on a 100% open source system. You can't even find online the Signal apk by their choice, so the only official way is to install it from the Google Play Store (and how do you verify that the apk that Google provided to you doesn't contain a backdoor?)
For a lot of people, that's exactly the concern.
> Also Signal is open source but not really open. In Telegram you can use whatever client you want, in Signal you are forbidden to use anything else than the official client.
There's nothing in open source that requires people running services to allow anyone to connect to it however they like. You still have the freedom to inspect, modify, run and distribute both the server and client, so it's to see what part of that in "not really open".
> And the official client is open source, but it depends on proprietary services, like the Google Play services on Android, so you really can't use it on a 100% open source system.
I agree, this is annoying. But it depends on your threat model and it seems that most people don't see Google as the threat to protect against.
> You can't even find online the Signal apk by their choice, so the only official way is to install it from the Google Play Store
https://signal.org/android/apk/
> (and how do you verify that the apk that Google provided to you doesn't contain a backdoor?)
>In Telegram you can delete or edit every messages you want whenever you want...
Remember the furore when Amazon were caught deleting books off people's Kindles? Everyone was rightly outraged by that. Yet so many people think it's a great idea that Telegram allows other people to arbitrarily delete content from your devices.
That's technically not true - it's not end-to-end-encrypted by default.
But yes, it is a design choice - telegram puts usability first - see all their innovations with large groups, stickers etc -, even at the expense of some privacy (i.e. no e2e-chats). But that's why you can have Telegram open on every device you own and probably why they are the second largest chat app by now.
Signal, on the other hand, values privacy first, everything else second. This is why this bug is open for 6 months - they see your chat log as a convenience feature and mostly a burden. In addition, Signal is a smaller team with less funding, so that their UI is not as polished is partially also due to lacking manpower, but its simply not a priority for them.
The Signal desktop app is Electron, where as Telegram is native. That's why and it's that simple.
There are many really good Electron applications, Visual Studio Code being one of them.
Most importantly, it's bad at protecting my privacy. You can't show any message to anyone on your screen or have someone look over your shoulder unless you want them to see who you recently communicated with in what order as well as the start of the most recent messages.
Thanks for mentioning that. I always wondered why there's no option to at least hide the most recent messages and dates (or temporarily hide the whole sidebar). I guess most people are either fine with it or use their phone instead (and using Signal-Desktop in public / when someone's looking is probably uncommon). It seems like at least Telegram-Desktop has this issue as well (not sure if there's an option for it).
Anyway, a slight modification to the UI to hide/minimize the sidebar would be something I'd appreciate for those rare situations.
Native: https://macos.telegram.org/
Qt client: https://desktop.telegram.org/
In addition, there’s a libpurple plugin should one want to use Telegram with Pidgin, as well as several other clients including one that’s TUI based.
The choice of client is a massive boon. It’s unfortunate that one can’t use a alternative Signal clients, because they’d likely solve many of the gripes people have with Signal.
Updates have been meaningful and minimal.
Can't say I'd be happy if it started adding a bunch of features I didn't ask for.
Opinions may differ here. Their sticker packs feature added at least 50 MB (mostly via node_modules): https://github.com/signalapp/Signal-Desktop/issues/3919
> Can't say I'd be happy if it started adding a bunch of features I didn't ask for.
What features do you mean here? The GitHub issue is a bug report and not a feature request. However, there are two features that could help: Backups (export and import functionality) and syncing older messages from the phone during the initial setup. Both of these features can be fully optional and shouldn't require much code.
And they do regularly add new features (that I didn't ask for but I get why) anyway: https://github.com/signalapp/Signal-Desktop/releases
- macOS client: https://github.com/overtake/TelegramSwift/graphs/contributor.... Written almost single handedly by one developer.
- Telegram Desktop: https://github.com/telegramdesktop/tdesktop/graphs/contribut.... Written almost single handedly by one developer, who also wrote almost all of the supporting Qt code.
This isn't worthy of the front page of HN..