Ask HN: Self hosted email, Cloud storage and dovecot | Hacker News Reader

Ask HN: Self hosted email, Cloud storage and dovecot

If I am self hosting my email for friends and family, but have a volume attached where dovecot stores the email, can I dynamically add another volume to the VM and tell dovecot to balance mail storage between 2 directories?

View on HN

That's best not done at the application (dovecot) level. LVM, md, or filesystems like btrfs or zfs can spread data across devices in myriad ways.

Kudos for doing this for friends and family.

Ok so I’ll just add 2 storage volumes, and use ZFS on Linux to combine them into 1 volume

It's so a reliability concern. If any volume dies you'll probably lose all the data.

Your nomenclature ("volume") sounds like you are using some kind of cloud system. Just expand your volume? It's unclear why you are asking for your ask.

There are volume size limits on my cloud provider, but it sounds like combining multiple ones together might not be a great idea.

It's totally a good idea.

There is a different question of whether that provides enough reliability, but that depends on if your cloud provider already offers safety in case of hardware failure for your volumes.

Note that even today, with a single volume, if you lose it, you lose all your email.

Most cloud providers would provide some resiliency for storage volumes though.

Would you prefer to use ZFS or LVM to handle something like the storage of mail for files which might need to add a volume from time to time?

I would personally use LVM because that's what I've been using for decades. If you don't know either, you might want to explore ZFS since it's a different concept a bit, but you don't have the baggage of "I already know this other thing" that might get in the way.

I'd certainly would like to start using ZFS at some point, but can't justify the investment.

Your question doesn't directly indicate which problem you are trying to solve. It would be helpful if you would mention the hosting provider as their service offerings do differ.

If you are trying to solve storage reliability for single mail server then creating a file system level failover would be way to go as mentioned previously (LVM, soft RAID over multiple volumes, etc).

If you are trying to solve mail service reliability/failover then adding second mail server would be helpful. In that case you can use dsync (does master/master sync) to keep emails on both servers in sync.

Nice I didn’t know about dsync. I imagine it works on large amounts of email, in case I want to host it for a nonprofit?

Not addressing your question, but FWIW:

Mail is the one thing I eventually caved on and offload to Microsoft. It's an uphill battle keeping your mail out of Spam folders. You setup DKIM, SPF, etc. according to the latest spec, and at the end of the day, the MS and Google still drop your mail in spam. They simply don't care.

Second, the forces and money keeping MS above legally bound 99.9% uptime tower in comparison to any one person's own resources. Is your daily driver even operational 99% of the time you use it? We hackers can't help but to tinker, so I suppose not.

MS has teams of engineers paid specifically to keep the thing running and NOT tinker. At only ~$10 / month that's a no brainer.

They should pay their engineers to fix their MTA-STS validation. Haven't done so in ages. If you think you're not up to selfhosting (which in my experience is quite feasible and comfortable): please choose a provider that is not one of the surveillance capitalists. Recommendations are Migadu and mailbox.org

Exactly the same here. I had email hosted with TSOHost here a few years ago and had no end of problems with bounce backs.

In the end I moved it all to GSuite for a few bucks a month. It's simply worth it to avoid the issues with sender rep.

I would do this with LVM. You can grow LVM to expand onto a second volume and expand the file system with it. Or ZFS.

https://wiki.archlinux.org/title/LVM#Add_physical_volume_to_...

However note self hosting mail used to be super easy but these days most report you have a real hard time getting people to accept your mail. When your IP has no prior history and small volume you’ll get false flagged as spam a lot. Likely to be your larger challenge.

What if I am doing this at a cheap VPS provider? Can I just add two of their storage volumes and somehow combine them into 1 bigger volume?

That makes it worse; cheap VPS providers tend to have lots of abuse issues, so their IP blocks tend to have poor reputation.

But besides that I can mount multiple volumes and use LVM to make it look like one?

Yes

Sadly self hosted e-mail is a 24/7 job where you will be fighting against big players for the luxury of sending e-mail from your own system. Despite the dozen or so different standards to attest you are a legitimate sender it has basically boiled down to a handful of big e-mail providers that trust each other and everyone else is eyed with utter suspicion and contempt - and it has nothing to do with the fact every member of the “in” crowd makes significant money selling access to their infrastructure - honest (sarcasm).

I self host my email spending a couple hours a month at most. But I have email related experience and if this is a new for you area you will need to learn it spending more time.

Risk of being blocked by big players (gmail/O365) is real but IMHO very low if you follow BCP. The main problem is that big companies make it very hard or impossible to contact them to resolve the issue.

> Risk of being blocked by big players (gmail/O365) is real but IMHO very low if you follow BCP

Can confirm, only issue I have is with outlook.com because their MTA-STS validator is… for lack of a better word: wrong.

I decided that replying to mails from outlook.com is not part of my mailserver's offerings and if it's required others channels have to be sought out. I'm not going to violate established standards just so Microsoft can do things their way.

Could you elaborate what BCP is? (I am not familiar with the term)

BCP is Best Current Practice, which for email includes at least having Forward-confirmed reverse DNS for mailserver hostname (with the same FQDN in EHLO hostname), SPF, DKIM, DMARC.

I'm sure most people here know about DMARC but there are still a lot of mail senders which either have no DMARC record, or send some messages in a way that either DKIM or SPF or even both fail, or have a syntax error in SPD/DKIM/DMARC DNS records.

On top of this it is good to ensure that your servers doesn't generate email backsatter when possible. Send rate limits is a simple but rarely enforced by small servers way to limit amount of spam which can be send if user credentials or web-app hosted on the same server are compromised.

One battleground tested self-hosted solution : Cloudron.io

I don't understand their service. It's $180/year or $30/month but you have still have to run your own server. What are you paying for that would be worth that amount of money?

I recommend something like Migadu instead of rolling it yourself.

Reply on news.ycombinator.com