Show HN: A fake SMTP server for software integration testing | Hacker News Reader

Show HN: A fake SMTP server for software integration testing

fakemail.stream

127 pointsaeaa349 comments

This is a side project of mine.

Use this as your SMTP server in a test environment to guarantee that your users don't receive test emails.

Looking for feedback, especially on the security side.

Open articleView on HN

I have used tools like these for nearly a decade and find them invaluable. My current favourite is Mailpit [1].

The “OG” I would consider Mailcatcher [2] from my Rails days

[1] https://github.com/axllent/mailpit

[2] https://mailcatcher.me/

Add https://mailsnag.com to the list. You can share email samples with others for the review and it also allows receiving emails for your prod env - send emails to your app and process them in it.

Don't they all "allow receiving emails from your prod env"? They all speak SMTP. What am I missing?

The idea is that, in a a test environment, we should not actually deliver the mails to the recipient, instead making them available on a web page (or through an API).

Add https://msgdrop.io to that list :)

An API driven approach like Msgdrop is more flexible, as many apps can’t configure a fake SMTP server, especially if you use tools like Mailgun or SendGrid.

The "OG" fake SMTP as a service would be Mailtrap [1].

[1] https://mailtrap.io.

I prefer self-hosting MailHog:

https://github.com/mailhog/MailHog

there's a few of them out there, I use Papercut https://github.com/ChangemakerStudios/Papercut-SMTP

I use papercut too.

From the Mailpit README:

> Mailpit was originally inspired by MailHog which is no longer maintained and hasn't seen active development or security updates for a few years now.

Curious as to why it would need updates.

Security updates sure, but what really needs to update on an STMP (+ TLS?) solution?

Third-party libraries? TLS has definitely had its fair share of implementation and protocol vulnerabilities.

Although in mailhog's case it looks like STARTTLS was never implemented. Hence the forks.

It's a web application too, not just a SMTP server. The MailHog web app is a little rough around the edges (it's an early AngularJS app) and had some bugs involving HTML mail content, so I can see the need for an updated version.

A few others (last I looked, mailhog was the best option IMHO):

- https://github.com/maildev/maildev

- https://github.com/Nilhcem/FakeSMTP

- https://gitlab.com/markbeeson/maildrop

- https://github.com/sj26/mailcatcher

I do the same with mailcatcher but by self-host I mean I would throw it in the projects docker compose file for local development and spin it up specifically for that project and take it down when not needed.

I use https://toolheap.com/test-mail-server-tool/

since folks are sharing tools they use to test, I have one to validate if your email sender can send internationalized emails

pünicode.com. Any local part will create a temp cache that you can check to see if your email system can deliver to international local and/or domain parts. EG, send an email to josé@pünicode.com and see it show up at pünicode.com/emails/josé.

Wow...I didn't realize there were so many of these. People have mentioned almost a dozen in the comments so far.

I too have one, but it is very barebones. No GUI, no API. I suspect it would fail in many cases that the others handle, but it is fine for my test environment. That environment is basically a bunch of services from work that in production run on separate servers all shoved into one test VM with a firewall that blocks most outgoing connections to keep things from escaping.

The firewall reroutes any attempted outgoing port 25 connections to localhost port 2000, which my fake SMTP server listens on. When something connections it creates a timestamped file, sends them a "220 hello" message, and then loops reading what they send. Everything they send is copied to the file.

If they send a "quit" command it sends back "221 bye" and disconnects and closed the output file.

If they send a "data" command it sends back "354 send the message" and then loops until they send a "." line. When they send that it sends back "250 OK".

If they send anything else it just says "250 OK".

That ridiculously small subset of SMTP turns out to be fine in my environment.

Here it is in case anyone might actually find it useful [1]. Building and running is simple. It's a single Java file, SmtpSink.java. Put that somewhere, "mkdir msgs" there, "javac SmtpSink.java", and then "java SmtpSink". The data for each connection will be in the msgs directory.

[1] https://pastebin.com/dqgGZB82

If "my users" might receive test emails, that means I'm using real user email addresses in my test data, therfore my test SMTP server should not be a random system on the internet but something I fully control.

I think that’s the point. You can self-host it. This is a demo but there’s a link to the source at the bottom of that page.

A little more information:

So, it's called fakemail but there is a real SMTP server in there. Attachments should work fine. Getting the web app to create SMTP accounts was quite tricky, I'm sure there are better ways but I ended up implementing the unix crypt() algorithm in C#.

Server is holding up fine so far (there was a rate-limiting bug which brought the site down yesterday). Logs show 37K unique IPs have accessed it since yesterday, and it seems to be using about 1% of the CPU (it's on a free VM in the Oracle Cloud).

There is a whole API sitting behind the web page, including proper authentication, but the frontend is very much a MVP.

Very few actual emails have been sent to it, so I'd love it if people could actually send stuff. There are a bunch of websites that can be used to send test mails, e.g.

https://www.gmass.co/smtp-test

https://www.smtper.net/

https://smtpserver.com/smtptest

https://dnschecker.org/smtp-test-tool.php

Also built something very similar - https://ephemeralpostal.com

Comes with a small API so your integration tests can actually check the contents of emails that were expected to be sent out. Did a Show HN a while ago with more details - https://news.ycombinator.com/item?id=40590670

Wow, this and all the various other options mentioned look very useful. Like others, I have hacked together configurations to do this in the past and would have loved to have these instead.

I had no idea on the availability of all the various options, which leads to the question: Where would one find examples of similar network test servers for other protocols/functions?

For example:

- A SAML IdP - Define accounts and complete a login, allow debug of request/response.

- A DNS Server - Define local domains and records, control whether Internet domains are resolvable or just local ‘corporate’ records.

- Syslog Server - Catch logs and make them temporarily available. (usually syslogd works for this, but maybe test harnesses have advantages).

- SNMP - Trap destination to capture/show alerts

I have a rule that any time someone ships a server, they're required to also ship a version that supports client testing purposes.

Of course nobody has ever observed my rule.

Doesn't make it a bad rule...

In what language though?

I wouldn't say it's a 'bad' rule - just ineffective.

Since it seems to be overwhelmed currently, I'd suggest letting users host their own version of this in their own networks.

I recently came across MailSlurper [1], which I think serves a similar use case.

[1] https://github.com/mailslurper/mailslurper

This is all open source by the way, and can be self-hosted. All the code is at https://github.com/aled/fakemail.

Really cool. I couldn't find it documented anywhere, but if you save the unique (guid) url generated, how long is that available for? "Forever"?

Yes, currently these are never deleted.

I will probably update it to delete accounts that have been inactive for a year or something.

On the real one, you can set it to -'save emails to filesystem instead of sending out', so a fake SMTP is not really required.

I use dockerized mailcatcher for this. Used it for years with ruby on rails and now use it for everything.

hugged to death?

anyway - i got a great 429 back with a suggested try-after -time. it made me smile in appreciation!

well done!

That's the wrong status code

The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time ("rate limiting").

Unless the server believes you as an individual user were sending too many it should not have been a 429 If the server was unable to handle the volume of requests more generally it should have been a 503 which also supports Retry-After

The 429 response is sent when rate-limiting (based on IP address).

Either something is wrong with my rate-limiting code or there are many people behind a single IP address.

Anyway the limits are increased now.

Possible your application isn't seeing real ips, maybe is seeing the IP of a load balancer or similar.

Maybe one request is one Too Many at this point. ;)

Call it a very dynamic rate limit that autoscales all the way down to zero.

IME, it is relatively common to get this response code incorrectly, i.e., not triggered by rate. IME, it can be triggered by sending one and only one HTTP request to a site one has never visited before. For example, a request sent to a certain IP address with a certain set of HTTP headers and header values using a certain HTTP method and HTTP version.

Despite what "Too many requests" suggests, people writing/configuring software are (accidentally) sending this error in response to requests based on quality not quantity.

In each case where I have received it, I was able to make the error go away by changing something about the request.

I don't want to burst bubbles, but postfix with few lines of config can redirect all email to DEVs.

You really want, whenever possible, to test everything using real tools. Doubly so, using tools you'll use it PROD. However, at least postfix is de-facto standard.

apt-get install postfix, postfix-pcre bsd-mailx, config and done.

Here's an example redirecting ALL outgoing emails, UNLESS they are to your OK domains. Redirects are sent to an alias in /etc/aliases, which you can point to anything. (Easier for DEVs to modify when required)...

  /etc/postfix/header_checks (adds a header with original TO) 

  /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ PREPEND DEV-ENV-REDIRECT: ${1}@${2}

This MUST be the same as the above regex, so that the TO preservation + redirect are both done in tandem..

  /etc/postfix/recipient_canonical_map
  /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ externalredirect@localhost

Then in main.cf

  # added
  recipient_canonical_classes = envelope_recipient
  recipient_canonical_maps = pcre:/etc/postfix/recipient_canonical_map
  local_header_rewrite_clients = permit_mynetworks
  #
  header_checks = pcre:/etc/postfix/header_checks

Then in /etc/aliases:

  #
  externalredirect: dev

  # dev's email.. (default unless set)
  dev: /dev/null

Preserving the original TO as another header ensures you can debug if required, whilst preserving 100% the original mail body.

Using a second redirect in aliases, allows you do something such as:

/etc/aliases:

  #
  externalredirect: dev, another_email_address_for_logs

  # dev's email.. (default unless set)
  dev: /dev/null

So it's super easy for a dev to just change:

  dev: /dev/null

  dev: dev@corp.com

Without the need to worry about overall redirect stuff.

NOTE that if you don't have a unique email for your company's DEVs to use, this won't work, HOWEVER... you can redirect with more refined controls above.

That is, instead of saying "if it's not to a company domain, then redirect this DEV TEST email!", you can "If not to this specific email address, then redirect to this specific email address".

The reason I have this setup to redirect of not corp domain, is that the env I have this deployed in is byte per byte 100% identical to PROD deployment, with only very, very, very, minor tweaks. About 20 bytes or so.

That way, all tests done in DEV are 100% identical to configs in PROD. You eliminate PROD deploy bugs more aptly this way. And so if local MTAs are postfix in PROD, then you can keep all of your PROD postfix configs, with these minor changes to lock down DEV. And, you can keep the all the config files, all the config, and just have empty header_checks, recipient_canonical_map files.

But this means that alert emails that might get send from PROD have genericized domains, so in such envs it's easier to NOT redirect corp dest emails carte blanche, and then send everything else to a redirect dest.

That way monitoring / emerg emails get through unvarnished.

It's always fascinating to me when people say "not to burst your bubble but it's really simple to use this other tool which can already do the task", and then write an essay about all the workarounds they had to apply to make the other tool work

Reminds me of https://news.ycombinator.com/item?id=9224

The thing is, it's not "another tool", but instead "a tool deployed in at least hundreds of millions of *nix servers, which is rigidly compliant with SMTP RFCs, etc, etc".

If you want to debug issues with sending mail from code, then ensuring you have something that will scream if you "do it wrong" is a big bonus. And having something tested for 20+ years with rigid adherence to RFCs, deployed more widely than any other *nix MTA, is probably a good thing too.

In terms of "essay", I know it looks big on your phone screen (I presume), but a few paragraphs of background info is hardly that.

Adding 8 lines to configuration files is too complicated?

And it only covers one scenario.

Most people these days are using a service provider for SMTP in production, so it isn't really possible to keep your prod and test configs in sync.

I do agree about using testing everything real tools, which is why this (fakemail) uses OpenSMTPD as the mail server.

All the work is in configuring it (similar complexity to your postfix configuration by the looks of it), interfacing to it, and deploying it (currently using ansible but will probably dockerize it).

The fact that no emails can get through to a real recipient is a feature.

Thanks. I think this is the way to do it because it will be a more realistic test.

I use MailSlurp

Down for me

Reply on news.ycombinator.com