The Quirks of Gmail UI
hanami.run
hanami.run
B1 is displaying a verified relay that doesnt match the sender domain.
The problem seems to be that the author doesnt want to display "via way.hanami.com", in which case he should set up DKIM for the domain he actually wants (the redacted domain) instead of "way.hanami.com".
Really shouldn't be advertising yourself as a "mail forwarding service" if you dont understand that...
Compare to Google's instructions for the-artist-formerly-known-as-G-suite: https://support.google.com/a/answer/173535
Gmail's UI has a lot of quirks (can we talk about how it's a productivity app with a fantastic search, that doesn't let you save searches?) but it's correctly calling out people who don't do DKIM correctly. OP's article doesn't even talk about anything other than DKIM; intended or not, the title is pure clickbait.
And that doesn't even get into the fact that any such labeling is visible as a tag on every matched conversation, leading to a need to have a concise name for your analysis to avoid information overload. Naming things is hard, and when designing interfaces, reducing cognitive load isn't just a nice-to-have.
https://irvineco.wixsite.com/goinggoogle/single-post/2014/10...
I understand DKIM and i have written this part in the article
> Somehow when we signed a message with DKIM, google decided to show the “Via way.hanami.run”, which is technically correct because the message lack of DKIM signature for original email(I made a typo and type gmail in original article) address
As in, I agree with gmail decision. My point is that, why when a message has nothing, gmail didn't say "warning: no DKIM signing".
I just hope gmail makes that fact clear on their UI when a message has no DKIM signing.
And sometime, to reduce minimal work for users, DKIM is optional for them. If they add our public key to `hanami._domain` we will sign it. But it's really tricky to configure DKIM for a normal user. Even gmail makes it optionally. In that case, gmail signed it with gappssmtp.com and yet, they don't say "Via gappssmtp.com"...
Because 99.9% of users would say "What is a DKIM?"
Google has decided that regular uses don't need to know if DKIM and SPF passed or not, because they don't know what that is anyway. They use those things in their spam filtering. If a message got past their filters, regular users assume they are safe, and even if it had DKIM status on it, that wouldn't change.
Power users can bring up the second screen you showed if they want to be sure.
In 2013, Google published a blog post on DKIM adoption and in 2016, they updated it: https://security.googleblog.com/2013/12/internet-wide-effort...
> 86.8% of the emails we received are signed according to the (DKIM) standard (up from 76.9% in 2013). Over two million domains (weekly active) have adopted this standard (up from 0.5 millions 2013).
> 95.3% of incoming emails we receive come from SMTP servers that are authenticated using the SPF standard (up from 89.1% in 2013). Over 7.8 million domains (weekly active) have adopted the SPF standard (up from 3.5 million domains in 2013).
> 85% of incoming emails we receive are protected by both the DKIM and SPF standards (up from 74.7% in 2013).
> Over 162,000 domains have deployed domain-wide policies that allow us to reject hundreds of millions of unauthenticated emails every week via the DMARC standard (up from 80,000 in 2013).
In conclusion, the evidence -- from Google itself -- shows that only as of 2017-18 has HTTPS adoption even remotely compared to that of DMARC adoption's statistics from 2013!
Gmail is terribly behind Chrome in this regard. Google Search began down-ranking sites not SSL protected in 2015, Chrome warned about submitting passwords on insecure forms in 2016, and by 2017 was rolling out or planning to roll out a series of changes to visibly mark insecure pages as insecure, one address bar change at a time.
In that same time, Google launched Inbox in 2015 ... and shuttered it by 2018, redesigning Gmail. They had plenty of opportunities to highlight insecure email transmission, they chose not to.
For email, the user loses deliverability and may not even be aware of it. Also it's often not within their control to fix. Marking non-signed emails as such would put a huge burden on email users and they might not even be aware of it.
Also, for websites, the website owner loses traffic if they don't offer a secure experience -- they are in complete control of their destiny.
For email, the user loses deliverability and may not even be aware of it. Also it's often not within their control to fix. Marking non-signed emails as such would put a huge burden on email users and they might not even be aware of it.
But JFI, the email that has invalid SPF and DKIM in the screenshot is the one forwarded by forwardemail.
While the behavior can be annoying, the first picture in the article conveniently cuts of the profile image near the sender details. I believe when both fail gmail adds a question mark icon of a red unlock. So no, it doesn't look more correct.
Ouch! But isn't setting up the DKIM the way you suggest the responsibility of the domain owner and not Hanami?
EDIT: Nevermind, I just read your replies below. Damn, forwarding is tricky business. OP should hire you.
EDIT2: typo
This is a generic problem with DKIM, it breaks email forwarding.
As I mentioned elsewhere, its not clear what the authors intent is - is it an originating server, or is it a relay. In both cases he shouldnt be signing with the relays key, but should either be signing with the domains key (if available) or forwarding the existing signature. Signing with the relays key is all but useless.
DKIM applies to what you set it to apply to in the DKIM entry. Here is what Gmail signs from the header (the body is signed by default) for their DKIM (taken from a recent Gmail originated message):
h=Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:
List-Unsubscribe:List-Id:Subject:To:Message-ID:Date:From:MIME-Version:Sender:
Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description:
Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:
In-Reply-To:References:List-Owner;
I kinda agree with what you said, but the point is that you shouldnt be mangling the from address when forwarding. That will definitely break DKIM, as its actually a required header fields in the sig - but theres NO reason to do that.
p.s. Ive clarified that i meant "original message" not "message body" in my post.
p.p.s Actually, I do think we are in agreement. I mentioned elsewhere that we dont know what the intent was - relay or origin, but in no case should they sign with the relays domain key.
The original DKIM signature breaks if you edit the message body or its existing header fields.
The "via" text is only shown if the email is DKIM-signed by a different domain name than that envelop From domain.
If the email is signed by the same domain as the From domain, there will be no "via" text.
They run an email forwarding service, so their SPF records will fail, but an untampered email should still pass DKIM. in fact, that's how most email forwarding setups work. If the message is tampared, and re-signed with the forwarding service's DKIM key, it's correct for Gmail/email-client to show the via-text.
To avoid this, the Hanami team could add a reply-to header that routes back to the original sender, update the from header to come from them (to pass DKIM and SPF), and instead of forwarding on directly, re-build the email and resend it as a fresh, DKIM signed-one. This is how we do it at IdBloc (we add the via bit):
from: Original Sender via IdBloc.co <sender-at-domain.com@users.idbloc.co>
reply-to: "Original Sender" <sender@domain.com>
To be fair, we don't actually know the authors intent (and I would argue that they dont either).
They should be either forwarding the signature (in the case of relaying) or signing with the senders domain key. They are instead signing with the relay domain key, which is all but useless.
At this point i feel like a lot of e-mail related topics are full of "unknown unknowns" to me and many other developers. It's hard to say whether we're missing bits of information that are considered common knowledge or maybe e-mail related development doesn't have enough discoverability (like blog posts, tutorials and so on) to make it easier, like configuring HTTPS in Apache2/httpd would have.
Or maybe the whole domain is just full of inherent complexity due to its history (SPF, DKIM, DMARC etc.).
> SPF
Is the relay authorised for this domain
> DKIM
Which domain(s) authored/approved this email
> DMARC
Formalises some behaviours for the above + reporting
The REAL problems with email come with deliverability. Basically an MTA not forwarding or accepting mail from you. This can be for a variety of reasons: you are misconfigured, they are misconfigured, one of your relays is on a blocklist, your relay exceeds a rate limit, your emails exceed some spam score, etc. This is the stuff that makes service management painful.
For example, gmail may give you a few "spam points" for sending them more email than they are used to from your relay - this means your emails end up in spam, get bounced and requeued for delivery (if you are configured correctly!), outright refused, or even mysteriously blackholed. It takes time to build reputation in that respect. Emails may be marked as spam by recipients, which can further tarnish your relays reputation with that provider (potentially multiple if they share reports with other providers).
Using an IP with a history of abuse, or having bad neighbours on your subnet, or having emails flagged with a spam reporting service can result in you being added to blocklists - requiring you to be vigilant in log monitoring, postmaster duties, etc.
Email isnt difficult, its just that bad actors have meant that you need to establish a positive reputation, rather than simply avoiding a bad one if you want to send any realistic volume of email to a large provider. DMARC/SPF/DKIM are intended to give accoutability in order to establish domain-based, rather than relay-based reputation - but ultimately its up to the receiving provider (and occasionally 3rd party relays, depending on your setup) how they handle it.
Other RFCs are either obsolete, or talk about some crypto specifics.
All you really need to know is that DKIM signs the message to prevent tampering. This includes the content and "from" header (optionally other stuff). Anyone can sign it, but its up to the verifier to decide which of the signatories to "trust". For example, you dont NEED to sign with the senders domain key, which is why google is flagging this 3rd party signature with some custom UI.
DMARC formalises this with identifer alignment, to ensure that a signatories domain key matches the "from" address domain.
Just because the "via" UI element doesn't appear in any RFC doesn't mean it doesn't represent something that does appear in an RFC.
I don't see that kind of thing in an RFC anywhere though?
It's very common to have From addresses different to Reply To; a contact form for example should do this.
With DKIM anyone can sign the email, its up to the verifier to decide who to trust and how to represent that. DMARC formalises who you should trust (i.e. the domain key for the sender).
p.s. It seems you can click on the "via" in gmail and it gives you some (rubbish and borderline misleading) information.
Then again what is email forwarding anyway, it's all such a mishmash
I send a lot of (wanted) mail and some subscribers have various forwarders set up which result in the eventual recipient server rejecting the mail due to the forwarded mail failing SPF and DMARC. It's a nightmare as we get the bounce backs telling us off but it's because the forwarders aren't, of course, sending from IPs included in our SPF record. (There are clearly ways to do it properly, but a lot of mail forwarding systems appear not to?)
The overall UX of Gmail as of 2021 is a mixed bag, and this is somewhat generously speaking. I mostly rely on search to find emails, as tags get hidden, the "Important" mailbox overwhelms messages the AI didn't deem "important" and the spam filter has never been worse. I was expecting a review of... the quirks of the Gmail UI.
More generally, the problem of email address exhaustion on consumer services is getting worse. Imagine being a kid making your first email account on Gmail or Outlook.com that doesn't look totally unprofessional. I got lucky to get full name @ gmail. (But, it's to Google's credit that they don't do the inane Yahoo approach of recycling addresses.)
Google Apps (or whatever they call it these days) is very affordable and gets many aspects of emails right. There are other well established alternatives too, like protonmail
So, unless you resort to an unreasonable suffix of digits, or a potentially unprofessional moniker, you've got no choices.
Then again, email may not be a viable personal communication method in the future. Looks like the future is chat apps.
Yes please. Especially since Microsoft offers it to a limited extent on Outlook.com for Microsoft 365 Home subscribers ('limited' in the sense, the domain must be registered with GoDaddy).
In case any Google employees are reading this: this would be a very useful feature for one.google.com.
> More generally, the problem of email address exhaustion on consumer services is getting worse.
And there's another pernicious problem: if you have a firstname@gmail.com or first.lastname@gmail.com, you're likely getting a ton of misdirected email even if your name isn't that common.
WE HAVE MODIFIER KEYS FOR A REASON
It drives me up the wall how absolutely godawful contemporary software is.
Unfortunately I'm starting to get the feeling that Gmail is no longer a high priority for Google, and so it's slowly bitrotting like so many other Google projects.
[1] Matches: {(to:me) (deliveredto:myaddress@gmail.com)} Do this: Never send it to Spam
0: https://support.google.com/drive/thread/75341417?hl=en&msgid... and https://support.google.com/drive/thread/5207602?hl=en&msgid=...
1: https://support.google.com/calendar/thread/12665854?hl=en&ms... (G response: https://support.google.com/calendar/thread/13429505?hl=en&ms...)
My point is that as an average user, when seeing this, what is their expectation. This is what happen when a user asked me about that today.
I understand DKIM. However, when one send a plain email without any DKIM, why google say nothing on their UI. If they say: "This email has no DKIM", it would be better. Now, the one who signed that email(original email didn't) get this "Via way.hanami.run" and an normal user will confuse when seeing it.
BTW, We do support DKIM signing in our dashboard, it will ask user to setup MX, SPF, and DKIM. However, many mail services will let DKIM an optional thing. Even google/zoho.
You can add domain and send email just fine without configuring DKIM for your domain. Google will sign your message with gappssmtp.com domain and won't say anything about "Via"
>>> Which one looks more trustworthy? Obviously A1 looks better. B1 has a suspicious text “via way.hanami.run”.
uhh... what is the problem with gmail showing the "via ..." badge? why, as an end user, should I feel less trust in the message?
Even if you do understand - isn't the "via" specifically designed to reduce trust? "Hey, head's up, this message actually came in through this third-party"
The reason is that if you maintain dmarc records you have to handle spam complaints and bounce reports. So if you’re using a provider like mailgun or ohmysmtp, you still have to set up a receiving smtp server even if you’re only sending outbound mail.
It's policy is set to "none". You normally do this when you are starting to investigate your DMARC setup and want the stats. This way you can fix things before you up the policy through quarantine and then to reject.
Thanks to all who take out their time to answer this 'ought to be obvious' question.
HELO/EHLO gives us a way to negotiate ESMTP, which is what practically everyone uses today.
Reverse DNS was added to SMTP servers as an anti-spam feature about 15 years after HELO was established.
But to be honest, I'm not sure HELO was ever really useful. In the very early years, it might have been appropriate to reject messages where the MAIL-FROM domain did not match the HELO domain. Not for long though.
It's better to be prompted for those, instead of just sending them unannounced, in case client just want to switch to TLS first via starttls.
Can it? Not every machine that sends mail is on the public Internet.