Sendy Is Insecure: How Not to Implement ReCAPTCHA | Hacker News Reader

Sendy Is Insecure: How Not to Implement ReCAPTCHA

victorzhou.com

134 pointsmatteocontrini72 comments

I am the author of Sendy.

There are a lot of miscommunication between me and the author of this post. The selected snippets of messages posted on the article seem to put me in a bad light, however it's only one side of the story. The selected messages posted on the article are ones that are favorable to his argument. For example the author did not post a screenshot of me saying that I will be looking into this but went ahead to write an elaborate blog post immediately to put Sendy in an unfavourable light.

Bugs and issues with security has been and always will be the top priority with Sendy over the years. I agree the client side parameter 'subform' bypasses the reCAPTCHA and should be fixed. It is an oversight. And it will be fixed.

Kudos for commenting here. I'm just an anonymous internet person but I'll share some advice anyway.

There are concrete things you can do to improve.

1) When someone reports a security issue to you, always always always thank them, even if you think that they're wrong, and are about to tell them. They're taking time that they don't need to take, with the goal to help you deal with a problem that is so much more yours than theirs. In other words they are demonstrating generosity, so thank them for that. If it turns out that they were wrong and that there is indeed no issue, well, no skin off your back; if it turns out that they are right, you'll be glad to still have them on your side rather than writing posts.

2) Be more curious. Instead of declaring "This is not a vulnerability or a security issue", as if the issue was closed, you can simply ask: "I'm not understanding why this is a vulnerability or a security issue. Can you explain and demonstrate a proof-of-concept attack?"

3) Consider getting advice from a security-minded person on such issues. It doesn't really take credentials. It just takes a certain kind of mindset that, in my experience, is not held by the majority of even very talented software developers.

I have to say, reading your response paragraph that starts with "if a human opens up his browser console to remove the subform parameter...", I recognized a very common feeling in me. Oh no, this person is just not getting security. Same facepalm reaction as the author. Having an API parameter that lets an (untrusted) client override a security measure isn't an "oversight", and more like a big design flaw. Kind of like if someone had a login API with a parameter called "pretend_password_is_correct" that let you sign in as anyone when set to true. If you're not seeing the issue when pointed out to you so clearly, it is really in your best interest to not make security decisions by yourself.

You shouldn't feel alone in this. Most developers that I've worked with, even top talent, cannot manage to put themselves in an attacker's shoes. Usually it's hard enough to put yourself in the normal user's shoes and get the thing to work for them. Following advice from people who have the skill of thinking like an attacker is the most valuable thing you can do to protect yourself.

Thanks for your valuable comment and advise. I appreciate it. Definitely something to be learnt here.

Unrelated to the issue:

I've been asking this hundreds of times but never got an answer, why doesn't Sendy add a visual e-mail builder?

Customers are going crazy over the issues with Wysiwyg and tables to make simple two-column designs, while drag/drop builders are straightforward and simple while keeping the code fully compatible with mailers.

Basically all competitors have drag/drop builders now, even the simpler one-time-pay scripts on Envato.

check out https://mjml.io/. Its a great solution for creating email layouts in code. I use it with vcode where it has an option to preview the result.

Many ESPs these days license 3rd party editors that cost them recurring fees on a per user basis. Unlike hosted solutions, like BigMailer.io or SendGrid (both license drag and drop Bee editor), Sendy charges a very low one-time (!) fee of $59 to download the software once.

Sendy's pricing model simply doesn't allow such 3rd party product licensing and I imagine maintaining a heavy front-end application like a drag-and-drop template builder across the variety of clients is too costly (dev and support time) as well.

Sendy has some massive problems with code quality and always had. The last time I took a look at it (because I needed to send emails), it was quite horrendous. I would not be surprised if it was insecure - was literally a "nooooooope" right back out.

Background: I write PHP for fun, I don't compete with Sendy, I don't have any stake in any service that competes with Sendy. I also don't expect PHP to be written in the most OOP-everything Symfony-esque way either. But this is a bit too much for me even.

It doesn't follow any modern standards or guidelines; it doesn't use any templating system, just PHP hardcoded inline with HTML in hundreds of files (no modern framework, lightweight or not), concatenated variables into SQL queries and HTML. The author seems to hate using { } for if/else statements, which has the potential of introducing fun bugs if not extremely careful. 1 and 2 letter variable names are common (and I don't mean in the way of $i or $j); parts of it border on insane or bizarre; there are 500 character lines where instead of an error generation feature they just die() with a full HTML document as a string for error pages; functions are written inline all over the place and they all use the keyword global; there are no parameterised queries - it's covered in repetitive mysqli_real_escape_string and query concatenation. There is a massive amount of copy-pasted code: instead of some kind of single database management layer, there's a re-definition of a function that connects to a database with the same name in dozens of files, all of which try to read global variables (doesn't take any parameters)

A client of mine uses it (believe me moving their mailing list into their main app is absolutely scheduled work) and thus I’ve had the “pleasure” getting to know it in an apparently non-common scenario for sendy users.

Our MySQL env is a pxc/galera cluster, and connections require tls. When I asked the developer about adding tls support (about 4 lines of code if you’re using mysqli as he is) I just got a flat “nope, not doing that”.

At which point I lost all reservations about deobfuscating the “protected” files and simply replacing the db setup call in every single god damn file with a single setup that accepts tls options.

I’ve worked on some garbage code projects, but sendy is almost deliberately bad. In one place it: disabled all error reporting; tries to connect to the database; fails silently if the connection doesn’t succeed.

There is zero logging (besides an ungodly number of notices and warnings about undeclared variables, etc) and the author has apparently zero interest in actually fixing or improving any part of it at a technical level.

For those who work in php and want a comparison: it makes Wordpress in general look like a well architected application.

That's all true (I haven't checked myself, but lots of people talk about it on Reddit and elsewhere).

Still, the big selling point is: Inexpensive. Easy to install (simple PHP). Looks relatively polished.

All the "competitors" are either much more expensive, or you need to be an enthusiast in the tech stack they have chosen.

phpList is free and open source:

https://www.phplist.com/

We've been using it for years without any issues.

Shameless plug: https://listmonk.app (FOSS. Written in Go)

I'm curious, what made you go with a multiple lists approach instead of tagging users on a single list?

A lot of popular (but expensive) services like ConvertKit have moved to a tagging approach many many years ago.

It's so much easier to manage your list of emails when it's a single list and then you add tags to specific users like "purchased X". This way the email address only exists in 1 spot and you can segment on the tags.

The Sendy approach (and from the looks of it your app too) becomes very unwieldy with having to manage, parse and merge multiple lists on a regular basis.

+1 for this approach. This was the #1 design decision we made with BigMailer.io back in 2015. It's not trivial to change the model once you go with unique lists approach.

Performance is extra helpful for making real-time segment size calculations. We use elastic search and it's costly.

listmonk supports arbitrary nested tags and attributes in the form of JSON. A subscriber can have properties like {"purchased_x": true, ...}. It's possible to issue complex SQL expressions to filter and send campaigns to subscribers.

The multi-list approach has several other benefits. When manager / sender (and other) permissions get introduced, it will be straight forward to restrict users to managing certain lists. In addition, multiple lists allow subscribers to selectively subscribe / unsubscribe from lists.

Internally, the structure is simple. There is only one subscribers table and subscriber data is not duplicated anywhere. List (foreign key) relationships are in a separate table.

That sounds very promising.

So in theory could you have 1 list and each subscriber has many tags, and then you can segment on those tags?

Also, do you have any public success stories beyond your own Zerodha campaigns? Have you compared delivery / bounce / etc. rates vs Sendy and other tools using the same email providers? Also how fast can you send emails out through SES?

It looks to me like it sends via a local mail server, not SES? Totally different .

Any SMTP server. SES also offers an SMTP interfaces.

Really nice, ever thought of adding a drag-drop editor?

Like GrapesJS: https://github.com/artf/grapesjs-mjml

That looks interesting. Will check it out.

@Tomte - You can use msmtp listening on localhost to forward to SES, authenticated

Okay. Again, different target group. Sendy targets people who have no idea what msmtp is. They upload the files to their web host, enter a database password the hoster have them and AWS credentials, and they are done.

There's also Mailtrain https://github.com/Mailtrain-org/mailtrain

https://dadamailproject.com

Also on github:

https://github.com/justingit/dada-mail

</shamelessplug>

Nice writeup, @vzhou842.

I feel like part of the problem here is just miscommunication. Victor and Ben seem to be talking past each other.

This line stood out to me:

>>There’s no way to implement Google’s reCAPTCHA in an API.

>That can’t be right - the reCAPTCHA documentation has a dedicated section on Server Side Validation!

I assume what Ben meant was that it's impossible to implement reCAPTCHA entirely in an API. The nature of reCAPTCHA requires you to have a UI element, which would be impossible in an API. Victor interpreted the claim as if Ben said that an API can't support reCAPTCHA, which would be incorrect, but may not be what Ben meant.

I feel like the lead sort of got buried in Victor's report. The headline to me is that abusers are actively automating phony signups with this vulnerability. I don't see that stated explicitly anywhere. The closest is this line in Victor's third email, after the conversation has gotten somewhat heated:

>What good is reCAPTCHA if anybody with a computer can write a script in 5 minutes to spam your email list with thousands of fake signups.

The fact that attackers are exploiting this in the wild seems to be the most salient point, but I'm not sure that Ben knew that from the correspondence shown.

> I assume what Ben meant was that it's impossible to implement reCAPTCHA entirely in an API. The nature of reCAPTCHA requires you to have a UI element, which would be impossible in an API.

That’s exactly what I meant. Thanks for picking up on this.

> The fact that attackers are exploiting this in the wild seems to be the most salient point, but I'm not sure that Ben knew that from the correspondence shown.

I know that and hence was working on a fix for the next update.

Even though some of my comments may not be in agreement with the author, but I did mentioned in my email conversation that I am looking into it. But of course that was being left out of the post, no screenshots of that comment was found in the author’s post.

If I had released the next update without addressing this issue then yes feel free to write a post with these accusations. But I wasn’t given the benefit of the doubt.

> Sendy treats these email addresses as distinct, but they’re actually largely duplicates because of the Gmail period trick.

But they are distinct! Why should anyone have to special-case some provider's hack? Do you expect Sendy to special-case plus addresses? For which hosts?

Sure, it's nice if Sendy handles special cases, but it's strictly a bonus feature, and not doing it is also correct.

(The main issue with the hidden field is a valid complaint, of course, and it really doesn't look good)

He didn't say they should have a special case. He just made a factual statement, "Sendy treats these email addresses as distinct, but they’re actually largely duplicates because of the Gmail period trick," and looked to implement recaptcha as a solution, which was poorly implemented. However, if you were going to do create one special case for a provider, gmail would be a good choice.

These addresses are distinct. They just point to the same mailbox. At best they can be called aliases. /p

Author here. My wording could probably be better b/c it makes it sound like I'm blaming Sendy, but I'm not trying to complain here - I agree they should NOT be special-casing this kind of thing.

The use of + for subadresses is somewhat standardized in RFC5233

Wow, the response where Ben claims bots aren’t humans so they won’t remove HTML form elements before submitting is an embarrassing look for Sendy.

For anyone wanting an alternative, there's Mailwizz - one time payment, and you can self host and connect it with SES/Sparkpost/Mailgun/etc.

I've mentioned it at least a dozen times here on HN because it's an underrated piece of software that works and is actively maintained, unfortunately their marketing is very weak.

Bonus: the code is not obfuscated, it's built on PHP using a proper framework and the author is very active on the support forums. I'm hosting it on Webfaction (currently migrating to Opalstack) for cheap and almost no maintenance.

Thanks for this, haven't seen it even though I've been actively searching!

I think it's hard to get taken seriously by companies when you sell a "script" on Invato too and I'm not sure why they do this? Why not just sell it yourself and receive more of the profits?

I guess it's somewhat easier if the product is good enough (Envato features it) and the developer cares more about developing than marketing it.

I guess that's true, everyone coming to the site will then see it featured and buy it.

Nothing prevents him from doing it in the future though, when he already have enough customers to make a recurring cost profitable.

That would make me concerned about the engineering culture at the company. The initial ill-reasoned response sounded like a response he gathered from the tech team. If they knew what they were doing they would agree with Victor.

Afaik sendy is a one man band playing a triangle.

Obviously, this should be fixed, but shouldn't you also be doing double-opt-in and making the user confirm the subscription by clicking a link in an email?

Definitely, and I was doing that, but everytime a new spam variant (with gmail period trick) of an email gets subscribed that same email gets ANOTHER double-opt-in email from me. This is my guess as to why I was getting reported as spam so much - people would literally be getting spammed by me (unintentionally).

EDIT: forgot to mention I'm the author of this post

Yeah we definitely use recaptcha in a GraphQL API for sign-ups to browserless.io. There is a doc on how to do so programmatically via JS in the browser. The docs aren’t great, per se, but it is definitely doable.

EDIT: docs are here: https://developers.google.com/recaptcha/docs/display#javascr... though the lifecycle of how this all works is confusing. I’ll have to do a write-up on this at some point.

Does anyone know of a self hosted email list management solution that uses a tagging based approach of managing users instead of multiple lists, supports auto responders and also has a decent API for at least adding users to a list?

I have not found any self hosted solutions (paid or free / open source) and I've looked a lot in the past. I've seen services like Drip and ConvertKit offer this but both of them are at the high end of hosting costs. To put things into perspective it's $80 / month for a list of 3,000 users with ConvertKit even if you send nothing.

One of the more polished tools I've seen is Freecodecamp's https://github.com/freeCodeCamp/mail-for-good project but it doesn't support auto responders or tagging. I've opened issues on this 2+ years ago at https://github.com/freeCodeCamp/mail-for-good/issues/196 and https://github.com/freeCodeCamp/mail-for-good/issues/197 but I don't think they are going to implement it.

BigMailer.io isn't self-hosted, but for a list size of <5000 it's completely free, and supports bulk + auto/drip + transactional (via API) campaigns. We also have drag-and-drop template editor, Zapier integration, and live chat support 7 days a week.

ConvertKits is expensive - all plans fall under BigMailer's free plan :-o

I could be interested in writing one, having just written a semi complex wishlist management app with some overlap in user management features. Would nodejs suit a project like this do you think?

I'm more of a Flask, Phoenix and Rails type of guy but if you prefer using Node sure, you probably couldn't go too wrong if that's what you have extensive experience in.

Also a Rails guy but have found Node is quite a good fit for async work like managing email queues. Would use Phoenix but don't have enough production experience in it yet..

My company sells a self-hosted email list management solution that meets the requirements you mentioned:

https://www.greenarrowemail.com/

Thanks but your self-hosted solution costs $600 to $900 a month to license it.

I was hoping to find a 1 time purchase solution that meets those requirements. Sorry for not being more clear in my description.

We also have a one-time purchase option. Although it starts at $6,000. We serve a lot of larger businesses with higher volumes, like Six Flags or Fry's Electronics, and email service providers.

Yep, I saw that. I didn't mean for my comment to come out rude either. Just catering to a different audience. Thanks.

I run a service in the same space (https://emailoctopus.com/amazon-ses) and we see a lot of Sendy defectors. Their code desperately needs a rewrite.

Just reading the first two responses from "Ben" in that e-mail thread and parsing his attitude should make it obvious to anyone that this is a project you should steer well clear of.

An update of Sendy had just been released to address the reCAPTCHA issue → https://sendy.co/get-updated

>Plus, it should be super easy for you to fix this - all you have to do is remove the check for "subform" and drop support for that field in your API.

It might not be that easy, because there might be a bunch of users currently depending on the current behavior, and as soon as a real ReCAPTCHA token is required, they will break. They might need to introduce a 3rd ReCAPTCHA option. So they would have 3 options "ReCAPTCHA off", "ReCAPTCHA legacy weak", and "ReCAPTCHA on".

Author of the post here - Happy to answer any questions. I'll be editing the post with responses/updates from Sendy. Hopefully this convinces them to release a patch!

Why would you willingly use such a garbage POS script? There's better services out there, or you could just roll your own.

Any suggestions?

Seeing as you asked, I’ll give my startup a shameless plug:

https://emailoctopus.com/amazon-ses

Here is a patch for subscribe.php (4.0.3.2) to address the captcha issue from this post and another issue that allows bypassing double-opt-in by setting silent=true: https://pastebin.com/dT1NszTt

this change requires verifying secret api key in the subform=no case and restricts opt_in bypass to this subscribe api usage (since captcha is not good enough to stop all bots)

I've seen cases where spammers abuse sign-up forms to generate spam (by using the name field for their content). Without server side validation it would be trivial for a malicious user to accomplish this.

Shameless plug - I run a hosted alternative to Sendy https://www.mailblast.io/

>// DO NOT USE - it's illegal and will get you in trouble.

>WARNING: DO NOT use this code to attack a real email list without permission. That's super illegal and can get you in serious trouble.

"Super illegal"?

almost certainly construed as some kind of CFAA violation

CFAA, maybe.

So are you recreating the recaptcha code in your custom form?

Author seems self-righteous and does not understand what he is being told by Ben from Sendy. This is additionally evidenced by hundreds of lines of email and a blog post for what could be addressed with 12 lines of code modification.

Sendy is open source (albeit not free) and running on your own server. Ben's response says because you are using a customized webform instead of sendy's provided forms, that you need to handle the captcha code on the form and siteverify on the backend on your own.

Before sendy had recaptcha support you would have to modify the subscribe code to include a call to captcha siteverify. Here's some PHP code for how you call siteverify before proceeding with something: https://www.adam-bray.com/2018/04/02/adding-recaptcha-with-p...

You can avoid running your own backend altogether now anyway since SES supports bulk template emails. You can store all your form submissions using a lambda/cloudfunction/webworker to verify the captcha and store the list. When you want to send email you can pull your list into something running on your laptop and then invoke the SES bulk templated email from there. You can use lambdas for pixels and custom links that update records for those users. I wrote my own angular-firebase version of this

My reading of the issue is that Sendy's webform allows the external requester to bypass the server-side captcha logic by changing a client-side "hidden" input. If you want to be protected then you have to customize the form.

I have the source code too and checked it already. The gist of the code is:

if subform:

   if captcha fails:

       feedback = "Failed recaptcha test"

... if feedback!='Failed recaptcha test' (&& other stuff)

   do subscribe

edit: misread the code and formatting on HN didn't even show my intent, but the subform check doesn't contain the subscribe logic. The bug is clearly that it doesn't check if the captcha has passed.

And the point is that anyone with even a modicum of dev experience can remove the `subform` field and automate submission to the otherwise-standard form and completely bypass ReCAPTCHA.

The issue goes even deeper: if subform is set to no then sendy considers the user as added via api. This should mean that it would verify_api_key before allowing such a submission, but sendy doesn't verify the API key for subscribe calls (doh!). Old forum posts suggest that double-opt-in is a solution, however not only can you bypass the captcha and form with subform=no, you can also bypass double-opt-in via the subscribe API by sending silent=true in your POST.

Here's the diff for subscribe.php to require captcha for all list adds: https://pastebin.com/tSLkvYME

if you need API support, you could use captcha_passed = verify_api_key($api_key) and include the api_key in the post request

> Sendy is open source

According to what or who? It ships with laughably obfuscated files, it makes no mention of open source in any documentation.

> Sendy is open source (albeit not free) and running on your own server.

How does that work? What's the license exactly?

It's not:

> This Agreement grants a non-exclusive, non-transferable license to install and use the Software on a single Website. Additional Software licenses must be purchased in order to install and use the Software on additional Websites. The Author reserves the right to determine whether use of the Software qualifies under this Agreement. The Author owns all rights, title and interest to the Software (including all intellectual property rights) and reserves all rights to the Software that are not expressly granted in this Agreement.

> [...] You may not:

> Distribute derivative works based on the Software;

> Reproduce the Software except as described in this Agreement;

OP might mean source accessible (rather than compiled or being a hosted service).

https://sendy.co/end-user-license-agreement

Reply on news.ycombinator.com