Automatically sending Webmentions from a static website | Hacker News Reader

Automatically sending Webmentions from a static website

jamesmead.org

75 pointschrislo29 comments

I implemented webmentions for my static site back around the end of 2019. My solution for receiving on my static site was to just log POSTs and review them with my eyes. Then if someone has actually sent a webmention (as opposed to pingback spam) I just use curl to respond. There's no need to have anything automated at all.

ref: "A static, manual system for receiving webmentions (and pingback) with nginx" http://superkuh.com/blog/2020-01-10-1.html

That's what OP originally did as well.

Just to correct your blog post, the Webmentions spec says in this scenario where you're just logging, you "MUST" return a 202 Accepted. I went through a similar phase (logsink with 204) which I corrected recently.

https://www.w3.org/TR/2017/REC-webmention-20170112/#h-receiv...

I was wondering if I should've sent this comment as a webmention instead but I guess old habits die hard.

Why do you need an always on service for this? Just send the webmentions whenever you compile your static website - that's the only time content actually changes.

On Pelican, you can do it using this plugin https://chezsoi.org/lucas/blog/pelican-pingback-and-webmenti...

Make sure to only do it after you've deployed it - part of the WebMention process is that the mentioned page actually checks the page that sends the mention for the given link.

(Also annoying if the link is added client-side.)

To be fair [emphasis mine]

> Upon receipt of a POST request containing the source and target parameters, the receiver SHOULD verify the parameters (see Request Verification below) and then SHOULD queue and process the request asynchronously, to prevent DoS attacks.

https://www.w3.org/TR/2017/REC-webmention-20170112/

Though, probably, this won't happen in practice. So better to be careful.

Even if done asynchronously, if the link is not present the moment you send the WebMention, it can still happen between sending it and deploying it. But you're right, if the two are close together and a receiver is actually processing them asynchronously, it'll likely go well.

I also use Remy Sharp’s code, but as a library. I have netlify call this glitch [1] I wrote on successful deploy. It compares the new sitemap.txt with one it has cached and scans new URLs which match a pattern for mentions to dispatch.

1: https://glitch.com/edit/#!/lean-send-webmentions

I like the idea, but I wonder if I should bother: in other words, are people actually using this? Is this the new pingback, because that seemed like a good idea, too, but it never got off the ground.

I helped create pingback [https://simonwillison.net/tags/pingback/], which eventually made it into WordPress and hence achieved pretty widespread distribution.

My concern here is spam. With hindsight both Pingback and Trackbacks (which Pingback was a response to) essentially became APIs fit spammers to add junk to your site. Can webmention avoid the sane fate?

I would think that because most of the sites are static, they are going to be doing some moderation before re-rendering and posting the linkbacks.

The README says that one of the steps is verification [https://github.com/converspace/webmention/blob/master/README...], but it's a little disappointing that it's essentially an "exercise left for the reader".

"The receiver SHOULD perform a HTTP GET request on source to confirm that it actually links to target"

We did that in pingback too. It turned out to be trivial for spammers to circumvent.

That's a bummer. Did you find out any other more robust ways to filter out spam ?

I gave up and stopped using it (well, I didn't bother reimplementing it on one of my various blog engine rewrites).

If I were to implement pingback or webmention today I'd use a moderation queue with the ability to allow-list trusted domains so they get to skip moderation in the future.

I've implemented WebMentions in a project that uses it as a push notification system for websites that integrate our widget (which is just a <script> tag they include on their page). That kinda works: if you integrate the widget, you know you can expect WebMentions from https://plaudit.pub, and thus add it to an explicit allowlist.

https://github.com/zerok/webmentiond has allow list and block lists :) I am very happy using it

Most POST spam is repetitive. It's very rare human POSTs are. If you filter out any POSTs that happen identically more than 3 times you remove most spam. It's not perfect but it makes it manageable. Of course this is a lot easier to implement if you batch process.

I've had them on my sites for a few years now, and even with bridgy passing tweet replies through the spam hasn't been too bad, but certainly adding an allow-list makes sense too. Having built Technorati that effectively did this at scale, I do appreciate the spam problem, but decentralising implementations has so far worked out OK.

Marty writes up some of the variety of implementations here https://martymcgui.re/2020/07/15/what-we-talk-about-when-wer... (which I saw because he webmentioned my in it).

I think Webmention is viable at the moment just because of the low adoption across the web. Just like with pingbacks and trackbacks, any kind of manual moderation becomes infeasible when valid posts get swarmed under thousands of automated spam posts.

The typical verification step (check if the originating site contains a valid link) is trivial to work around. One attempt to address that is Vouch [1], but I believe it's largely untested in the wild.

[1] https://indieweb.org/Vouch

BTW, there's a pretty good discussion of pingback compared to webmention here, including how to bridge between them and some of the spam problems too https://indieweb.org/pingback

I’ve seen it on a number of blogs. Below a blog post, it shows who linked to it. Anyone who finishes reading the post can look at these responses, read them, follow them to their source. Does that qualify as “using this”?

Sure. What I meant was, of course both pingback and webmention are being used, but they did not give rise to the kind of decentralized conversation that I hoped they would when I learned about them. Instead we got Twitter. But I still think this is what people should be doing. I suppose I’m asking, should I use both pingback and webmention? Or just the latter? Does this replace pingback, or should I implement both?

This is a great idea, wonder how I've missed this.

I’ve never even heard of Webmentions. Should I have?

Webmentions are the modern standard that evolved out of trackbacks and pingbacks.

If you’ve written a blog post that references or links to another blog post, Webmention is the standard way to notify them.

An example use case for Webmentions is blog comments. A blog may publish a post which is a reply to another blog, then send a Webmention to notify them. The author of the blog post being replied to could choose to display the comment below their post.

It still amazes me that people have managed to create a viable system of decentralized comments across independent, statically-generated websites. Perhaps I should heed the (humorous) warning by Ben Werdmuller here, though:

https://aaronparecki.com/2013/05/21/4/xkcd#mentions

The page I linked to was mostly intended as a nice example of Webmentions being used in the wild, but I also thought it made a nice point about the risks of centralized systems versus decentralized ones.

Examples of the latter would be spam (as mentioned elsewhere above), and coordination problems when trying to upgrade the standard in potentially backwards-incompatible ways. Hopefully the continued adoption of Webmention will prove that these risks are manageable.

I am more interest in the action flow then the webmention

Reply on news.ycombinator.com