Use AWS Lambda to self-host the comments for your blog
jimpick.com
jimpick.com
For those who work in Python, there is a project called Zappa (https://github.com/Miserlou/Zappa) that allows you to take an existing project in a framework such as Flask and "host" the whole thing with lambda. The project is still young, but I imagine this sort of thing could be very useful to people doing a hobby project that don't want to pay for full-time hosting, but want their site to be available full-time. Hopefully, we get to a point soon where this sort of framework is ready for production use, though I don't know that it's there yet.
The project is maturing rapidly and has picked up some (soft) support from some major tech companies, which is neat.
I have been working on another major new feature that should be announced with a demo hopefully next week.
Same as lambda. So together, first million requests are free. After a year, you'll pay ~$4 a month it looks like, for lambda (40 cents for the first million if you stay under 100 ms per and 128 megs RAM; 20 cents for the function, and 20 cents for the processing), API gateway ($3.50 for the first million calls), S3 storage (a gig is 3 cents), and traffic (I'm assuming small amounts of traffic, obviously). $4.50 if using Route 53, and $5.50 ($12 per year, paid up front, but I'm averaging it out) if you get your domain through AWS. That's not bad for site hosting + comments.
I do wish they'd enable the Amazon Certificate Manager to be used on the API Gateway though.
Lambda is more flexible, but not as cheap.
If I get some time this weekend, I'll clean it and push it up to github.
[EDIT] Here it is. This has been working for us for almost a year. If there is interest, I can clean it up.
I've been considering adding comments to my static website for a while. I already deploy the website over IPFS, so that [in theory] the entire website can be distributed, and it can be duplicated/cached, etc when I have a spotty internet connection. Ideally, I would also have the comments published directly on IPFS as well, rather than having the comments be retrieved from somewhere external to the IPFS network. Your code should let me deal with that - just have the comment system fire off an email to me when a user posts a comment, and a daemon running on my PC can publish the comment next time it's online. Plus, I get easy email notifications :)
Such an approach seems a bit convoluted though. I'm not used to web programming, wherein it seems like every popular approach to hosting something as simple as a blog comes with pretty significant drawbacks or limitations. I'm much more fond of working in environments which offer an objectively "right" way to do every task.
Very cool idea! Thanks for sharing it :)
Luckily, AWS lets you share API definitions so I added that to the repo. It's the swagger.json file. I'll try to get this all automated to make it super simple, but for now you just have to replace all those placeholders.
Feel free to ask me any questions here, or using the comment system on the blog post. :-)
I know it's stupidly small costs (Lambda is wonderful for that), but some people still don't understand how to translate the pricing structure to "the real world".
[1] https://s3.amazonaws.com/lambda-tools/pricing-calculator.htm...
It's running in the minimum size of RAM: 128MB
What could happen with Lambda and similar services is that costs will take back under the spotlight "old" languages that run with little memory and little CPU cicles. I'm even thinking about C. I'm not into Go, but that looks like another candidate.
Scenarios: those ubiquitous web services running on a single VM because it's more than enough. The customer has to pay for it all the time and could save money if the app can run on Lambda. Enough to pay more in developer time?
I think most code running on lambda is not going to be CPU bound. In lambda-comments case it's mostly going to be idling as it waits for TCP/IP traffic to/from Akismet, S3, DynamoDB, Slack notification webhooks, etc.
From the looks of the git repo it's at least 65 cents per month but then there's a number of other services with no price reference.
What would you estimate it would cost per month to host a blog that gets 50 comments per month and what would the cost look like at the end of the year given costs would rise over time due to needing to store more info.
For argument's sake, let's assume no AWS free tier.
On paper, it should cost under a dollar per month. The main cost is just the DynamoDB instance. I don't think it's going to consume much S3 storage.
I guess if your blog has a lot of traffic, the network transfer might add up -- but it's just little JSON files.
https://jimpick.com/2016/05/05/introducing-lambda-comments/#...
For 8000 visitors, it cost me $1.37.
The numbers might be skewed upwards a bit because somebody posted a gigantic blob for one of the comments to test if I had implemented a limit on the length (I hadn't).
Hard to say for sure but I wouldn't be surprised if a normal blog received 5-6 comments per 10,000 views.
Why pay for a subscription comment service when it can be implemented as a script in the cloud that only runs on-demand?
If you will be able to make it less "technical" and more "plug-and-play", I think this will be cool viable solution to integrate (to be read: for customers like ours, that uses serverless computing in enterprise apps -> https://github.com/MitocGroup/deep-framework).
I did still get some comments too!
Anybody can still make a site in a few minutes on a number of places, they just aren't going to solve the abuse problem themselves without using a hardened tool.
Although this adds a bit of complexity compared to Disqus it sounds like a fun solution to implement, potentially with the addition of a way to moderate comments.
Then the comments would be in the DOM before the JavaScript is even loaded. I've done that with React apps before - it works great.
If that was working, I could rebuild the blog after every comment even, using a system like yours.
Any information written in disqus comments is only slightly more accessible than the dark web.
It's also not "serverless" as the repo readme claims.
http://serverlessconf.io/ https://serverlesscode.com/ https://github.com/serverless/serverless
Of course there are servers out in the cloud somewhere. But you're not renting them 24/7.
I'd love to experiment with a completely server-free P2P system, perhaps using WebRTC and WebTorrent (although in that case, there are still some servers involved).
Describing it extremely poorly, quite frankly. Also - this project is not just consuming resources from the totally not server based resources it inherently relies on, a key component of it runs on those totally not servers.
After reading a little on the pages you linked to, "Server-less" is just a 'cute' way to say "our app is vendor-locked to AWS and their Lambda/API gateway/hosted DB/hosted storage/etc services"
> Of course there are servers out in the cloud somewhere.
Trying to explain a trendy mis-used term with another trendy mis-used term doest't help your case.
Have you ever heard the phrase "there is no cloud, it's just someone else's computer"?
> But you're not renting them 24/7.
So, any application where the server-side component is scaled up-and-down dynamically, based on some kind of workload based schedule, is server-less?
No one is claiming that email works without servers and clients.
...who cares?
Okay, let's try another analogy. Let's say I don't own a car. Or lease. Or rent. Or borrow. That makes me "carless", right?
Moving on: I only take taxis or Lyft or whatever to get to my destinations. That would still make me "carless", right? I don't manage the fleet of cars on call for me. I don't have to worry about their maintenance. I am not responsible for a car, so I am carless.
You, the person being driven around, can be "carless" and you, as a site owner, can be "serverless". For the ride to be "carless", it would have to not involve a car. For the comments to be "serverless", it would have to not involve a server. This system involves a ton of different servers, so it can't be "serverless".
I am a person. I am in zero way related to the creation of oxygen. I require oxygen to live. Am I oxygen-less?
I could go on and on about the MFS, but I choose to leave it at that... the MFS are awesome. It even leaves room to the imagination. ;-)
Why do you feel the need for yet another marketing buzzword ? IMHO saying that makes one sound like an someone that don't know what a server is, but maybe it's just me. Just because you're not managing a server doesn't mean it doesn't exist, it obviously does, the code isn't running on each client but remotely.
Wireless internet is still connected to the rest of the internet via a wire somewhere; the obvious point being made is that your phone or laptop doesn't need a wire. Serverless here doesn't mean that servers no longer exist, but that you are not working at the server level to accomplish what you're trying to do.
Serverless is totally inaccurate for scripts uploaded ON a server. Since they are executed on a server. The scripts are executed and its result are potentially served from A SERVER. The fact that you don't manage it yourself doesn't change that.
That's an horrible example you are using to try to make your point, it just doesn't work.
So, just like every PaaS everywhere, since long before the new buzzword.
Wireless refers to a specific component of the connection - the the connection between your device and the relevant wireless base station.
No one technical is claiming "wireless internet" - and if they are, they're as guilty as the server-less crowd - they're claiming a wireless local area network (wifi), or wireless wide area network (3g/4g/WiMax, etc) - they're not inherently "the internet" they're just a radio based method for transferring packets of data.
Here, you have developers literally uploading the code they've written ( the javascript code that runs on AWS' 'lambda' service ), configuring 5 separate rented services and then claiming "look ma, no server".
The three referenced pages are:
Server-less conf - a conference about building apps on AWS lambda
Serverless blog - a blog about building apps on AWS lambda
Serverless framework - a framework for building apps on AWS lambda.
Every single "server-less" thing I can find, comes back to "using AWS Lambda". Which then makes me wonder why the need for a buzzword like "Server-less" - it's not like it's trying to describe a range of approaches and technologies (like say Web 2.0 was doing).
Ohhh of course. How stupid of me. It's deliberately vague and non-specific. You can sell a client on "we use a server-less architecture" because they think it means literally that - no servers, just computer guy magic. Is it as easy to sell a client on "we built the app to rely completely on 5 different services from a single company that has a history of anti-competitive behaviour"? I doubt it.
Lambda's published reference architecture is asynchronous invocation via queues listening to pub/sub events, which in integration terms is the antithesis of the client-server model. There is no listening server in the client-server sense.
So whilst the term "serverless" has obviously been chosen as a catchy marketing term for the mouthful that is "asynchronous event-driven self-scaling platform-as-a-service", it's not wrong.
Of course that's true, but "serverless" implies that you don't have to care about the "server" part of your application -- that is the stuff that isn't related to solving your problem and you probably don't care about anyway.
I like the term as it embodies what PaaS is supposed to be about: not paying a set fee per month for a server that you only use a couple of hours here an there but rather on a usage basis.
Hell, think about companies where infra is managed by a dedicated team of skilled experts. The developers there aren't patching or maintaining the servers.
What other projects exist to run a blog on Lambda? Apart from going full in on a locally-compiled static site that I just upload to S3, that is? (Hugo, Hexo, Jekyll, etc) I'm imagining a full admin web interface that uses a Lambda-driven API, themes, a front page with dynamically generated content from the Lambda-driven API, etc.
Granted, if you ever pass the free tier limits, normally the price breaks of these PAAS providers are high so cost does not scale 'by the cent' like AWS; however, if you pass the free tier you must be making good traffic which should translate to good income so infra cost shouldn't be a problem.
One of my motivations for doing the project was to just learn how to provision a bunch of AWS services and make it reproducible.
I might write a little shim so you can also just run the API as a standard Node.js app. At the end of the day it's just a JavaScript bundle and a single REST API method.
Even with a service like Heroku, a choice needs to be made for what service to use to store the data (eg. postgres). Heroku's free tier is almost like a course-grained Lambda because the VMs spin down after a period of inactivity.
I'm actually really excited about https://zeit.co/ - check that out.
In the long run, the hosting cost for this type of application should be pretty close to zero. People will choose their cloud/hosting solution based on other factors.
The modern programmer is quite good at playing Legos with the myriad of micro webservices out there (which is good play/training) but has forgotten how to think like an 'engineer' must think.
I'd defend this particular Rube Goldberg machine by pointing out that 99% of the time, none of my code is running on the server side - it's all Amazon S3. The only time my code runs in the cloud is when a comment gets submitted.
I'd argue that it's pretty easy to reason about. I've got some more blog posts coming. :-)
In fact, after reading about your software I am inspired to think and try to come up with such solution, let's see :)
It will desperately need a moderation queue.
Even smaller established websites get a ton of comment spam these days. I'd not even consider setting up any commenting system that doesn't have robust spam protection.
I need to add a little authentication interface for the admin - the complexity starts to pile up. But I think it can be done nicely.
Is The Lambda free tier something separate from the free tier you get on first signing up for AWS? I used up my AWS EC2 free tier years ago and I don't think you are eligible again once you use your free tier. I wish they'd put regular pricing at the top instead, i.e. for someone who doesn't qualify for the free tier you start paying from request #1.
I wish there was away to get stuff like this to work in a way that didn't take an all-or-nothing approach to noscript-filtering. As it is, now I have the option of no comments, or allowing scripts from: "s3-us-west-2.amazonaws.com", which almost might as well be "anywhere and everywhere". FWIW I see a similar problem with cloudfront and CDNs in general.
I think you can avoid the transpiling/babel dependency.
Great work!
I'm still using async/await, ES6 modules and some other more experiment ES-next features, so I think it will be awhile before Babel/Webpack can be completely removed. Of course, I don't really have to use those features. :-)
I wonder if the author considered AWS SimpleDB instead of dynamo. Costs are a bit cheaper.
At the moment, I think only DynamoDB Streams and Kinesis event sources will batch events together and deliver them to a single function (instead of many parallel functions). It's hard to understand just by reading the docs.
http://docs.aws.amazon.com/lambda/latest/dg/intro-core-compo...
I think the platform is evolving quickly, so I wouldn't be surprised if they come up with some more options in the future. I'm using DynamoDB streams since it's cheaper than Kinesis.
In contrast, SimpleDB seems to only charge for actual throughput used.
Kinesis might be a good solution if you wanted to use this code to run a Disqus-scale service with millions of blogs. Not much would need to change.
http://docs.aws.amazon.com/lambda/latest/dg/intro-invocation...
The Concurrent Executions section of this page suggests that anything other than Kinesis and Lambda Streams will cause parallel work, which is not what I want:
http://docs.aws.amazon.com/lambda/latest/dg/intro-core-compo...
You might be able to on Wikis though.
TBH, the thought of managing the attack surface of comments is kind of scary... Will definitely be referring back to this.
I personally just delegate the commenting to github and render the comments with javascript through github api.
http://ivanzuzak.info/2011/02/18/github-hosted-comments-for-...
Here's a nice overview of how this works.