Hacker News Reader

Comment by adtac | Hacker News Reader

Lol it's so surreal seeing your project when you're randomly browsing the frontpage. Anyway, creator here, AMA!

>adtac has succeeded in reducing download to just 11 kB and isn’t done yet

Someone raised an issue to support Brotli in Commento (which I wasn't aware of) [1] and I did a quick test; it'd decrease the JS+CSS payload even further to under 8-9 KB. Nearly all modern browsers support Brotli [2], so I'm really looking forward to this.

I now realise this could be a modern take on the famous C10K problem, just with a different meaning: developing web services with under 10 kilobytes of payload :)

[1] https://gitlab.com/commento/commento/issues/125

[2] https://en.wikipedia.org/wiki/Brotli#Browser_support

View on HN

The brotli experience has been frustrating.

All modern browsers support it, so you'd think it would be an easy win. The top nginx search is an official looking Google project which is long abandoned and doesn't support current versions of the library. There's an official nginx module which is commercial, and then finally there's a supported fork that that sits on about page three of Google.

Shared hosts don't generally support it. AWS Cloudfront doesn't support it, along with several other CDNs. Big + to Cloudflare for proper support.

It reduced my bundle sizes measurably. We all hate bloat and want to reduce bundles, but there seems to be a community inertia around mass deployment.

Ah, that sucks. The hosted version uses Cloudfront as CDN for static assets, too bad they don't support it.

>official looking Google project which is long abandoned

Is [1] what you're talking about?

[1] https://godoc.org/github.com/google/brotli/go/cbrotli

The [1] I should have linked is here: https://github.com/google/ngx_brotli/issues/72

Maintained fork: https://github.com/eustas/ngx_brotli

Anyone know why brotli needs to be generated at the web server level? Just curious why/if it can't be done at the web app level.

Its generally used as a data transfer compression system. So the web server needs to compress the data before its sent to the browser. If you did it at the web app level (client/browser side), the data would already be transferred so there are no size savings.

Many web servers will serve file.ext.(gz|br) as a pre-compressed option. In order for brotli to beat gzip on size, you need more CPU time, so it's best if precompressed. This also means it's not really an option for dynamic content.

Consider the instructions for the nginx fork:

https://github.com/eustas/ngx_brotli

The "filter" module operates totally within nginx, and behaves roughly how we're used to with gzip. The "static" module allows you compress files as part of your build and just serve them - but the server still needs to understand the relevant headers and how to deal with them.

Ever looked into OpenResty?

Take a peek at https://github.com/sjnam/lua-resty-brotli.

Is it easy to move from hosted Commento into self-hosted Commento, and vice-versa?

Not yet. You can export comments from Commento as a JSON, but you can't import from the same format. Sucks, I know, but I'll try to implement this soon.

Do you see any issues with scaling in this project? I'd be very interested to know what you chose as backend architecture and how you handle so many requests parallel to each other, as I've struggled to build a good multi-threaded express backend in the past, would be awesome to hear some other stories. Do you think your service can handle a massive surge, let's say a really big player would pick you up?

Edit: Also really surprised about your pricing strategy. What if somebody wants to surpass the 50k pageviews? Wouldn't it be better to at least give those another tier?

I've done some basic benchmarks and it has scaled pretty well. Mostly thanks to the excellent performance and scalability that comes from choosing Go and Postgres. IMO backend servers are where Go excels.

I'm planning on adding Redis support soon enough so that sudden spikes would have little impact on the database server (which is nearly always the bottleneck, unless you're compute-intensive, which isn't the case with Commento). So that if you suddenly get a few hundred thousand visitors, most API responses are cached and only a fraction will actually require disk access. Like reddit does, where it'll temporarily say there N comments, but when you open the page, there wouldn't be that many. A few seconds of inconsistency is a fine compromise in the comments world. Of course, Redis isn't going to magically make everything faster, so I'll have to be careful with where I introduce it. Optimise for the common case and all that.

I'm still in uni, and I've never actually worked on large-throughput, high-scalability stuff before, so all this has been a great learning experience for me. If you have any suggestions or war stories that might be relevant, I'm all ears!

I'm by no means an expert in web backend architecture, but what you have done seems pretty solid to me with my limited knowledge. We've been mostly working with Express, that's why I asked to hear some feedback about how others have done.

And also, very well done! Your software seems to work very fine, a lot of thought already went into scaling and general stability of the backend. And for me, as fullstack working for a design company, your website layout and design is pretty good. If you didn't have help with that, kudos to you, good work, especially for someone still in college :)

Edit: Website performs pretty good in a Lighthouse audit, apart from Accessibility features.

I am happy to help/pitch in for the Redis support. Let me know, if you would like to collaborate.

I'd love to get help with Redis. I don't know anything beyond the absolute basics, so it'd really be useful if someone with more experience give lend a hand! If you'd like to discuss, please add your input here: https://gitlab.com/commento/commento/issues/75

That's pretty cool. Last I checked on my own site Disqus downloaded 1.3 MBs of bunch of stuff, which I think is just ridiculous. After caching it's only 2 KBs but it still bothers me - what in the world it's downloading? main.js seems to be 1.2 MBs and the main culprit. EDIT: well that was the ungzipped size, 263 KBs gzipped the CSS was larger with 353 KBs. But however, it works and has a free-tier so I guess it'll do. Maybe one day I'll try this out, but for now I can't be bothered with self-hosting.

How does it take care of spam? I hope it's not via Akismet, as I've not found it to be good enough.

When did you start working on Commento?

Around Apr 2017. Released v1.0 (almost a full rewrite, but still in Go) in Jan 2018. Launched the hosted version in Feb 2019 after almost 6 months of private beta.

Cool! Did you consider the pay-per-use model instead of a fixed monthly subscription?

How is the hosted version doing?

It's doing alright. Pays for its server bill, so I'm happy about that :)

Just emailed you about an enterprise account. My biggest concern would be longevity. Our org is an institution decades old and can't have a startup sunsetting on us a year after implementation.

Reply on news.ycombinator.com