Hacker News Reader

Comment by cvrajeesh | Hacker News Reader

May be you should consider rejecting random urls from your thumbnail generation endpoint. Now I could generate thumbnail of any website e.g. my blog https://hnews.xyz/thumbnail/?url=http://rajeeshcv.com

View on HN

It's a hard one to do.I agree you can do that now. I'm thinking on a solution, but it's not really easy. Right now I have no solution.

It shouldn't be too difficult - surely right now you have some sort of task that scrapes the Hacker News API to present the stories? Generate the thumbnail then, store it on S3.

Another solution I can think of is - encrypt the URL with a symmetric key. Decrypt it to generate the thumbnail

Thank you for the suggestion. I'm not so familiar with this. Will this work on frontend too? If I expose the secrets on the fronted, then it's pointless. Do you have some suggestions on how to do this? Appreciate the help! It's an important aspect and a very resource intensive process that I need to protect.

Your decryption/encryption secret (key) is stored on the server side only and if a client requests a thumbnail, you can validate the URL value by successfully decrypting it with this secret and probably do a simple HTTP URL validation check of the decrypted value. And you have to encrypt all thumbnail URLs with this secret before passing them to the client.

To the extent that you are pulling new stories directly from Firebase on the client side, encrypting (or signing) urls with a symmetric key will not work. This is because you would need to embed the secret in the front-end code which means the secret is no longer secret ;-)

You could also tail the Firebase feed from a server process, generate the relevant images and only serve images for pre-existing urls.

You could your scraper service a unique token that random people won't have so they get 401ed.

Do what https://hackernewsfilter.com/ does, use a Bloom filter. It's simple enough.

Thank you for this idea. This seems like a quite friendly one. So you are using the Firebase API, and each time a website is added you add i to the Bloom filter? This seems it will work for my website, but then the screenshot api can be use only if you previously pushed that url in the bloom filter

Setting up a same-origin policy (https://developer.mozilla.org/en-US/docs/Web/Security/Same-o...) would prevent people from embedding the image in other websites.

You could try generating a unique token for each page-view, and use this as an access-token for all other resources pulled by the page.

The reality is that it's not too many people who will misuse your API... so don't pick a mechanism that will place undue load on your server until you really have to.

Reply on news.ycombinator.com