The quote used is from excellent article here: https://bgr.com/tech/the-internet-isnt-free-it-never-was-and...
255 karma · joined March 18, 2022
- I don't want my content on those sites in any form and I don't want my content to feed their algorithms. So I do not care for opengraph or previews.
- Using robot.txt assumes they will 'obey' it. But they may choose not to. Its not mandatory in anyway.
- Yes, they can fake UA. This does not mean I should not take any measures to block them just becasue they can fake.
Using Nginx to block Meta, Twitter and ChatGPT access to your sites - https://news.ycombinator.com/item?id=36476243