Hacker News Reader

Comment by nopassrecover | Hacker News Reader

They are publishing them - it has bitten us (e.g. expired one click links for customers ending up on Bing from their emails)

View on HN

I think this is where we use meta tags.

All pages with one click links should have no index follow or no index no follow. Your seo consultant (if you have one) should have advised you on this.

I am not saying this excuses the privacy violation but just suggesting there are things we can do...

Bing appears to be ignoring those headers for links crawled from emails

Worse would be links that are private to the people who posses the url. Like a private video on YouTube or a private document in google docs. The security depends on the URL being secret. This would silently publish secret information.

If those pages have no proper meta tags or robots.txt, there’s absolutely nothing wrong with this. Security by obscurity was never a good approach; from Proxies to security scanners, there has always been software that crawls unassuming URLs and published the results somewhere, if only a report to the admin.

robots.txt disallow is ignored for my production site at least. This is super bad.

Same for us - we have robots.txt disallow etc. and the relevant headers for personal customer links and Bing is ignoring and publishing all the same

If you can say for certain that the links being published are coming from the malware scanning, and not being taken from users' browser sessions that are using Microsoft Edge you should elaborate on this.

That would be even worse.

Nobody is saying it isn't.

It's about trying to get to the core of the issue, not just the random speculation going on in the article and in this comment thread.

I would be pretty mortified if browsers were using user browser sessions to scan content and pass it to bingbot…? What about if you’re browsing something local? Or your bank account?

I would be too.

The point I was making is that someone should research this instead of relying on wild speculation as the basis for the conversation.

Reply on news.ycombinator.com