Hacker News Reader

Comment by technion | Hacker News Reader

I've been complaining about this for a while. Everything I try to test with urlscan.io (as a safe sandbox to view a phishing site) has for a while just given me a Cloudflare captcha. Everything I submit to certain automated malware services just come back as "not malicious" despite being clearly malicious, I'm guessing because automation doesn't get past the catpcha.

View on HN

This is going to get 10x as 'fun' once Web Environment Integrity means that malware developers can demand attestation that you're a real victim and not a URL scanner

That is exactly my worry. It's hard enough as it is with VPN / IP detection, headless detection, Captchas, etc. WEI would probably mean game over for all but a handful of Google-sanctioned bots.

I believe they simply need to integrate a captcha provider and write rules for detecting these captchas on various websites.

Many scraping products and the internal scraping systems of companies (especially retailers) implement such mechanisms.

CEO of urlscan.io here: We will be evaluating the use of captcha bypass services soon, we are fully aware of the growing problem of captchas being abused by threat actors to hide their malicious websites from automated analysis tools such as ours.

It's honestly a frustrating problem as we're effectively in an arms race with the likes of Cloudflare, Google et al, and we're in the same boat with less reputable services like scrapers and bots. The recent developments around Web Environment Integrity are concerning to us for the same reasons. Feel free to reach out with any questions or suggestions you might have!

I am not familiar with your product but that won't stop me from suggesting that maybe a Y/N API needs a 3rd answer: N/A.

A captcha might serve as a strong negative signal.

Given the volume of automated link scraping that you do, would it be possible for your company to arrange for a deal with Cloudflare allowing your servers past their Captcha gate, at least around the free-tier Captcha which the scammers most likely use?

This right here is just such a scandal waiting to happen.

Check https://docs.google.com/forms/d/e/1FAIpQLSdqYNuULEypMnp4i5pR... (linked from https://blog.cloudflare.com/friendly-bots/) if you didn’t already.

Disclaimer: I work at CF

So is there a solution to this? Kind of feels like the bad guys win.

The bad guys did win. Cloudflare is used by everything now. They're becoming a single point of failure.

Why do you need a safe sandbox to view a phishing site? Afraid the site's design could hypnotise you into entering your credit card details?

Phishing sites are often also loaded with malware

Afraid the site's design could hypnotise you into double clicking the exe and accepting the browser's, MotW and smartscreen warning dialogs?

More worried about 0-day browser exploits. If you know a site is malicious, why risk it?

And the malware magically gets installed on your computer...how?

Here are 3 CVEs I randomly found from just this year. There are a handful of these every year.

https://nvd.nist.gov/vuln/detail/CVE-2023-3079

https://nvd.nist.gov/vuln/detail/CVE-2023-37450

https://nvd.nist.gov/vuln/detail/CVE-2023-32439

I guess you've never heard of browser exploits?

Didn't think about that, good point

If you don't run adblocker pretty much any website is loaded with malware. New York Times was infecting people with Bahama botnet some years ago. What's stopping other (non)shady websites from doing just the same?

I want to browse the net with only bitmaps being sent to me. I do not want to execute your javascript or markup or make requests other than what i have initiated. Please god someone make a docker container i can spin up on a throwaway computer that renders pages and sends bitmaps of the site to my main computer

Edit: even better, every tab is a separate container instance, i close tab, it nukes the container

To render pages on a throwaway computer and send only bitmaps, connect to the throwaway computer via VNC and run an ordinary browser.

As for improving on tab isolation over what existing browser sandboxes already offer, lightweight virtualization a la Firecracker seems like a more useful increment than containerization, and, assuming each tab has a similar VNC-like connection to its browser VM, virtualization would also make the whole "throwaway computer" setup less necessary to provide a meaningful security improvement.

Sounds a bit like WebRenderingProxy.

Multiples reasons actually.

1. To bypass your corporate proxy (ok, not the best reason);

2. urlscan also allows you to pivot easily and find the same phishing kit as they have a nice database of scans;

3. urlscan also allows you to see the website from different proxies, which can be useful if there is geofencing.

It isn't that simple and threat actors are far from being dumb :)

I would say that you don't absolutely need a service like urlscan.io to look at a single suspicious URL, but running a URL through our sandbox will create a point of time snapshot that you can share with others, compare to existing scans and use as evidence for further actions like takedown.

Disclaimer: I'm the CEO of urlscan.io

the phishing attempt from the url that looks like it comes from your bank can also have exploits to attack your browser, other than getting you to enter your credit card details.

I would be worried about CSRF and other potential exploits..

Reply on news.ycombinator.com