Hacker News Reader

Comment by calebhwin | Hacker News Reader

The main sort of parallelism we exploit is across distinct websites. For example "find me the cheapest rental" spawning tasks to look at many different websites. There is another level of parallelism that could be exploited within a web site/app. And yes we would have to make our planner rate limit aware for that.

Absolutely agree there are ethical considerations with web browsing AI in general. (And the whole general ongoing shift from using websites to using chatgpt/perplexity)

View on HN

> There is another level of parallelism that could be exploited within a web site/app. And yes we would have to make our planner rate limit aware for that.

People are already deploying tools like Anubis[1] or go-away[2] to cope with the insane load that bots put on their server infrastructure. This is an arms race. In the end, the users lose.

[1]: https://anubis.techaro.lol

[2]: https://git.gammaspectra.live/git/go-away

IMO it depends on how this tech is deployed. One way I see this being extremely useful is for developers to quickly build AI automation for their own sites.

E.g. if I'm the developer of a workforce management app (e.g. https://WhenIWork.com) I could deploy BLAST to quickly provide automation for users of my app.

That's my point. You can use a knife to slice bread or to stab your neighbor. We're seeing an unprecedented amount of stabbings. People are getting away with murder, there's no accountability. Refining the stilettos doesn't help the problem.

I just block every AI bot that doesn’t give me traffic or other benefits. I know I can’t block them all, but it won’t be for a lack of trying.

> In the end, the users lose.

I think it would take more substantiation to claim this. Maybe 10 out of 1000 websites will get closed, but the users will be able to use AI tools to use the remaining 990. Not sure about you, but sounds like a win for users to me.

>Maybe 10 out of 1000 websites will get closed

this may be true for the time being (or not), but will sure change if/when more [websites] become aware of what is going on. The result will be 10 out of 1000 websites will remain open, and not the ones you actually want. The more pressure there is on the sites/servers, the more these will have to act to stay online to begin with.

I find it extremely unlikely that 99% websites will shutdown just because they discover they are being used via AI. I don't think even 1% would shut down without replacement.

Yes, and these are the websites that aren't behind CloudFlare or some other CDN. The holdouts of the open, independent Internet. Sure, let them burn.

Do you have any stats?

A few vocal people on HN barely scratch the surface.

> Absolutely agree there are ethical considerations with web browsing AI in general.

I'm personally not sure there are, but I'm curious to hear what those are for you :)

Maybe more of a legal than ethical consideration but web browsing AI makes scraping trivial. You could use that for surveillance, profiling (get a full picture of a user's whole online life before they even hit Sign Up), cutting egress cost in certain cases. Right now CAPTCHA is actually holding up pretty well against web browsing AI for sites that really want to protect their IP but it will be interesting to see if that devolves into yet another instance of an AI vs AI "arms race".

Reply on news.ycombinator.com