Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by zerop | Hacker News Reader

Full thread

zerop·

Great, congrats on your launch.

1. Does it take care of Bot detection. Most sites will have it.

2. Is this something similar to Firecrawl - https://www.firecrawl.dev/

View on HN

Yes, it has an extensive proxy IP and retry system in place to bypass bot detection.

I’m also trying to gather more feedback to identify the killer feature:

- Adding vectorization to Pinecone out of the box? - Adding multiple integrations like n8n, etc.?

Any crucial pain points to avoid?

Are you concerned about making a product that does this? The legal aspect of accessing a computer system that is intending to block your use seems worrisome.

It is the responsibility of the user. Everyone should be responsible for their own actions. We still allow knives to be sold, and most people use them for good.

Now imagine that knife stabbings became so common that almost everyone started wearing body armor and you start selling body armor defeating knives explicitly. I can honestly see why most people would be upset about that.

I don't see that as a good analogy. There's very limited space for this functionality to be used legitimately / legally - anyone permitted to scrape content is likely able to access the data without the protection measures in the way.

I'm fairly sure circumvention is a (prosecuted!) crime in several countries - curious if you're across that angle, and/or have legal advice/direction you can share?

Reply on news.ycombinator.com