Indeed, amazon has different layouts and can be tricky. For now, our model is resilient to minor changes but we are working on improving it - amazon.com.au look like a good test ;-)
There is plenty of differences among which 1/ we don't rely on classic selectors (CSS, xPath, etc) which allows us to be resilient to website updates 2/ we offer a simple UI that automates data selection and structuration and 3/ we are available on Windows and MacOS =)
Good question.
However, that would require websites' users to validate a Captcha every time they navigate it, which is not optimal in terms of user experience.
We don't crawl the websites yet.
However, you can create an API on a given webpage and gather data from similar webpages on the same website by calling the API with the new URL.
UiPath and others RPA actors allow their users to mainly automate local processes on Windows.
We are cross-platform and focusing on the web, which makes us useful for other use-cases like gathering data or automating navigation sessions!
That's a good point.
Our app is validated by Apple on MacOS and the Windows version will be soon. Also, we have thousands of users and you can google us : no claim for spam at all =)
Note : FyI, we worked on a SaaS version but the user experience was not slick enough in our point of view (e.g. iframe of the websites).
Yes! we have been quite occupied since the end of YC but we plan on releasing it soon =)
Please ping us at hello@dashblock.com and we will inform you when the version is live!
Agreed, some websites are really reticent about scraping.
But let's think about Google, they are scraping the whole web regardless of the ToS of the websites, so it all boils down to one question : do you create value for the website owner ?
That's why we want to focus on use-cases where we create value for both, our users and the website owner. If you think about Yodlee/Plaid in the banking sector, they built partnerships with the Banks but continued scraping them because most of them didn't provide an API.
Correct! Also, Diffbot extracts automatically generic entities (e.g. product name and price, comments, etc) while we let our users choose exactly the data they want on any webpage =)
Thank you! Kimono Labs built an amazing product and has been finally acquired by Palantir. The main difference between us comes from our machine learning model that gives stability to extraction over time and parse webpages in a generic way. We are also working on automating navigation, which is something they didn't do =)