HPouillot

I was a huge fan of Kimono as well. You can't edit an API for now but we will add this feature in the next release.

Thanks for your feedback !

Thanks for your feedback !

Thanks !

You can create an API for any website (news website included) from our Mac/Windows software and you can access this API from anywhere. It runs on our servers and you can query it from any language you want. Let us know if you need more help hello@dashblock.com

Thanks for you feedback ! We talked to the co-founders of Kimono Labs and their approach was a bit different. Our goal is to automate processes on the Internet and scraping is just the first step.

Timing is perfect because to do that, you need a robust headless browser and a smart way to locally identify the elements on the page if you don't want to maintain your scripts. That's why we use Puppeteer and TensorflowJS which didn't exists 2-3 years ago.

But sure, there are website owners who don't want an API for their website. Our plan is not to fight against them but to start with owner that are already convinced that they could benefits from automating the usage of their website. The banking sector understood that, and that's why Yodlee and Plaid are so successful today.

And if you step back, there are tons of websites that don't have the ressources to create an API (30% of the websites have been created using Wordpress) and don't know the value they could generate from it.

So yes, we'll have to overcome a lot of challenges to build this technology and make it accessible to everyone but we are convinced that the Internet will be used more and more programmatically in the future and we are just paving the way for it ;)

Thanks ! 1. It changes the user experience but the underlying model stay the same and will allow our user to record session with inputs and clicks in next releases. 2. Indeed, if you can replay a session you check the data is what you expected. What solution have you tried so far ?

We use machine learning to extract the content of the page, which means when the webpage changes you don't have to update your selection like Octoparse.

You have 10k API calls when you sign up and 1k per month after that. Does it answer your question ?

We did :) It works on all kind of pages. You just have to set it up on one page and it will work on all similar pages of the website. Did you have in mind to train a model to recognise careers pages across websites ?

Ahah, great ! The minimum required version is 10.10 (Yosemite)

If you want to do that on the web, you'll have to render the page in an iframe to select the content and most websites don't allow it. In short, the user experience is way better with a software.

We don't provide an API to manage other API yet, but this inception use-case is interesting. Could you specify what your app would like to do ? We render the Javascript of the page and for now we don't provide a way to specify if you want to render the page or not but we plan on doing so.

We don't use sophisticated methods for now, we just use a serverless architecture, so IP changes at each invocation. Feel free to contact us at hello@dashblock.com if it doesn't work for your use-case :)

It doesn't require Artificial General Intelligence, with enough training data (crowdsourced data and human-edited metadata like JSON-LD or RFD), we can classify automatically the attributes on the page (product name, movie title, creation date, author), structure them and recognise the type of entity.

Feel free to contact us if you want to invest (hello@dashblock.com), we are currently raising funds ;)

Yep, that's our plan :)

Thanks for the advice, it makes a lot of sense !

Thanks Radu !

Indeed, we are rendering the whole page with the javascript, that's why it takes longer than a curl. For now, it's especially useful for dynamic pages but we also plan on supporting pages that don't require rendering.