Hacker News Reader

Comment by paulirish | Hacker News Reader

We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU...

The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.

View on HN

Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry

A few hours later: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...

After his announcement, Vitaly answered to the team: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQ...

> >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work.

> We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime.

> But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!

Also see https://github.com/Vitallium/phantomium

It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken

That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.

Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?

Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks.

We laypeople can only find out things via word of mouth or observational tests on assumptions.

> The DevTools Protocol is the primary API for headless Chrome

Why not the WebDriver protocol? That seems to be exactly what it's intended for…

WebDriver protocol has limited APIs. For example, no networking status. (WebDriver rejected to implement it, they told you to use Proxy for debugging)

Reply on news.ycombinator.com