Comment by mirimir | Hacker News Reader

Comment by mirimir | Hacker News Reader

Parent Full thread

Yeah. It's a lot of stuff to cover. I should maybe have put all the details in GitHub, and just presented the key results.

I did scrape ping.pe using a shell script with numerous lines like this:

> google-chrome --headless --run-all-compositor-stages-before-draw --virtual-time-budget=25000 --print-to-pdf='[IPv4-date].pdf' http://ping.pe/[IPv4] && sleep 4

Also, if I hit it too long, it started providing bullshit results. So I ended up doing it from Whonix, so each request came from a different IPv4. And I did try to contact them to arrange payment. But got no reply.

I did read about RIPE ATLAS. But it seemed necessary to host a probe. And then get credits to use other probes. But maybe I misunderstood. So is there an API?

I didn't do any systematic cross-verification between ping services. For each VPN server IPv4, I just got the minimum minrtt value, for all measurements from all ping services. However, I did verify that ping probes generally yielded plausible minrtts for VPN server IPv4 that didn't show implausible minrtt with any ping probe.[0 vs 1]

0) https://cdn-resprivacy.pressidium.com/wp-content/uploads/201...

1) https://cdn-resprivacy.pressidium.com/wp-content/uploads/201...

Yeah, I sort of assumed anyone interested in this sort of thing would either already be hosting a probe, or find it obvious that they might as well do so. I've had one lurking in my basement for a little over a year now, and it's racked up several million credits which I'm happy to transfer to people doing interesting research cuz I'm never gonna use 'em myself. PM me. You can't spend more than a million a day without special permission...

I read the commercial-use permission differently — I think it's granted by default if you give credit and link to them as per these terms. IANAL but do your own reading: https://atlas.ripe.net/get-involved/commercial-use/

The ATLAS toolset is here, including APIs: https://atlas.ripe.net/measurements-and-tools/tools/

Speaking of Github though, if you're looking to take this even further, sharing your scraping and analysis scripts might find even more collaborators and ideas. But I understand if that's not permissible under your work-for-hire agreement or whatever.

Thank you very much for offering probe credits. I don't know how to PM on HN, however. Perhaps you could email me? The address and Keybase URL are in my profile.

Rereading the commercial-use terms, I get that the rules for default permission all work for me.

I got approval to share the data. So I'll do a GitHub repository with that and complete explanation of how I collected and analyzed the data. Before doing more of this, I must get a MySQL instance up, and learn enough R and Python to do the final analysis and charting.

Hosting a RIPE Atlas probe means you will generate credits, which you can then spend on measurements. All Atlas measurements require those credits as a form of payment. In short: there is no free API.

You can also receive credits as a donation, e.g. by someone who hosts a probe.

This is an interesting article and I’m sure someone with a constant supply of credits would surely donate a few of them, for you to continue this research.

Thanks. That's what I thought.

I gather that there are lots of RIPE Atlas probes. So yes, it'd be very cool if someone would donate credits. However, there is the fact that commercial use requires approval from RIPE. And that would likely include work done for hire.

Reply on news.ycombinator.com