phsource

Good catch here on the confirmation email link issue! I just fixed that. I've noticed for creating an email subscription, it takes a while for the overlay to close; did it never close for you? I wasn't able to reproduce this, so let me know if you have more details.

I can't stress just how important (and how hard) it is to get a great source of data for airports, so kudos on the work! When building All the Flight Deals (https://alltheflightdeals.com) and BookWithMatrix (https://bookwithmatrix.com), airport data was essential.

It seems like your database covers the basics! However, timezone data and airport type/size were pretty important information for us, so we ended up getting these from:

- OpenFlights [1]: this dataset was great since it had timezone too, which was really helpful for calculating flight lengths, etc.

- OurAirports [2]: no timezone here, but the "type" and "scheduled_service" columns in this dataset are essential. "Type" lets you distinguish between small/medium/large airports, and "scheduled_service" lets you easily filter out airports without real flights (which you often might not care about).

- IATA tzmap [3]: this was used for filling in timezone data and is derived from the Geonames database.

- Random other GitHub Gist [4]: I have no idea where this data comes from, but it was surprisingly complete and has a few golden nuggets like "num_flights" and "runway_length" in addition to "timezone". The presence of a "woeid" suggests Yahoo-related origins, but it's hard to be sure.

Long story short, it'd be AWESOME to have one complete, updated database with all this data in one place, but part of the struggle is definitely having more data than just longitude/latitude.

[1] https://github.com/jpatokal/openflights/

[2] http://ourairports.com/data/

[3] https://raw.githubusercontent.com/hroptatyr/dateutils/tzmaps...; http://download.geonames.org/export/dump/

[4] https://gist.github.com/tdreyno/4278655

This is really cool! Out of curiosity, where do you get the data right now? I've worked on a couple of projects, but always had trouble finding data for low-cost airlines other than just by scraping.

Also, just a few UI things that bugged me:

1. It would be amazing to show the price for each flight, in addition to the combined price.

2. I originally tried London Gatwick, but didn't get good results until I tried London Stansted. It would be great to consolidate cities' multiple airports into one.

3. When linked to the resulting sites, the site opens with the flight origin's country's language (e.g., a flight from Budapest would be in Hungarian), which makes it pretty hard to book. Can you link to everything in EUR and in English, especially if the site's English?

Overall, a great start! I'd totally use this if I just had a week free and were starting out in Europe.

Oh my gosh... don't get me started on NTLM. Chrome Network Tools doesn't expose it at all, so the proxy was a lifesaver in our case.

We've shifted to using https://github.com/joeferner/node-http-mitm-proxy as a part of WrapAPI Proxy (https://wrapapi.com/proxy), which is a zero-install proxy in the style of mitmproxy and Charles.

The node proxy is really great in that it's fully extensible, allows you to generate certificates, and filter/save the kinds of traffic you get to simple JSON structures. We've found it to be a huge boon in development, but it's clearly inspired by mitmproxy (which predates node), so credit where it's due.

For someone on a Javascript stack, I highly recommend combining a requester (e.g., "request" or "axios") with Cheerio, a server-side jQuery clone. Having a familiar, well-known interface for selection helps a lot.

We use this stack at WrapAPI (https://wrapapi.com), which we highly recommend as a tool to turn webpages into APIs. It doesn't completely do all the scraping (you still need to write a script), but it does make turning a HTML page into a JSON structure much easier.

Interesting write-up on the additional work of almost implementing a spreadsheet in the back-end! I'm curious as to what kinds of functions exactly are these.

In the article, you mention that "We chose to compute asynchronously in case computation takes too long or runs into a fatal error" -- coming from Excel, how do you handle errors? And how long exactly do these formulas potentially take? (As someone without a lifesci background, it's hard to wrap my head around the concrete parts)

After reading the API docs, it appears that the FCC operates a mini-imgur/pastebin/file hosting service to help attach files to FCC filings:

https://www.fcc.gov/ecfs/public-api-docs.html#Full-Filing-St...

Unfortunately, these "temporary" file uploads end up accessible from the main FCC domain (i.e. fcc.gov), unlike e.g., Google (e.g., "googleusercontent.com" vs. "google.com"). In Google's case, the separate domain helps distinguish the content as unofficial.

It's understandable why it was originally engineered this way, since it's probably easier to create a subdomain under fcc.gov rather than to get an unrelated domain, but that's why we ended up here!

This looks really cool! Quick question: is there an easy way to export the recorded notes elsewhere? Often, what we try to do for meeting notes is post them in a Google Docs after the fact, and it'd be great to have this as an almost-appendix.

I completely agree. The article's author suggests that the study's result is too good (to be true), and casts that as the sole reason to reject the study. I think that such a surprising result definitely deserves scrutiny, but barring problems with the method or stats used, it's merely surprising, not "impossible".

Within the paper itself, the authors clearly considered this:

  A key aspect for interpreting the association
  between the ordinal position of a case and parole 
  decisions is whether an unobserved factor determines
  case order in such a way that yields the pattern of
  results we obtain.

They then interview judges and lawyers and examine the process to rule these out! Which is rare enough for a quantitative study like this (where they often publish the conclusions and just say "it could be X"). Unless the author can cite a more specific grievance, I'll trust the data.

Speaking as someone who spent a brief stint at a Japanese (but more recently multi-national) car company, I definitely felt that the foreign workers were given more leeway and almost expected to deviate from the traditional norms. My manager, who was a transplant from Britain, was much more willing to leave work right around 5:30, even when there were plenty of other workers (mostly Japanese) still around. It definitely won't be viewed favourably, but on the other hand, there is an attitude of "they're gaijin/not Japanese, so naturally they don't follow all our norms."

This is incredibly detailed; in short, CDNs, cookies/authentication , tons of subdomains, and 3rd-party/user-generated content make it a pain to move onto HTTPS.

I was chatting with a non-engineer friend about why it's hard to estimate how long tasks often take, and this seems like a prime illustration: the dependencies are endless.

I also love the Easter egg:

"The password to our data center is pickles. I didn’t think anyone would read this far and it seemed like a good place to store it."

To be honest, I think email accounts (especially the main ones) have always been a huge target to thieves, since many services still use it as the primary password reset/two-factor authentication method. In that sense, not much has changed.

WrapAPI is in many ways inspired by Kimono for its ease-of-use, but is much more powerful. You can send POST requests and fill out forms, as well as create different types of outputs and chain them. Kimono and import.io, on the other hand, are very much designed for scraping tabular data.

If you're pretty satisfied with import.io and Kimono, it might be worth to just keep using them! Give WrapAPI a shot, though, since it can do a lot of things those tools can't.

  "All types of stalls including clothes,
  counterfeit goods and food stalls will
  be banned from main city roads"
  - Wanlop Suwandee, a chief advisor to
    Bangkok's governor, told AFP.

Based on this statement, it's not nearly as bad as we'd expect, since most of Bangkok's street food are not on the main city roads, but rather on smaller streets. When I visited, vendors most often were situated in side streets coming off the main streets, or near transport hubs like the river ferry stations.

We currently don't deal with it, but when using WrapAPI, you can specify your own proxy servers and we'll route the requests through them.

This was yet another of the features that we added only after getting feedback last time around from HN, so good catch!

Hey everyone! My name is Peter, and I'm the creator of WrapAPI. We recently launched a new version (thanks in large part to the feedback from the HN community from our launch last year [1]), and would love your thoughts.

In short, we've:

1) Improved the Chrome extension to guess what parts of your requests are important.

2) Made a new browser-like "builder" that should be much more intuitive to use compared to the old system.

Let us know what you think!

I think the most open (and perhaps "easiest for users") is still just using email and password. It's the baseline, and unless social features or real-world identity verification is a core part, email/password is available as a signup method.

It's also open (two text strings), decentralized (you can use whatever password manager you want), and simple (people have been using it for ages). So before implementing anything else fancy, I'd highly recommend doing email/password first.

Looking at this list, the professions that Turnbull wants to remove is incredibly broad. As a Canadian in the US, I'm not 100% clear, but it seems like the 457 visa is a lot like the H1-B.

With that comparison, it's really surprising to see professions like:

  * Chemical engineer
  * Petroleum engineer
  * Electronics engineer
  * Web developer

on the list at the bottom.

If someone announced they were taking these professions off the H1-B list in the US, I'm pretty sure there'd be a huge uproar. This is not even to mention that lots of these sound pretty important for Australia's resource economy. Can someone educate me on the situation from a more Australian point of view?

I know this is wishful thinking, but as a fan of the Python Flask (the Python web server) [1], I really hope that somehow this ends up in their hands.

[1] http://flask.pocoo.org/

Good catch! I think you're right; Tesla probably now buys pretty generic cameras (from various suppliers) and then integrates them, so the cost is definitely not that much.

EDIT: Ignore the first paragraph below; @dyarosia mentioned correctly they're working on their own vision system, so they probably buy generic camera units.

(IGNORE THIS: The surround cameras are likely still in collaboration with Mobileye, since in their case, Mobileye doesn't just give the camera hardware but also has a fair bit of software that makes all the cameras come together. It's not too likely Tesla's duplicated this so quickly.)

Ultrasonic sensors though are dirt cheap (in the few dollars range), and they probably come from one of the many tier 1 suppliers (Bosch, Delphi, etc) that makes them. These are the technology that powered those beeping back distance sensors for 10+ years.

The real news here is that again, LiDAR is absent. It seems like Tesla's pretty confident they can get to full autonomous self driving without the point cloud data that LiDAR provides. Now THAT'D be impressive!

Some of these planes are just filled with stamps! Got this one:

http://i.imgur.com/zTN5ERk.jpg

We launched WrapAPI (https://wrapapi.com/) a few weeks ago with the same functionality, but a bit more complex and powerful process to get set up. You can not only specify CSS selectors yourself but define them point and click.

The barrier for starting with JamAPI is impressively low, though! Kudos on the developer-friendly user interface.

Thanks for trying it out this far fjallstrom! Out of curiosity, which part of the process did you get stuck on? We've been trying to work on the UI a lot, so would love the feedback.

This isn't the first time Twitter has bought a pre-launch company for this much, and the answer is probably the same way that Twitter managed to value Vine at $30 million before its launch: http://allthingsd.com/20121009/twitter-buys-vine-a-video-cli...

For any pre-launch company, this would mean looking at the product, the team involved (as in acquihires), and the fit with the parent company.

Edited original, but the code is now available at https://github.com/pxpeterxu/tinderwick

Edit 2: took this down due to Facebook authentication issues. Apologies for sending out a half-baked product, but thanks for all the feedback!

Creator here! I realize that you have to input your Facebook username and password, but this is the only way to make it work so far, since we can't get Tinder app tokens. If we used the regular Facebook API, it would fail the URL test. After the initial authentication, the server only works as passive proxy for traffic to Tinder. If there's another way, let me know, and I'll gladly implement it.

Edit: the source code is available to view at https://github.com/pxpeterxu/tinderwick. It's a bit messy, since this was a quick hack.

Sorry about that; I just fixed this.

This doesn't work with a web application, as we need to request a token directly from the FB API which would check for the URL to match the approved URLs for Tinder.

Creator here! I'm really sorry about having to do this, but this is the only way to make it work so far, since we can't get Tinder app tokens; after the initial authentication, the server only works as passive proxy for traffic to Tinder. If there's another way, let me know, and I'll gladly implement it.