Hacker News Reader

Comment by sphix0r | Hacker News Reader

We (me and a friend) did something similar for multi city round trips within Europe (https://tripchemy.com).

We created some data structures that allow almost instant searching of such routes and we have scrapers running regularly on Ryanair, easyJet, wizzair, and Transavia. You can query the algorithm here: https://algo.tripchemy.com/routes/TSF?year=2020&month=02&day...

It's based on the following Open source project which my friend made back in the day https://github.com/Whazor/Roundtrip

View on HN

I've always wondered how things like this work; how do you actually get all the routes and price information? Do you need to scrape every route on every possible data individually?

We crawl the fare information ourself from the airlines'websites.

Great website, although I have one minor nitpick.

When searching for flights to or from London it's not clear which airport is which out of LTN, LGW and STN.

Screenshot: https://i.vgy.me/J1F0j1.png

Would it be possible to add the IATA code in brackets (after London) and possibly another item in the dropdown for all London Airports?

Cool site! Curious to know what do you use for scraping, without getting blocked?

Thanks a lot! We use scrapy[1]. Our crawler's behavior is very polite so most of them won't block us. Easyjet for example is very strict against crawlers and will block most of them(and us).

[1] https://scrapy.org/

> Easyjet for example is very strict against crawlers and will block most of them(and us).

It seems that easyJet has outsourced its anti-scraping protection to the company called "Distil Networks"[1].

I wonder how could they block screen scrapers using Chrome on Android with mobile IP addresses.

[1] https://www.youtube.com/watch?v=43wh3beHSBo

Is there a reason this wouldn’t work in the US (obvi the prices would be higher). But hypothetically could you support North America and or South America?

The only thing is that we have the crawl US airlines and support an additional currency(USD). So yes, there is no reason why this shouldn't work for North and South America. Do you think there is a market for this in America?

Skyscanner does support multiple trips, doesn't it?

Reply on news.ycombinator.com