Hacker News Reader

It would be hard to find someone to just give you away that sort of database since everyone collects and store data according to their needs. Looking at your blog posts, it seems that you know programming. Have you tried scraping the data from the respective stores?

View on HN

I know how to program a crawler (did that for the Windows Store), but the Windows Store simply has a sitemap with links to all the apps. I couldn't something similar for the other stores.

Here is a crawler for the Play store (Python): https://github.com/ohwang/scraper And for the App Store, try the following: (C#) https://github.com/geykel/AppStoreScraper (JS) https://github.com/johncch/AppStoreScraperJS This only scrapes the icons though, but I bet you could adapt it to scrape info too.

Thanks! I'm currently trying to write my own scraper for the Apple App Store (C#). Had some code lingering around from the Windows Store crawler that I wrote earlier. I can reuse some parts of it.

I'm not sure if the Play Store lists the entire list of apps. The headers say 'popular paid' and 'popular free', but I'm not sure if these are just the 'most popular apps' or 'all apps'.

Reply on news.ycombinator.com