Hacker News Reader

Comment by mg | Hacker News Reader

Top level domains by popularity:

    grep -oP '\.[a-z]+(?=,)' current.csv | sort | uniq -c | sort -n

  ...
  15840 .pl
  17914 .it
  20182 .de
  21690 .in
  27812 .ru
  29194 .jp
  30359 .org
  35741 .br
  36675 .net
 406052 .com

.com domains by popularity:

    grep -oP '[a-z0-9-]+\.com(?=,)' current.csv | sort | uniq -c | sort -n

    ...
    365 tistory.com
    370 fc2.com
    408 skipthegames.com
    489 online.com
    515 wordpress.com
    707 uptodown.com
    880 schoology.com
   2570 fandom.com
   2651 instructure.com
   3244 blogspot.com

View on HN

It might be worth updating this comment and explaining your second query.

People seem to think it is somehow measuring visits to those origins. But it's measuring how many unique subdomains are listed for those domains

Rather amazing seeing almost abandoned blogspot.com there at the top.

Also interesting I haven’t heard about half of them. Some are nsfw, apparently.

These aren't sorted by number of visits, but by the number of rows in the list of most visited sites. Essentially which sites have the greatest number of frequently visited subdomains.

Loading the data into the duckdb cli [0] and doing the first query:

    create table current as select * from '202211.csv';
    select * from current;
    ┌────────────────────────────────────┬─────────┐
    │               origin               │  rank   │
    │              varchar               │  int32  │
    ├────────────────────────────────────┼─────────┤
    │ https://hochi.news                 │    1000 │
    │ https://www.xnxx.xxx               │    1000 │
    │ https://www.wordreference.com      │    1000 │
    │ https://finance.naver.com          │    1000 │
    │ https://www.macys.com              │    1000 │
    │ https://www.xv-videos1.com         │    1000 │
    │ https://fr.xhamster.com            │    1000 │
    │ https://poki.com                   │    1000 │
    │ https://salonboard.com             │    1000 │
    │ https://clgt.one                   │    1000 │


    select tld, count(*) 
    from (select reverse(substr(reverse(origin),1, position('.' in reverse(origin))-1)) tld 
            from current) 
    group by tld 
    order by count(*) desc;
    ┌───────────┬──────────────┐
    │    tld    │ count_star() │
    │  varchar  │    int64     │
    ├───────────┼──────────────┤
    │ com       │       406052 │
    │ net       │        36675 │
    │ br        │        35741 │
    │ org       │        30359 │
    │ jp        │        29194 │
    │ ru        │        27812 │
    │ in        │        21690 │
    │ de        │        20182 │
    │ it        │        17914 │
    │ pl        │        15840 │
    │ ·         │            · │
    │ ·         │            · │
    │ ·         │            · │
    │ za:5002   │            1 │
    │ lk:8090   │            1 │
    │ org:1445  │            1 │
    │ co:14443  │            1 │
    │ ar:3016   │            1 │
    │ net:8001  │            1 │
    │ care:9624 │            1 │
    │ au:8443   │            1 │
    │ com:333   │            1 │
    │ edu:9016  │            1 │
    ├───────────┴──────────────┤
    │   2076 rows (20 shown)   │
    └──────────────────────────┘

[0] https://duckdb.org/docs/installation/

Are the bottom entries domain:port? I feel like the table would be more interesting excluding those.

sort -r will reverse the order from most to less popular.

uniq -c is in the pipeline because it counts the number of uniques

Ah, right, missed that.

it's amazing that fandom is number 3 and wikipedia is not even there

Wikipedia uses a .org domain, so it won't show up on "most popular .com domains" lists. (And I think the parent comment is searching for domains with lots of subdomains, which is why providers like Blogspot and Fandom show up.)

Reply on news.ycombinator.com