Hacker News Reader

Comment by enjayz | Hacker News Reader

I'm a co-founder of Mixnode and we want to "make web-scale data affordable for everyone".

Having access to trillions of data points from the web is a super power reserved for only a handful of companies; what if anyone could access and analyze these data points without having to hire 25 experts in distributed systems and raise VC... Imagine the explosion of innovation this will bring about.

We want that explosion to happen sooner rather than later.

If you find this mission intriguing please send me an email: nj@mixnode.com

View on HN

I worked for 2 years on the crawl infrastructure team of a well-known SEO/analytics company that was pulling in over a 120 billion web pages a month. It was definitely one of the most difficult projects I've ever worked on and we did have a team of 6-7 very incredibly smart people -- not "25 experts in distributed systems" though :P

This is a very lofty goal and I'm not sure how you are going to tackle it with 3 people, however I'm rooting for you and would love to get my invitation soon. Best of luck.

> pulling in over a 120 billion web pages a month

Can you tell more? That's 45K pages per second, and assuming that each page load on average takes 1 second you already need 45K workers. Are you talking requests or really loading web pages and evaluating them?

I find your mission interesting, but I was turned off by the fact that you're a Canadian company with ~3 people. I wish I could set a reminder to ping you in like 3 or 4 years. :( Best of luck!

What's wrong with it being a Canadian company?

I live in the US, specifically the Bay Area. Based on past experience, it’s unlikely Mixnode could pay me a salary anywhere close to what I could earn here, even after factoring in cost of living. Emigrating is also not practical, even if getting a visa wasn’t a problem.

I live in the East Bay. Here is a cost comparison between Oakland, CA and vancouy, BC: https://www.numbeo.com/cost-of-living/compare_cities.jsp?cou...

Oakland costs 15% more, but it’s unlikely this company is going to pay me ~85% of what I make here if I emigrated (and, even if they do, I have a significant amount of student loans as fixed costs). As a remote employee, I’d need ~100% of what I’d make around here, and that would come with significant tax headaches for them.

As a Canadian who moved to the Bay area, this is true. Finding a Canadian company that is going to be competitive with Bay area salaries is almost impossible even if their revenue is in USD.

Even large US companies generally require internal transfer to Canada to take a pay cut in addition to changing to Canadian dollars.

To make it worse, Vancouver has a high cost of living but low salaries.

If you are remote, you don't need to be in the bay area. While a harsh reality, the bay area is a crazy expensive place to live, and I would say do so if you have to, not because you want to. And if you do want to, work where it makes sense.

Actually, I do need to be in the Bay Area for personal, professional, and health reasons. But, yes, I take your point. I could certainly live on 3/4 or 2/3 of a Bay Area salary in the middle of nowhere, if I wanted to.

You could live on 1/4 of a Bay area salary.

Not with my student loans.

Reply on news.ycombinator.com