Hacker News Reader

adamseabrookon How Nestlé gets children hooked on sugar in lower-income countries

I think you have to bid on their auctions. It is more of a wholesale thing for people buying tons of it. https://pure-product.com/products/whey-protein-isolate-20kg-... they are reselling one for $709 USD

adamseabrookon How Nestlé gets children hooked on sugar in lower-income countries

Unflavoured whey protein isolate isn't hugely popular but you can get it online.

US: https://nutricost.com/collections/protein/products/nutricost...

AU: https://www.bulknutrients.com.au/products/whey-protein-isola...

edit - I should add, you can get a bit of an idea of the quality of the protein powder by just looking at the percentage of protein in it. Nutricost is 85% and Bulk Nutrients is 87%. Doesn't leave much room to shove other ingredients in with protein levels that high.

You can also go buy 20kg bags of WPI from NZ (which is where I think most of these retail products get their WPI from) https://www.nzmp.com/global/en/products/ingredients/types/pr...

adamseabrookon Uber launches flat fares in San Francisco through subscription

They also have flat fares in Chicago for $3.12 with uberPool, Monday to Friday 6am to 10pm. All trips must begin and end south of Irving Park, north of 71st Street, and east of Western Avenue. https://www.uber.com/info/chi-uberpool-312/

adamseabrookon Ask HN: Stripe or Braintree?

We use pin.net.au for https://meanpath.com as the hosted payment pages they offer allow us to capture payment without needing to do any advanced programming. Their fraud protection is impressive and blocks 90% of the payment attempts we get from people dipping stolen cards.

For https://www.betterteam.com we chose Stripe as our developers are more familiar with the API. Braintree do have some features Stripe does not but the documentation of their API is not as straightforward as Stripe.

adamseabrookon Arrested by Undercover Officers After Trying to Sell His Laptop Online

You can self sponsor with this https://www.border.gov.au/Trav/Visa-1/189- bit of mucking about to get but once you have it you can work for any company. Otherwise company will need a company to sponsor you on https://www.border.gov.au/Trav/Visa-1/457- which is not hard to get but does limit the companies you can work for as not all will know how to do it.

adamseabrookon Parsing 10TB of Metadata, 26M Domain Names and 1.4M SSL Certs for $10 on AWS

https://czds.icann.org/en It has been largely automated now so you can request access to the files with one click vs having to sign and email hundreds of forms. Approval seems to be automatic for most of them.

adamseabrookon Parsing 10TB of Metadata, 26M Domain Names and 1.4M SSL Certs for $10 on AWS

http://meanpath.com/freedirectory.html

Feel free to grab a copy of our domain list. The "All domains with NS records" is the one you want. Has 191 million in it.

adamseabrookon Slack Platform Launch

No need to build your entire startup on Slack but a solid integration can drive serious initial revenue for your startup especially if you do not get a lot of direct traffic. "One click install" is what helped Woocommerce and others get massive growth on the back of Wordpress through top exposure in the plugin directory.

Slack is still very much at the bottom of the growth curve. I have seen electrical contractors who need a way to chat with onsite workers at various projects switch from using WhatsApp/SMS to Slack. If one click job scheduling apps start appearing in the Slack App Store they will be quickly adopted by these businesses. I would be surprised if Slack or something like it has not completely wiped out internal email in 5 years.

adamseabrookon Atlassian's success reminds what a crappy place Australia is for tech business

There is always more the government can do but there are two major government assistance programs I was not aware of till I founded a startup here. https://www.austrade.gov.au/Australian/Export/Export-Grants/... will reimburse 50% of your sales and marketing costs. So you can go spend $100k on Google Adwords and get $50k back effectively meaning you can outbid all your competitors for the same keywords. https://www.ato.gov.au/Business/Research-and-development-tax... is more well known. What is less well known is that if you make a loss in the year you make the R&D claim they will credit the offset into your bank account vs carrying the offset forward to next year. I am not sure what the equivalent of these two grants is in the US but I know Singapore is even more generous with tax holidays and 500%? of R&D spend grants. Edit - forgot to add there are R&D consultants who will even advance you a percentage of your claim amount in advance to help with cashflow so you don't have to wait 12 months to get the money back.

adamseabrookon Ask HN: Who is hiring? (December 2015)

meanpath.com - REMOTE (Any Country) - Full Time - Many Positions

We are a 100% distributed company with staff in Australia, United States and the Philippines. Our tech stack is all Haskell with a very minimal amount of front end that we are hoping to expand with the help of the following new hires.

Currently hiring for:

  * Front end developer

  * Product manager

  * Content writers

  * Designers

meanpath crawls 200 million websites and compiles reports on the technology these sites are using. We currently track over 9,000 different bits of tech which sometimes gets picked up in the press http://techcrunch.com/2013/07/28/bootstrap-3-goes-mobile-fir...

Most of our customers consume this information via our API but we are going to start work on creating a front end so non-technical users can access the information easily. We are entirely bootstrapped and profitable. CTO is based in Ann Arbor, MI and CEO (me) is based in Sydney, Australia. We have a huge team of researchers in Manila who compile and enhance the reports.

Visas?:

We have both US and Australian corporate entities so can sponsor or transfer visas in both countries. If you do not already have a H1B we cannot help with the application process for that as we need to fill these roles now.

Interested?

Email adam@meanpath.com

adamseabrookon Ask HN: Should I deploy my code and lay off 80 people?

Speaking as a hiring manager I always look for achievements like this when parsing a resume or interviewing a candidate. The best candidates always have a few standout achievements that really grab your attention.

If you had to summarise your time at your current employer to just a few dot points this one will immediately get noticed: - In my own time I developed a system that helped semi-automate the work of 80 analysts. Overall cost savings in year one were X and time to delivery reduced to Y.

You could also look to launch a business around what you have built but it sounds like it may be too small of a niche to just drop into any business without extensive customisation.

adamseabrookon Thumbtack Raises $125M at a $1.3B Valuation

I find the business model fascinating and it is amazing to watch them scale. My family business used to spend over $20k per year on YellowPages ads which drove about 8-12 calls per day at a very high close rate. As YellowPages has died off the money has been invested in an online presence but the skill required to design the site, keep it updated, and to manage all the online advertising spend is well beyond a normal person.

The fragmentation impact of having your customers looking for your services in dozens of different ways vs opening the phone book is costly and complex to overcome.

Once you pay a designer, hosting provider, and then pour the rest into Adwords it is very hard to get the same 8-12 calls per day with $20k of annual spend.

Thumbtack are helping solve that problem and syphoning all the money these businesses used to spend on YellowPages or are wasting on poor Adwords bidding strategies. Paying per lead is not that much different from paying $20k to YellowPages who give you no guarantee you will ever get a phone call.

adamseabrookon A Web Crawler with Asyncio Coroutines

meanpath.com can do around 200 million pages per day using 13 fairly average dedicated servers. We only crawl the front page (mile wide, inch deep) so the limiting factor is actually DNS. Looking at the network traffic the bandwidth is split evenly between DNS and HTTP. Google public DNS will quickly rate limit you so you need to use your own resolvers (we use Unbound).

Unlike Blekko we are just capturing the source and dumping it into a DB without doing any analysis. As soon as you start trying to parse anything in the crawl data your hardware requirements go through the roof. parallel with wget or curl is enough to crawl millions of pages per day. I often use http://puf.sourceforge.net/ when I need to do a quick crawl

"puf -nR -Tc 5 -Tl 5 -Td 20 -t 1 -lc 200 -dc 5 -i listofthingstodownload" will easily do 10-20 million pages per day if you are spreading your requests across a lot of hosts.

adamseabrookon Alan Turing's notes found being used as roof insulation at Bletchley Park

http://www.theaustralian.com.au/news/world/turing-papers-tha...

adamseabrookon Kimsufi: the affordable dedicated server

I have used a few of the Kimsufi boxes and had no issues with them. At meanpath.com we use 40 SoYouStart servers and 5 big OVH servers and have never had any issues. Excellent support was provided when we did run into problems with quick hardware swap outs or whole server replacements.

If you want to test out the speed of the network feel free to download some of our publicly released crawl files on http://archive.meanpath.com which is on a 2gbps burstable pipe.

adamseabrookon A competitor emailed me moments after I signed up to Stripe

Based on the fact that he got the email 40 minutes after his query to Stripe I doubt any of these companies were involved.

To do a crawl across a meaningful percentage of the Internet and refresh this every 24 hours is hard enough even if you only capture the front page. Very few sites include stripe.js in the source on the front page. So you would need to crawl deep enough into a site to hit the checkout page and capture stripe.js in the head section which is exponentially more difficult.

Indexing your crawl and passing this off to a team to extract contact details and send it to a client so they can reach out all within 40 minutes I would say is nearly impossible. It is possible they already had his details but I cannot see anything on his site that would trigger a "likely to switch payment gateways" alert.

My guess is this was completely random. A good sales rep will make 60-100 outbound calls per day along with a bunch of emails. Multiply this by the number of outbound sales reps in the payments market and you have a lot of sales activity.

adamseabrookon We Need Viable Search Engine Competition

We (meanpath) released[1] some crawl data which may be of interest to those of you hacking away on a search engine project.

[1] http://blog.meanpath.com/meanpath-january-2014-index/

adamseabrookon 4.2.2.2: The Story Behind a DNS Legend

I believe all the DO droplets default to 8.8.8.8 so Google are probably applying some rate limiting across the DO network block during usage peaks.

adamseabrookon ZMap: Internet scanner maps all of IPv4 in 45 minutes

Running this from any web host will almost certainly get your server turned off along with a pile of abuse emails. To make it run even faster you should scrub all the Bogon routes on this list: https://www.team-cymru.org/Services/Bogons/http.html

I would also scrub all the sinkholes and captured botnet C&C ip addresses as hitting those will lower the reputation your netblock. List we use at meanpath is: http://mirror1.malwaredomains.com/files/domains.txt https://zeustracker.abuse.ch/blocklist.php?download=domainbl... https://zeustracker.abuse.ch/blocklist.php?download=ipblockl... http://malc0de.com/bl/IP_Blacklist.txt http://hosts-file.net/download/hosts.txt http://www.joewein.net/dl/bl/dom-bl-base.txt http://www.dshield.org/feeds/suspiciousdomains_High.txt http://www.malware.com.br/cgi/submit?action=list https://spyeyetracker.abuse.ch/blocklist.php?download=domain... https://spyeyetracker.abuse.ch/blocklist.php?download=ipbloc...

adamseabrookon Twitter Bootstrap Now Powering 1% of The Web

I know it is not quantifiable but the effectively free marketing and goodwill that Twitter get from open sourcing Bootstrap must be worth many millions.

adamseabrookon Twitter Bootstrap Now Powering 1% of The Web

The good thing about Bootstrap is that it is stamped on pretty much every page of a site. Our sample index was mainly front pages so it was easy to spot. Total size of the web is a very hard number to pin down. Verisign quote 252 million base domains worldwide across all TLD but we have some base domains in our index with over 1m sub domains.

adamseabrookon Twitter Bootstrap Now Powering 1% of The Web

I did have "styled by" as the original headline. Bounced it off a few people who were not that sure what I meant so changed it to "powered by".

adamseabrookon 81,186 websites still running vulnerable versions of W3 Total Cache

A quick followup on this post from a month ago https://news.ycombinator.com/item?id=5873730 http://hackertarget.com/running-wordpress-patch-your-plugins...

adamseabrookon OVH Security Incident

It is annoying but the cost of having someone sit there and click renew for 4 hours once a month is still less than the $35k extra we would have to pay to host elsewhere.

adamseabrookon OVH Security Incident

I believe we also had a bunch of servers in that same rack. Downtime was around 5 minutes from memory.

adamseabrookon OVH Security Incident

We tried a number of hosting companies before settling on OVH including the premium dedicated companies. They do have the occasional hiccup but overall we have been very impressed with the pricing and the support both reactive and proactive. The day we signed up for our 50th server we received a nice personal email from the head of their support in Montreal.

adamseabrookon Meanpath - Search the page source, server headers and visible text on 150m sites

Correct. Adidas would be able to see sites talking about their competitor Nike that are not linking to Adidas and also do not contain Nike as part of the domain. This makes sure you screen out all the Nike sub-sites like runwithnike.com which obviously don't link to Adidas.

adamseabrookon Wappalyzer uncovers the technologies used on websites

https://meanpath.com crawls the source and server headers.

adamseabrookon Amazon Drops Price Of EC2 Dedicated Instances By Up To 80%

We use OVH in Montreal.

adamseabrookon Amazon Drops Price Of EC2 Dedicated Instances By Up To 80%

Those servers run our ElasticSearch index so move TB of data around inside the network and out to customers. Our actual crawlers could happily sip from a 5mbps pipe.