HNHacker News
TopNewBestAskShowJobs

daave

354 karma · joined November 13, 2011

Aussie in Seattle. Dad. Network/Storage/API geek.

Engineer at Snowflake. Co-founder at Radiopaper, PerfectSchedule. Ex-Google. Ex-Mixpanel. Ex-Stripe.

Views here are my own.

submissionscomments
daave··on UPSat – the first open source hardware and software satellite is delivered
The menu of the linked post has an entry named 'Source' which points to:

https://github.com/librespacefoundation?query=upsat

daave··on A SpaceX Falcon 9 rocket has exploded at Cape Canaveral
How close is the shelter to the tower? That's gotta be terrifying. I'd have thought people would need to be pretty far away just to avoid permanent hearing damage in an event like this.
daave··on Verizon: We Can't Become Dumb Pipes
> The reason why Google didn't offer cloud services earlier is because they were using up all of their cloud. [citation needed]

It was a (poor) business decision not to go for public cloud earlier, but not driven by datacenter capacity.

daave··on Ask HN: What are your favorite board games?
Diplomacy is wonderful. I find it hard to get 7 people to commit to 4+ hours to make it through a game, so the mail option is good, but IMO in person is more fun.
daave··on HTTPS crypto’s days are numbered. Here’s how Google wants to save it
So you're assuming there are compromised CA keys in the hands of governments & corrupt corporations?

Even if this is true, isn't it mitigated by certificate pinning?

daave··on Show HN: OzBonds, do-it-yourself inflation protected pension
Looks like Safari doesn't provide the Intl object (ECMAScript Internationalization API), which I depend on in my module initialization.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

daave··on Show HN: OzBonds, do-it-yourself inflation protected pension
OP here.

A couple of small known issues I haven't fixed up yet:

  * I'm not correctly taking into account the number of coupon payments in the current year and final year for each bond, just assuming it's 4 like every other year.
  * Prices are updated manually, I've got some code that scrapes the ASX to get new prices, just needs to be hooked up.
I'd be particularly interested in feedback on usability, or ease of understanding from non-finance experts. I've added a bunch of long-winded copy that tries to explain what this is for, but I'm not sure it gets the point across clearly.

Once this is a bit more polished, I'll publish a version that does the same calculations with US TIPS instead of Australian eTIBs.

Thanks!

daave··on Apply HN: Hackinator – Allow hackers to invest in YC startups using code :-)
Not YC-specific, but this sounds a fair bit like OfferBoard: http://www.offerboard.com/
daave··on Notes on Google's Site Reliability Engineering Book
In my interviews you code in whichever language you prefer. Some interviewers will ask you to use a specific language that's mentioned on your resume. In general I think that if you show strong coding skills in some language, it is believed that you won't have much trouble teaching yourself the languages your team uses (typically some subset of C++, Java, Python, Go, Borgmon).
daave··on Notes on Google's Site Reliability Engineering Book
I'm an SRE-SE and regular do phone interviews for SRE-SE candidates.

While I do tend to spend more of the interview time talking about sysadmin tools, operating systems, networking, databases, security and troubleshooting, I still expect candidates to have reasonably good coding chops.

The difference is that the coding questions tend to be more task-oriented or procedural (i.e. log processing, building automation pipelines, implementing standard unix cli tools, etc.), rather than the algorithmically challenging or math-oriented problems that we'd usually ask SWE candidates.

Both the SE and SWE side SRE candidates need to be able to design and reason about large systems, making trade-offs between performance (especially latency), redundancy and cost.

daave··on Moving on from Picasa
Unlike Google Reader, this product was replaced with a newer (and, IMO, better) alternative, Google Photos. Albums you had in Picasa aren't gone, they're just accessible through the new UI. Why should Google maintain two competing UIs for the same product?

[these opinions are my own, not necessarily those of my employer, Google]

daave··on Major Linux Problems on the Desktop, 2016 Edition
The site is now blocked by Google Safe Browsing as a result of these complains. Author has been emailed asking that he corrects this behavior in order to be removed from the blacklist.
daave··on Major Linux Problems on the Desktop, 2016 Edition
The site is now blacklisted for Chrome users. I've emailed the author letting him know about this issue.
daave··on Major Linux Problems on the Desktop, 2016 Edition
When visiting this blog post on my Android phone, the ads (or something) redirect me to a page with popups telling me my phone has a virus and encouraging me to install an app to 'clean' it, using Google branding on some dodgy domain. I seem to get a different one each time.

Reported it at https://www.google.com/safebrowsing/report_phish/ but mentioning here so that others don't fall victim to it.

daave··on Show HN: YouTransfer – Self-hosted file sharing
Sounds fairly similar to bradfitz's Camlistore (https://camlistore.org/).

If you're familiar with that project, could you comment on the main differences in YouTransfer?

daave··on Cookies Lack Integrity: Real-World Implications
> I can just set my cookie to $their_session_id:$hmac_of_their_session_id

If you can steal somebody else's cookies (which are not Channel-bound) then that's true. If you can only steal or predict somebody else's session ID's, the HMAC provides protection.

It's not atypical for session IDs to be simple counters that get incremented for each new session. If your session ID is 100042, it's a pretty good bet that 100041 and 100043 are valid session IDs as well, and without HMAC, a user could take over these sessions trivially.

The even better mitigation to cookie theft, which I also mention, is TLS ChannelID. ChannelID creates a unique private/public keypair for each new TLS connection, and sends the public part along in the TLS handshake. Then, when you resume sessions from the same machine, you can prove that you have the private part and the server can accept your existing cookies. With this approach, cookies are no longer bearer tokens and stealing cookies becomes worthless.

This can be hardened even against local malware running as the same principal as the user doing the browsing if the browser's ChannelID implementation generates and stores the private key inside a TPM or HSM.

daave··on Cookies Lack Integrity: Real-World Implications
Exactly. A better, or at least more general, mitigation than enabling HSTS (though that's a good idea anyway), is to not design your web application in such a way that a modified cookie in the client creates a vulnerability. Since cookies are stored in the client, they are always going to be susceptible to malware on the user's machine. So, trusting that the contents of a cookie were authored by the service receiving them is a bad idea in general. Cookies should be stored along with some additional information that verifies their authorship.

A relatively simple way to accomplish this is to have your application include an HMAC in the cookie contents, and verify it whenever the cookie is received. e.g, if you are storing $session_id in a cookie, change your cookie contents to be "$session_id:$hmac_of_session_id", and verify the HMAC every time a cookie is presented.

Now a user, or malware, or a MITM, is not in the position to take over or modify a different user's session simply by altering the cookie, since they will not be able to produce a valid HMAC (the key is never shared with the user).

If even storing the key in your web frontends is too risky, you could use RSA or DSA signatures, only store the public key in the web frontend that verifies cookies, and store the private key in a more hardened cookie-signing service that isn't directly exposed to external networks. This service can be invoked when new sessions are created or upon user login, if applicable.

On top of this, if the client supports ChannelID, you should include the ChannelID in the message that is HMAC'd, so that stolen cookies cannot be reused on other machines.

daave··on A dataset of every Reddit comment
> Since the full dataset is ~285GB, you only get 4 queries per month.

That's only true if your 4 queries need to read every single column.

One of the big advantages of BigQuery's column-oriented storage is that you only pay to read the columns that are actually needed to answer your query.

For example, this query to extract the top 10 authors only cost me 19GB to run (and took 7.0s):

  SELECT
    author,
    COUNT(*) AS COUNT
  FROM
    TABLE_QUERY([fh-bigquery:reddit_comments], "table_id CONTAINS '20' AND LENGTH(table_id)<8")
  GROUP EACH BY
    author
  ORDER BY
    COUNT DESC
  LIMIT
    10;

  author,COUNT
  [deleted],228425822
  AutoModerator,3677774
  conspirobot,575576
  ModerationLog,547671
  autowikibot,402076
  PoliticBot,388395
  imgurtranscriber,360248
  dogetipbot,358093
  qkme_transcriber,301968
  TweetPoster,293309
daave··on Why Spotify Pays So Little
I don't think sampling's even necessary. Presumably Spotify log every play. If they pushed their logs into BigQuery or something similar, it would be trivial to calculate the revenue breakdown the way the OP describes. 'Big data' is here, and it works. With 100 million users at 1000 tracks per month, we're 'only' talking about 100 billion or so rows to process each month.
daave··on Disqus: It's Still About Realtime, But Go Demolishes Python
Aside: for SFTP we use (and like) https://github.com/minusnine/gosftp/
daave··on Save more with Google Drive
> I wonder if Dropbox have plans to start utilizing their own data center to get costs down...

Mid last year a recruiter from Dropbox told me they were looking into this, and wanted to make it happen in 2014.

daave··on Save more with Google Drive
3x redundancy is very cost ineffective. See this 2010 Google presentation (slide 13) on the 'Colossus' File System, which uses Reed-Solomon encodings.

http://static.googleusercontent.com/media/research.google.co...

There's really a need for a good open-source equivalent to Colossus, which would allow startups like the hypothetical one you describe to do storage must more cheaply and reliably. HDFS (with HDSF-RAID) is probably the closest.. but it's still a long way off.

I'd be interested in what other assumptions you're using to come up with the $5/T profit margin. Are you considering the cost of ingress and egress bandwidth, cooling, personnel to enact repairs? With a large enough fleet you'll have many drives/machines die every day. What about a team of engineers in an on-call rotation to keep to respond when (inevitable) failures occur? Just pointing out that drives + power is only part of the total cost of running such a service.

daave··on Identify with email: no need for passwords
> It would tremendously simplify for services. They now need not implement any cryptography.

Uhh.. I sure hope the cookie tokens are not stored directly on the server-side, but rather verified against a (cryptographic) hash of same. These credentials are as valuable to an attacker as a password (albeit relatively short-lived).

daave··on Dropbox disables ASLR on Windows
My 'source' was merely hearsay :)

It seems there's a wikipedia article on the dispute: http://en.wikipedia.org/wiki/Fewer_vs._less

Which mentions the 'rule' started to get pushed around in the 1800s, though 'Less has always been used in English with counting nouns.'

I suppose we'll always be stuck with this ambiguity in English since there's no formal authority to tell us how we should speak/write.

From my own observations, having grown up in Australia and since moved to the US, I must say the common practice is notably different, I almost never heard anyone say fewer in Australia, but have noticed that many Americans (like the poster here) seem to be on a crusade to ensure less isn't 'misused'.

daave··on Dropbox disables ASLR on Windows
Hardly sounds definitive. From your link:

"There is a debate about whether the word should be 'less' or 'fewer'," a campaign spokesman said. "Saying 'up to ten items' is easy to understand and avoids any debate."

and

A Tesco spokesman said: "The debate about what is right has been going on for years now, and I still don't think we know if 'less' or 'fewer' is correct."

daave··on Dropbox disables ASLR on Windows
That's a rather US-centric remark.

'less' is an acceptable comparative for both countable and uncountable nouns in all modern dialects of English except for US-English, where it's strictly expected that fewer be used for countables.

Perhaps caiob is from the UK, Australia, India, South Africa, New Zealand...

daave··on Ask HN Googlers: would you invite a fellow geek to visit the Googleplex?
Interesting. I've never been to MTV, but as a Googler who frequently volunteers to give tours for school/university groups in the NYC office, I'm thrilled by how wowed the visitors are at our office space.

Perhaps we Googlers start to take for granted some of the things that really are quite extraordinary - like the 150ft to food policy, the pieces of historical computing machinery that are kept around, the guest chefs that visit, the artworks that have been commissioned, the fact that we hold regular 'espresso office hours' and occasional mixology classes, and even the corporate essentials that Google really pulls off, like the Tech Stops, phone rooms, visitor badging system and video conferencing setup. Even though it's 'just an office space', it isn't like any other, and getting to see it as a potential future employee can be pretty motivational.

daave··on Google blocks Chromecast app that let you stream your own videos
How is making internal changes to an unpublished API evil?
daave··on Ask HN: Google Chrome heuristic warnings pose threat to our business
I believe they actually whitelisted/disabled the part of the classifier that is affecting your site(s) more-generally, so it should be working across all domains. That said I know very little about the internals of that system.

I'm sure they will contact you via regular customer support channels (ie. not HN) if they need any more info to debug the problem.

daave··on Ask HN: Google Chrome heuristic warnings pose threat to our business
I work at Google but not on this product.. so I escalated your issue to the team that works on the anti-phishing classifier. They're looking into it now, and put you on a temporary whitelist in the mean time (should take effect within 30 mins).
← PreviousPage 2 of 4Next →