1, privacy - they would have to send you a copy of someone's video for that. 2, I expect the end user bandwidth is worth more than the CPU time you'd provide, so the credits they could give wouldn't be worth it to you.
1,156 karma · joined June 30, 2009
www.thelowlyprogrammer.com
If this is a serious statement, then let me give you a serious answer: any minor increase in processing power won't inconvenience Google in the slightest. And even if it did, that still wouldn't matter. If it is good for web security and users - which I believe it is - then we're in favour of it.
The consideration here is whether the algorithms are going to do the right thing, forwarding people, pagerank, referers, etc in the same way as before. That's not a question I can answer, but I'd go with whatever Matt Cutts has said.
[1] http://www.thelowlyprogrammer.com/2010/03/writing-efficient-... [2] http://www.thelowlyprogrammer.com/2012/08/primes-part-2-segm...
For comparison, I've used Go to implement a segmented version of the true Sieve of Eratosthenes. It's a standard array-based sieve, chunked up. It takes advantage of goroutines for parallelism, and implements an efficient algorithm at the same time. The code is a lot longer than this one because it tries to do a lot more (memory efficiency, large number support, etc), but it should give you an idea what a full implementation might look like. http://www.thelowlyprogrammer.com/2012/08/primes-part-2-segm...
The product itself appears to be essentially a 'simple' algorithm for Facebook, letting you avoid the pseudo-intelligent EdgeRank algorithm they use and letting you see an unfiltered stream of a specific subset of the content. If it really is that simple, the key questions are whether users will go to a third party to get it and if Facebook will refrain from copying it in-house. Call me skeptical. Taking the points in turn:
1. User Aquisition
> For Hipset, user acquisition is very obviously built into the product’s and team’s DNA. A well executed PR exercise this weekend generated a ton of interest from early adopters - who all signed up via Facebook auth. Meanwhile, the product aggressively (that’s a complement) integrates with Facebook’s Open Graph. Every time i engage with content on Hipset, it gets posted to Facebook. So all those early adopters using the product are actually busy sending notifications to their impressionable friends on Facebook about this cool new service. [...]
To paraphrase the blog, Hipset is better at 'growth hacking' than the blogger's product OneRiot was. It can go viral, and ... and what? It's positioned here as a feature for Facebook, so presumably a set fraction of the users would desire it off the bat. Getting in front of them is a recipe for an initial userbase, but there needs to be something beyond that. What's the long-term growth strategy, continue to hope Facebook doesn't provide a native view for this feed and growing alongside them? I don't buy it.
2. Users want it.
Again paraphrasing, the blog suggests that users want an undiluted feed. One benefit of this is that bands can directly email users who have subscribed. Except...what happens when a few are too chatty? Either they back down from that 'undiluted' idea, becoming simply a different algorithm, or users need to do all that micromanaging Facebook hides away, and get fatigued. There probably is a market for some who want control over everything themselves, but I'm not sure it's on Facebook in the first place.
The author also suggests that this is a perfect market for Mobile - where Hipset isn't yet trying to be. That's a significant pivot, although a reasonable one.
3 - Facebook doesn't need to copy Hipset, because it's complementary.
Facebook probably doesn't need to copy Hipset, no. But they have a vested interest in keeping everyone going to facebook.com, and not splintered into a set of micro-communities. Which means banking on their disinterest in the long run doesn't seem like a safe bet - and if Facebook decides to provide even half the features of Hipset, momentum and the hidden costs of each additional service will do the rest.
Now, there may well be a lot more to Hipset than this blog post is making apparent. I don't know. But there is nothing here to convince me that it's going anywhere, aside from a blip as the internet goes "oh, that's cool!"
Now, I highly doubt the solution space is anywhere near that large. But without knowing more about it we cannot infer anything from the 13/10000 statistic presented.
If the bloom filter says 'yes I've seen it!', and the opposite filter says 'nope, not here', which is accurate? You know it's either a false-positive from the bloom filter or it a false-negative from the opposite filter, but that's the limit of your information. Effectively, you've partitioned your set into three: [definitely not, ambiguous, definitely seen].
If you want to know 100% of the items you've stored with full fidelity, you need to store information for all of them (e.g. in a hash table). Which is a valid thing to do, but requires a lot more memory than the author of this article was willing to use.
http://en.m.wikipedia.org/wiki/DNS_root_zone has some details.
With that in mind, let us examine the flaws in the proposal at hand.
* 1. Lettered roots
This proposal puts the existing DNS root under a lettered virtual root above it, with implicit resolution to the local AIP. The existing DNS root locations are ALREADY indexed by letter, so this is a recipe for confusion. Even more importantly, this system _will not scale_: There are 26 possible letters, if drawing from the ASCII set only, which permanently restricts the number of autonomous zones. What happens then?
This could be resolved by using a unique suffix scheme that does not conflict with the existing or requested TLDs, but would make it that much harder to type an external DNS address. yahoo.com.extdomA for general use would be quite unfortunate.
* 2. Who hands out the AIP designations?
If every AIP must have a single unique designation, there must be an organization handing them out. The ICANN would be the obvious choice, but that brings us back around full circle.
* 3. Ownership conflicts
As rfc2826 points out [2], the internet is built on the assumption that domain names are unique. With multiple implicit zones, either the same entity must be able to control their domain within each or we will end up with conflicts. If yahoo.com resolves to the 'Yahoo' corporate entity in most AIDs, but is controlled by Baidu in one, can they claim it? If not, what about the user confusion that would entail?
Regardless of the answer to this question, I expect in an AID world everyone would start using external domains for the stronger guarantees they provide. So Yahoo would be permanently yahoo.com.A. Which is complicated by...
* 4. Blocking.
If AIPs start blocking resolution of specific external domains, what happens? Obviously China would like this, but for the internet at large, having siloed intranets would likely be a huge problem. Every time someone misconfigures BGP and one region of the internet cannot talk to another, things break. A shifting set of resolvable domains would likely cause exactly the same headaches, only they wouldn't go away with the next BGP update.
* 5. Proxying and scale.
The AIP DNS are required to proxy requests to external domains (3.2 from the draft). Presumably this is to facilitate blocking, but it would also impose significant load issues and key bottlenecks. Note that right now the only equivalent is the root DNS, and it only handles resolution for the TLDs. Something far larger would need to be set up to be able to handle the load of proxying all external requests.
Overall, this proposal has far too many foundational issues to be seriously considered. I am personally happy it was drafted - work to break the One True Root should be done in the open with all relevant parties involved. But this draft isn't going to cut it.
[1] http://en.wikipedia.org/wiki/Alternative_DNS_root [2] http://tools.ietf.org/html/rfc2826 (IAB Technical Comment on the Unique DNS Root)
> The strong consistency properties of F1 and its storage system come at the cost of higher write latencies compared to MySQL. Having successfully migrated a rich customerfacing application suite at the heart of Google’s ad business to F1, with no downtime, we will describe how we restructured schema and applications to largely hide this increased latency from external users. The distributed nature of F1 also allows it to scale easily and to support significantly higher throughput for batch workloads than a traditional RDBMS.
I've seen that before myself at other companies, and it's a shame. A healthy codebase is an investment in the future - if you're not taking the time to cultivate it you're sacrificing long term usability for short term gains. The larger the codebase the more difficult the task, of course, but for us that's just an excuse to solve the next hard problem :).
One more good link on the topic: our use of Clang to find and fix bugs in our existing codebase, as we find new classes of 'gotchas'. http://google-engtools.blogspot.com/2011/05/c-at-google-here...
Yes, in fact. We have some great tools that give us full search over our entire codebase (think Google Code Search), and you can add a dependency on a piece of code without needing to have it on your workstation already. The magic filesystem our build tools use knows where to get it and can do so on demand. Combined with good code location conventions, an overall attitude that promotes reuse over rewrites* and mandatory code reviews where someone can suggest a better approach, we do a pretty good job. Not everything is eliminated, of course, but I'm pretty happy with the state of things.
To your example, we'd use the STL for most of our sorting needs, but if you were to want, say, case-insensitive string sorting, I can tell you where to find it (ASCII, UTF8, or other). If you want a random number, any RNG you could want is available. Most data structures you could name have been written and tested already. Libraries for controlling how your binaries dump core, command line flags are parsed, callbacks are invoked, etc etc are readily available. We really do reuse code as much as possible, and it's wonderful to have ready access to all of this whenever you could ask.
*At a method level, anyways...we're famous for writing ever more file systems ;).
* Single rooted tree. Separated repositories would make it harder to share code, leading to more dupication.
* Build from head. We build everything from source, statically linked. No need to worry about multiple versions of dependencies, no lag between a bug fix and it being available to any and all binaries that need it, whenever they're next updated.
I don't think that an "internal Github" is going be a magic bullet here. It's more likely it would be a matter of trading one set of hard problems for another, as we all of a sudden need to figure out how to do cross-dependencies sanely, deal with multiple versions of libraries, etc, at scale. You are correct that one monolithic Perforce repo is a bit of a pain point, but that doesn't necessarily mean that the right decision is to shatter our codebase into different pieces - we'd rather make our repo scale better. For reference, we've already got hundreds of millions of lines of code, 20+ changes/minute 6 months ago (so what, 30+ now?), and plans for scaling the next 10x are in motion.
If you're interested, I recommend http://google-engtools.blogspot.com/. It details a number of the problems we've run into, and our solutions for dealing with them at scale.
1. China 'borrowing' traffic: http://arstechnica.com/security/news/2010/11/how-china-swall...
2. Myriad undersea cable cuts slowing / segmenting the internet
3. Juniper routers going down: http://www.networkworld.com/news/2011/110711-internet-outage...
4. There was an incident (2 years ago?) where an European ISP broadcast a low-cost route incorrectly and blackholed a large portion of the internet, although I can't find a link right now.
With respect to buffers specifically, AT&T's network issues when the iPhone launched were largely self-inflicted by having overly large buffers: http://blogs.broughturner.com/2009/10/is-att-wireless-data-c...
So yes, widespread issues are certainly possible. The internet is not nearly as resilient as people would like to believe.
The benchmarks linked ran on a 4 core machine with varying amounts of parallelism, generally with Java being more parallelized. When running a server farm, as Facebook does, aggregate CPU time is more important than wall time^, since one can easily scale the number of processes to fit the machine. By CPU time the difference is down a factor of two at least, possibly more.
Be careful with benchmarks. They distill a comparison into a single value for easy use, but it's not going to be the most appropriate metric in all cases.
^So long as wall time is "sufficiently small", which it generally is.
MySpace failed because a better competitor came along. It had a history of predators, spammers, and viruses, and was hard to use well. (Did you see the pages people made?). Ads were one cause among many, and it's doubtful how much they contributed. For comparison, Facebook has ads, but people aren't flocking in droves to G+, and I haven't heard the lack of ads cited as the reason for any of those that are.
Radio stations certainly have ads, and those that claim "x songs without commercials!" usually put self-advertisements in instead, which they count separately. It's still advertising, just a different sort. Overall the whole process is pretty duplicitous (imho).
AdBlock has minimal market share [1], so the prevalence of AdBlock isn't an argument in your favour.
You may be correct that people hate ads, but the evidence you've given does not support your conclusion.
[1] first hit guesstimates 2%: http://www.quora.com/What-is-the-percentage-of-Internet-user...