EricBurnett

Depends on selected compressor, but yes, you can. I've definitely observed zstd-1 to be a net savings, where compression/decompression costs were offset by pushing fewer bytes through the RPC and network layers - and this was only from observing the endpoints, not even counting CPU on intermediate proxies/firewalls/etc.

I wouldn't normally expect gzip to be a net savings (it's comparatively more expensive), but depending on compression ratio achieved and what layers you're passing the bytes through, I'd definitely believe it can be in some contexts.

Related tip: anywhere you're looking to deploy compression in a backend, consider zstd (https://facebook.github.io/zstd/). I've found it suitable for streaming compression (Gbps+ flows without exorbitant CPU usage), storage compression (high compression ratios at higher levels; fast decompression), and found implementations available to every language I've needed. It comes out strictly better than gzip ~across the board in my experience, and should be the default compressor to choose or start evaluations from if you have no other constraints.

I don't think it's yet deployed in browsers, so I'm restricting my recommendation to tools and backends. IIRC brotli is worth considering for browsers, but I haven't deployed it myself.

> 5) Forge -- basically Blaze's remote build execution (what Bazel calls RBEs). Almost every build at Google is extremely parallelized, and if you need to run flake tests, running a suite of tests 10,000 times is almost as fast as running it once.

This is available outside Google now - start at https://github.com/bazelbuild/remote-apis or https://docs.bazel.build/versions/master/remote-execution.ht... . It's a standardized API, supported by a growing set of build tools (notably Bazel, Pants, Please) with a variety of OSS and Commercial implementations. At this point almost anyone can set up Remote Execution if they wish to, and remote caching is even easier.

Minor terminology correction: RBE generally refers to Google's own implementation of the same name; Remote Execution (RE) and the REAPI are used to refer to the generic concept.

(Disclaimer: I work on this at Google.)

HTTP/1.1 is not being retired though, so I'm not sure I follow the argument about any forced switching cost for small companies. (Which I attempted to cover in my 100k estimate anyways, though you're free to dispute it). Off-the-shelf servers will implement it as an option if you want to turn it on, some clients will support it, and for the rest, status quo should hold? Not for some - e.g. firewall manufactures will presumably have to care - but the vast majority I don't really see being negatively affected by the existence of this standard.

Speed increases will presumably be consumed by continued bloat on part of publishers, as has historically happened in all tech innovations. Which might argue that it doesn't matter regardless and no optimization is ever useful, but that's both defeatist and ignores companies who do care (e.g. CloudFlare), for whom 4% probably matters now and will matter still in a decade.

(Disclaimer: Googler on build tools; Canadian)

Tiered how? By IP? With most content served by a CDN or DDOS shield already, that's arguably fine; most meaningful on the customer IP where you'd want fairness across customers anyways. Or do you mean internet providers will privilege their own content? Already happening, to the extent allowed by law (I'm looking at you, FCC). Not likely to change with HTTP/3 - empirically, needs to be addressed by regulation and not technology.

(Disclaimer: Googler on build tools).

I hear this argument a lot, and fail to be convinced. Optimizations at scale matter a lot. Let's posit that 100,000 developer-years have been spent implementing HTTP/2 and tools to interact with it. (Hopefully a great over-estimate). Let's say we have to spend all that again to move to HTTP/3. Is it worth it?

Well, per random internet stats I haven't validated, there are about 4b internet users, spending about 3h per day on the internet apiece. I could imagine 5% of that is spent on synchronous resource loading. That's about 10m/day/user - or about 28M years/year spent waiting on resources to load. A 1% savings (the minimum of CloudFlare's range) is 280k human years / year. And that's the bottom end - could be as high as 4% (1.1M years/ year), and without BBR (negatively impacting high packet loss and high throughput connections in particular).

Is "persons average time on the internet" comparable to "paid developer time"? No. A lot of that user time is timeboxed "wasted" time, in the endless content scroll. With that in mind, is it worth it overall? Probably. I've tried to estimate in favor of developers, but I still get their one-time effort paid back 2.8-11x _each year_ in user time, under unfavorable conditions. And this is growing rapidly, both in user count and time per user.

And that says nothing about the privacy benefits of encrypting headers and whatnot, which you should probably ascribe nonzero value.

Why is this more plausible? You don't know her, and are applying a psychological condition to her with no basis whatsoever. Just because you don't want to believe it's true, I assume?

My wife has something similar. Caught it in late February - at the same time as me - and _still_ has debilitating coughing fits some days, and no energy to move some days. Unless you're also going to tell me that's in her head too, and mine.

This all rings true to me. I summarize senior engineer responsibilities as "think holistically" - cost of development, cost of maintenance, path to adoption, upfront investment required before you can determine if you'll succeed/fail, etc all need to be factored in. In many respects, it's the art of solving the right problems (and then solving them well).

If you're curious, my own toolbox (Senior Staff SWE at Google):

* Understand your problem space in all dimensions, including prior art internally and externally, failed attempts in the space, tradeoffs and constraints. Takes a year or two at minimum with heavy research and detective work, so start with a small seed area and grow, and figure out whose opinions to lean on in the meantime.

* Inventory your teams and their projects. Get a feel for the problem each is solving, whether you believe it's possible, and the staffing needed for that. Look for:

---* Succeeding teams. Observe from a distance.

---* Under-valued teams. Help them articulate true value to leadership. If they disappeared, what would the cost be? Including externalized costs elsewhere in the company.

---* Under-staffed teams (usually also under-valued). Teach them to deprecate features and cut problem scope to reach a sustainable subset - the worst thing is growing tech-debt with no path out. If leadership doesn't like the cuts, they can help prioritize differently, or invest. But do not allow them to request the impossible or unsustainable.

---* Under-executing teams. Remain skeptical you fully understand the problem (is it harder than you think?), and look closely for education gaps you can solve in-place with mentoring. But it may ultimately be a staffing problem - e.g. wrong lead for the job - or a team culture problem, and those must be addressed too.

---* Over-staffed teams, or teams solving unimportant problems. Flag it to management - generally either their scope will be grown, or engineers can be pulled off to work on higher-impact projects.

* Do the math. Understand costs and opportunities (what areas are overly expensive), growth rates, usage metrics, reliability. Develop a gut-feel you can use to sanity-check what you see and are told.

* Look ahead - keyword "unsustainable". Where are costs growing too quick? Scaling approaching unsupportable limits? Toil growing linearly with usage? Users taking features in an unexpected direction? Architectures breaking down? Work with teams to have a plan, and escalate the risks upwards for visibility.

* Predict the future. Based on all this, what problems should be solved but aren't? What leverage points do you have to change the model, and solve a different problem more successfully? Pitch, sketch, and prototype to sell the importance or opportunity.

* Roll up your sleeves - "architecture astronaut" is a pejorative for a reason. Dive in to your struggling projects and work alongside the team, providing an extra pair of hands and educating through demonstration. Get involved in outages and their resolution. Build tools and dashboards. Take responsibility for success in at-risk areas and do whatever is required, whether that's writing code, doing analysis, PMing, or simply making yourself available.

* Be a diplomat. Recognize that all success is a team effort, and your path to impact is through and alongside others.

The brain is always a fun comparison. It's this crazy super-scalar architecture that can do thousands of trillions of primitive operations per second (petaflops), yet with pretty terrible latencies: propagation through synapses on the order of single digit milliseconds, individual neurons only fire order of 1 time per second, input processing latencies in the hundreds of milliseconds, and primitive "logical" algorithms like "how many numbers can you count in a second?" netting out to "less than 10".

So we end up in this state where what the brain does well computers are still (comparatively) terrible at, and what computers do well the brain is (comparatively) terrible at. We're slowly bridging that divide, with e.g. TPUs focussing on larger volumes of low precision operations happening in parallel, but we've got a long way to go yet.

It's worth re-reading Buffett's 2019 annual letter (https://www.berkshirehathaway.com/letters/2019ltr.pdf ; first page especially) in this context, as he was already warning against over-reading oscillations in the value of their owned assets.

Now, it would be certainly interesting to know if Berkshire Hathaway would deem this movement to be "other than temporary", and to look at the magnitude of their writedown if so. But remember that they take a very long view - if anything, we might see them to start deploying assets soon to buy up more temporarily-devalued-but-financially-sound businesses to add to their portfolio.

I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.

The differences between bazel and BuildXL are less about platform and more about philosophy. Bazel is about opinionated builds - declare everything, and you get fast correct builds. Hard to adopt, but super powerful. BuildXL is more about meeting codebases where they are - take an existing msbuild|cmake|dscript build with its under-specified dependencies, and figure out how to parallelize it safely. This delivers quick wins, but comes with real costs as well (notably: no caching; labour-intensive magic to guess an initial seed of deps, with a performance cliff when something changes).

(Disclaimer: Googler, work closely with the bazel team. I like bazel; I like BuildXL for expanding the space of build ideas, but have long-term concerns it settles in a local maxima that'll be hard to get out of).

You say that like bazel hasn't already been proven out on a few billion LoC :). You're quite right that exceptions will exist - your job is to either fix them or squeeze them into the build system. Both are well trod paths. You're quite right that these edge cases aren't _easy_ - but in what system would they be?

> there will be circular dependencies--you will have to deal with it.

Add them both to the same compilation unit (cc_library, whatever). Or extract an ABI (e.g. Turbine) and compile against that.

> There will be two different C compilers on two different architectures. You will have to deal with it.

Poke around https://github.com/tensorflow/tensorflow/tree/master/third_p... . I see arm and x86, Windows and many variants of Linux, multiple versions of GCC, multiple versions of clang, multiple versions of cuda, python 2 and 3, and names I don't even recognize.

See also https://docs.bazel.build/versions/master/platforms-intro.htm....

> Libraries will be in the wrong place--you will have to deal with it.

Just write the path to the file, or write a build rule that makes it visible in a standard location, or make it part of the platform definition, or use something like https://docs.bazel.build/versions/master/be/workspace.html#b... to alias it into your workspace. (Not recommended, but supported.)

> Libraries will be in the wrong place--you will have to deal with it.

Platform-specific library paths are very common. These days it's probably better specified as part of your "platform", but https://docs.bazel.build/versions/master/be/functions.html#s... is a convenient way to make in-place selectors.

I won't pretend bazel can express _everything_, but there's little you can't hack in _somehow_ with sufficient motivation, and moving towards the bazel-recommended patterns brings growing peace-of-mind (and faster builds).

(Disclaimer: Googler, work closely with the bazel team. I like bazel).

I love this example, but more because it shows just how obtuse make is even for simple programs. There's one self-explanatory line in this whole program (`SRCS = foo.cpp bar.cpp baz.cpp`), and everything else is magic you'd just have to cargo-cult forward.

I _think_ that it's also broken, if I copy-paste it: your comment renders with spaces in the `clean :` stanza, but I believe make requires that to be a tab character?

While certainly simplistic, the bazel example shows one obscure feature (glob) that's both still fairly obvious, and unnecessary for a direct comparison to your example. The rest reads clean, and could be replicated/modified by someone with no clue fairly straightforwardly.

Don't get me wrong, bazel BUILD files will often hide a whole lot of magic. But the benefit is, for a newcomer the files they have to interact with on a day-to-day basis are friendly. Take tensorflow, one of the most complicated bazel repos I know - browse to a random leaf BUILD file and you'll probably find it quite readable. (I randomly clicked to https://github.com/tensorflow/tensorflow/blob/master/tensorf... , seems pretty clear - especially when I consider it's compiling kernels for GPUs, which I know nothing about.)

(Disclaimer: Googler, work closely with the bazel team. I like bazel.)

(Disclaimer: at Google)

It's not quite so black and white. It's true that repo-wide refactorings often get carved into little changes and so aren't made atomically, but they're the exception rather than the rule. Any small change, e.g. changing an interface and the 5 callers of it, _can_ be made atomically. And changing code that's reused a small number of times is a far more common case then changing core libraries the whole company uses, so atomic submit ends up being hugely valuable.

That's not the non convex case though (as e.g. 1 and -1 is). You're right that chance can be convex; the premise of the article is that that's the property of interest, and that randomness alone isn't sufficient.

The expected value of an infinite series of coin flips centers on 0. Sure, you might get lucky, but you can't leverage that as a strategy to ratchet up value. That's the point about luck in the non convex case.

To your mining example, that's where prospecting comes in. Use your knowledge to make checking candidate locations cheaper (increasing convexity) - knowledge of geology to approximate likelihood, improvements to technology to determine if gold exists, etc. Then go check as many candidate mines as possible. It's much more cost effective to have a lot of shallow mines than it is to extract every ounce all the way down to the crust from a single mine.

It looks more like Go code than a bash script - the tradeoff we settled on is that pretty much as soon as you need to add any sort of logic it's no longer really a simple "script" and you _know_ it's just going to grow into a monstrosity. Better to use a language with real functions, real error handling, that you can actually unit test, etc. In that sense, I guess you could say we write lots of little tools moreso than we write scripts.

For something of this form, if the standard library has the functionality, we us it - os.Mkdir() instead of `mkdir` and so on. But to simplify shelling out, we have a little library that includes the interface

  type Runner interface {
	Execute(dir, name string, args ...string) (*CmdOutput, error)
  }

so it's easy enough to call miscellaneous programs and get the exit code / stdout / stderr. It also supports printing and executing a command, etc.

Iteratively executing a program of this form looks like `go run whatever --flag=value`, though your shebang hack looks like it'd also do nicely.

(Disclaimer: at Google)

My team has ~200k lines of Go code and I'm exceedingly happy with the state of the codebase. Having previously maintained a C++ codebase of similar size, I can say that the pace of changes is higher, the effort necessary for large scale refactorings is lower, and our ability to reason about the system is similar.

A few examples:

- Refactorings are simpler due to the use of consumer-side interfaces. Say you want to inject an in-memory cache above a backing store. To do that you probably have to change the constructor and provide an implementation matching the 2-4 relevant methods. That's it.

- Tracing code is slightly worse, due to having to track callers through said duck-typed interfaces, but on the flip side multi-threaded code is sufficiently simpler to reason about that I call it a wash. Having previously had to do threading in the form of "control flow" state machines, and then fibers (which were better but not perfect, and still aren't widely available), Go constructs are great. Locks where appropriate, channels where appropriate, overall very fast and clean code.

- Performance is good, and reliable. Not as good by cycle-count as C++ - and e.g. the comparable RPC libraries are definitely less mature than Google's very-well-kicked C++ libraries - but on the other hand it scales almost linearly. We started a system at ~5 cores/task under AutoPilot, and then when we next got around to adding more tasks it was peaking at ~60 cores/task at essentially the same per-core throughput. I've never managed to write a C++ server that can accidentally scale concurrency by >10x without hitting _some_ bottleneck.

- We use Go for ~everything. Server code, definitely Go. Client tools, also Go. Simple scripts, bash until they need their first 'if' or flag or loop, then Go too.

- I'd prefer real generics to interface{}, but the number of places it comes up is minimal enough that it's no more than a minor annoyance.

I can't speak to the issues of package management - we dropped compatibility with Go's native package structure fairly early on and went all in with blaze/Bazel (http://bazel.io) to coordinate builds and dependencies and whatnot, and haven't had reason to try modules yet.

Hmm, good for usage you say. Ok, let's stick to "standard" imperial units. How many feet in a yard? Yards in a mile? What the hell is a quart? And since base 12 is great, what is 12 feet called? 144 feet? A 12th of an inch?

Or, ounces: How many ounces in a gallon? A pound? And how many _kinds_ of ounces are there, anyways?

Fun fact: The term “ounce” is of Latin origin from the word “uncia” which means “a twelfth part.” http://www.differencebetween.net/science/mathematics-statist... So...what's an ounce a twelfth of?

Or, say, a pint. What's the definition of a pint?

I use [1]. Does this work for you? I'm not sure the exact criteria at play.

[1] https://chrome.google.com/webstore/detail/gerrit-ui-switcher...

I liked https://www.lrb.co.uk/v39/n07/daniel-soar/the-most-expensive... for its explanation of the political situation the F-35 is being built in, and the multinational ties that make it extremely difficult to cancel.

TL;DR: The scope of this project (intentionally) means there are customers in 10s of countries, with most of them also contributing portions of the plans and so invested in the sales of the others. Just in the US alone contractors account for more people than the Coal industry we're hearing so much about right now.

I have not got nearly enough information to say if it's a good or bad plane. But I find it extremely interesting to think that to a large degree, it doesn't matter.

Another fun start I recently learned is that about half of the area on the die on a modern chip is taken up by wires for the clock alone, in large part to ensure the length of wire to each component is the same length.

http://m.cacm.acm.org/magazines/2017/1/211094-exponential-la...

Surprisingly this is an ambiguous question to answer due to the nature of the expanding universe, but if the statement is that "the origin is 2.5 billion light years away" it was _probably_ transmitted 2.5 billion years ago.

http://curious.astro.cornell.edu/about-us/104-the-universe/c...

The catch here is that if DFP were to enforce technical standards to the extent of including performance, publishers would leave en masse. What they want from DFP (as I understand it) is hosting that just works; they don't want anyone in the way causing some ad campaigns to not get trafficked at the right time, or killed after serving for a while, or anything else. And worse, I think that you'd find that a very large number of ads wouldn't comply with any such standard - display ads aren't good, by and large. So even with a perfect offering publishers would look at the numbers and say "you know what, nevermind...turn the crappy ads back on." A pretty good approximation for publishers I've found was to say "if you give them a knob between anything and revenue, they'll turn it towards revenue." That by and large includes sacrificing user-first principles.

So one difference to your example is that viewability is desired - it's a direct metric that helps to approximate value, which advertisers like. But performance is not. Users cannot generally tell _which_ ad made their browser lag, or that it wasn't the page's fault. The best you could possibly do is to measure long-term-value, but because any given publisher only gets a tiny fraction of a user's time, and any single ad that much less, my recollection is that the LTV isn't really moved at all regardless of experience. It's going to take an industry-wide shift to move the needle here.

HTTPS is a closer comparison, except that it takes big parties again to force the issue - it's user positive but revenue negative and lots of work. So it comes back to the issue of who can get this change pushed through, without some party (pubs, advertiser, or users) just walking away. Look at examples of where/why HTTPS has been adopted if you want to see more.

AMP may get to that point, we'll see, but I like it as a possibility. And it's doing more to simply not show ads if they're taking too many resources (with browser assistance iirc? Delay loading? Don't recall what I've read), which is good in other ways.

Which reminds me, that's another plausible solution that I'm liking...if browsers can be more involved in aborting rendering bad iframe'd content, or preventing them from janking the page, etc, then the burden is squarely on the advertisers to fix or suffer reduced impact, irrespective of publisher/ad network. That's great, it puts the incentives cleanly where they should be. So I'm also hopeful that browsers can push through positive results here. Doesn't help with content badness (only latency and other such browser-measurable "bad ad" issues), but is a great starting place. (And to be fair, I don't think AMP solves for content badness either).

> That mobile doesn't have a way to block ads is probably a failure of Google, Apple and other players to allow extensions.

To be honest, I see this as short-term painful, long-term desirable. Ad blockers make the incentives of the web worse - they mean that publishers get punished for bad behaviour of other sites. Whereas without, users make more direct value judgements...if your site is covered in ads and a terrible experience, I'll stop visiting it. (I'd accept arguments that 100% ad blocker adoption could push us to a better place so we should let this run its course instead, but when I think it through I think it more likely pushes towards walled gardens and the death of the open web, so I'm skeptical.)

To your second point, I don't think that Google has the power you think it does to strong-arm other ad networks, and that'd be a really dangerous thing for Google to consider doing anyways. Publishers are often struggling right now (especially newspapers), and so revenue is their only thought. They'll go with whoever brings in the most money, because otherwise hastens the death of their company. This is why I like the AMP argument - it isn't forcing anyone, but it makes the argument that "if users can expect a good experience, you'll make more money, in spite of not being able to be as free with the technologies you jam onto the page". Yes, it's a weaker appeal, but that's what I like about it...because it has to be predicated on actually working better, rather than about pushing people around.

You read correctly, but it also doesn't tell the full story. I've edited to be "not related to AMP" to try to convey this better.

In short: AMP itself is not part of Ads, but there is a group within Ads focussing on AMP - because one of the biggest blockers to AMP adoption is (was?) getting ads supported within the model.

My apologies. I work at Google, used to be in Ads (not AMP), these days I'm in Technical Infrastructure.

Can we assume they'd be willing? Here in Canada I use orders of magnitude more water than oil products, and would be willing to pay commensurately less.

This post seems to conflate two ideas (1) AMP being an insufficient implementation (update-ping not working, and that being an awkward pattern anyways) and (2) AMP being actively harmful to the web ecosystem in principle.

(1) Comments on the implementation I don't know nearly enough to contest. It's a new technology, and will take a lot of changes in and out of Google to get right. It's also early days - continue raising a stink about things like this and I expect it to get better quick.

(2) The principle of AMP leads to a very interesting discussion. The fact that the top-level URL users go to is not the canonical URL for the page does stray into "breaking the web" territory, at least until it's better supported in browsers. On the other hand, as I understand it that's a technical requirement for AMP to be able to verify the content served through it, which is a requirement in turn for keeping AMP page quality high (in terms of latency, etc, rather than content judgements). Which I think is the most enticing part of AMP, and not something that should be given up lightly.

I see AMP as a possible answer to the tragedy-of-the-commons problems that have resulted from standardizing on third party display ads for monetization, third party scripts for analytics, etc. Giving someone else a hook into your page that isn't incentivized to keep it fast and reliable leads to an end state where publishers have no reason to clean up the experience on their own sites because that'd cost them money, and because the bad experience on every other site prevents them from being commensurately rewarded for being a good player. (E.g. adblockers affect every site equally). And the third party tech providers have no incentive to fix things for similar reasons - alternative providers out there would cause the same poor user experiences anyways, and make more money from doing so and enabling them to sell to publishers better. If we cannot rely on principled actions from publishers or tech providers, how does the experience of the web get better enough broadly?

AMP to me is an answer to that - maybe not the best answer, but the most viable one I've seen so far. It allows for direct control of what is/isn't possible, and if it gains traction may have enough clout to push for some of those better experiences that aren't arising naturally.

The other possible answers I see don't seem to be working better. Relying on regulation seems destined to failure; user experience is too messy to craft rules around, and nobody is placed to enforce them globally. Relying on common gatekeepers seems to be failing - Google algorithm changes worked for a while to incentivize page quality, but I don't think Facebook is (or should?) making equivalent judgements about the quality of content users link to for ranking purposes, and it seems even harder to enforce in other direct-user-sharing sites like Twitter.

[Edit] Disclaimer: I work for Google, in Technical Infrastructure. I used to be in Ads, though not related to AMP.