[1]: http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on...
737 karma · joined March 26, 2014
[1]: http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on...
You can't do this in general, because we often want to build applications where the user expects their actions won't be reordered. People often forget this: the user is sort of the unstated extra party in a distributed system. A canonical example is privacy settings on a social network. If I want to hide my profile from a user by unfriending them and then post something unkind about them, it doesn't really work for me that the application is free to reorder my actions.
At some point your desire to have HA via a weakly consistent system conflicts with intuition and usability of an application.
1. Bank transactions are easy to express as a join-semilattice[1], which means its intrinsically easy to build a convergent system of bank accounts in a distributed environment. Many problems are not easily expressible as these and usually trade off an unacceptable amount of space to formulate as a join-semilattice and still work with complicated types of transactions.
2. A bank transaction has a small upper bound on the types of inconsistency that can occur: at worst you can make a couple overdrafts.
3. A bank transaction has a trivial method for detecting and handling an inconsistency. Detection is easy because its just a check that the transaction you're applying doesn't leave the account with a negative balance. Handling is easy, because the bank can just levy an overdraft charge on an account holder (and since they have the ability to collect debts on overdrawn accounts, this easily works for them). Complicated transactions often don't have a trivial method of detecting inconsistency and handling inconsistency is rarely as simple as being able to punish the user. In fact corrective action (and likewise, inconsistency) is often non-intuitive to users because we expect causality in our interaction with the world.
> The real world rarely involves having a consistent view of anything
I strongly disagree with this. Most of our interaction with the world is perceived consistently. When I touch an object, the sensation of the object and the force I exert upon the object is immediate and clearly corresponds to my actions. There is no perception of reordering my intentions in the real world.
We expect most applications to behave the same way, because the distributed nature of them is often hidden from us. They present themselves as a singular facade, and thus we typically expect our actions to be reflected immediately and in the order we issue them.
Furthermore, this availability vs consistency dichotomy is an overstatement of Brewer's theorem. For instance, linearizable algorithms like Raft and Multi-Paxos DO provide high availability (they'll remain available so long as a majority of nodes in the system operate correctly). In fact, Google's most recent database was developed to be highly available and strongly consistent [2].
The biggest challenge is definitely how easily exceptions can slip by handling code unless you make strict use of throwIO with synchronous exceptions (or something equivalent like throwM from exceptions). My favorite example is the following:
let foo = try . return $ error "foo".
Which evaluates to "Right * Exception: foo".
To name a few:
1. Poor debugging tools. Unfortunately this is sort of intrinsically tied with non-strict evaluation. Typical evaluation stepping debuggers would be sort of unpredictable in haskell. Along these lines, haskell doesn't have stack traces enabled by default, and reading them is sort of tricky.
2. Clumsy exceptions model. There are asynchronous and synchronous exceptions, with the later being further broken down into exceptions in pure code or exceptions thrown in IO. Only exceptions thrown in IO are catchable. You can think of exceptions in pure code as similar to "panic()" calls in other languages, except they're even tricker due to lazy evaluation, and aren't necessarily guaranteed to be triggered due to slipperiness with laziness. Also since exceptions are used for interrupting computations (timeouts, for instance) and there isn't a way (other than some type class conventions) to distinguish between interrupting exceptions that should be allowed to propagate and exceptions due to "exceptional circumstances", catching all exceptions safely is a tricky matter.
3. Record syntax leaves a lot to be desired. It's incredibly easy to run into situations where you would have conflicting functions due to record syntax. Lenses are a partial fix for this, but its sort of annoying that something like row polymorphism isn't just a part of the language.
4. Laziness can make reasoning about time and space complexity trickier. Generally this is a little overstated, but you'll occasionally run into space leaks.
There are definitely real problems with the language, and some of them (such as poor debugging capabilities) can be legitimate showstoppers to use in industry, but generally I find that its advantages far outweigh its negatives.
What does this even mean? It's impossible for me to reply to this unless you're more specific about what you're talking about and how it involves the organizers for Nodevember. Also "taking a wider context" is useful for understanding something only once the specifics of what actually happened become apparent. Otherwise its impossible to understand what the context even is.
The last controversy like this I can remember was Curtis Yarvin being disinvited from LambdaConf, and even then people were bizarrely claiming that disallowing a white sepremacist from speaking was uninclusive. I'm not trying to link these events (there's no evidence to claim that the circumstances are the same... though bizarrely the guy who wrote the blog yesterday is trying to link them in some way favorable to Crockford[1]), only pointing out how completely counter-productive it is to attempt to place something "in a larger context" before anything is really known about the details.
> That means inclusive for people like them, and exclusive of people not like them.
This is just a slightly stronger form of what I said above. You have no evidence with which to draw this conclusion.
> They do not seem to particularly care about being inclusive of minorities, who are just a smokescreen (see, e.g., https://www.reddit.com/r/javascript/comments/50u77r/douglas_...). They appear to be guided by an intuitive sense of "our sort of people".
Again, on what basis? This comment is from a person saying they'd be in favor of uninviting Crockford if he was responsible for being toxic or inappropriate. They seem to be concerned with accusations about one of the persons involved with calling for him to be replaced as the keynote speaker.
> Don't you think he might be obstinate for a reason? He's responsible for one of the most important pieces of infrastructure in the world: bugs in the Linux kernel could cause life-or-death situations.
I'm so sick of seeing software engineers justify arrogance or invective as though they somehow make maintaining critical software feasible. This is indefensible macho bullshit that needs to dropped. Maintainers have plenty of tools at their disposal for when code doesn't meet their criteria: the primary of which is to not accept change sets. Belittling contributors or engaging in childish invective isn't a valid tool for turning down code. At best it turns away people who could be guided to making productive contributions (and yes, there are plenty of people in the linux kernel community responsible for this), and at worst it scares away other potential contributors who are concerned with their own fallibility and the prospect of being publicly humiliated for making a mistake. Sorry, but no. I don't think he has a legitimate reason for being obstinate.
> The world needs more programmers, and we shouldn't care what they look like, but inclusivity is not an end-in-itself. "Egotism and profound condescension" is terrible if you think the tech community exists to provide a weird social club, but it's essential if the goal is to help programmers write better software - the master programmers have earned their big egos because they know how important it is not to f... certain things up.
Sorry, but I don't buy this "programmer bootcamp" model of teaching developers. There's plenty of psychology and management research out there showing that ostracism and negativity increases stress[2], decreases social cohesion[3], decreases performance[4], etc. In short, this is complete bullshit.
[1]: http://atom-morgan.github.io/in-defense-of-douglas-crockford
[2]: http://webuser.bus.umich.edu/janedut/POS/Heaphy%20and%20Dutt...
[3]: http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&uid=2009...
[4]: http://www.sciencedirect.com/science/article/pii/S0278431911...
I saw the tweet, my point here is that's a single tweet raising a red flag about what is undoubtedly an issue that was talked about to a greater extent behind closed doors with relevant parties. If you find that justification to be insufficient irregardless of whatever the backing details may be, then feel free to outrage on, but I think that's pretty lamentable.
The fervor over this issue seems to be predicated on the belief that what appears on twitter is the only set of events that lead up to him being disinvited, which cannot be demonstrated and conflicts with Nodevember's statement on the issue.
> This is a roundabout way of saying that I'm making things up.
Yes, I think anyone who has come to the conclusion that the only reason for this is liberal virtue signaling instead of an actual aspiration to create an inclusive environment is actually making things up, since there isn't enough evidence to draw that conclusion.
> Crockford doesn't need to do anything. You're implying that he's the only one who should be bothered.
I'm implying he should at least be the first person to be bothered, unless the full details come to light. Otherwise this is a lot of rage stoking over absolute bullshit.
> But the whole point of incidents like this is to show that people like Crockford aren't welcome. Linus Torvalds gets the same s... from the same people for the same reasons.
I think there's a legitemate reason to disinvite people like Torvalds who have a long history of being brutally condescending. And more importantly has been asked ad nauseam to please be kinder to people and has been firmly obstinate. Disinventing a personality like his works precisely towards the goal of being more inclusive. Hell I see a blog post every other week on HN about "imposter syndrome" and how that debilitates people in our community, so it makes complete sense to me that people who are dramatically negative towards others are toxic and harmful to the goal of creating an inclusive environment. This notion that we have to accept egotism and profound condescension in this community is extremely backwards.
I don't see anything in this chat log that changes anything.
Instead of inserting your own narrative into how and why this occurred and pouring gasoline on the fire, how about let cooler heads prevail. If Crockford feels wronged by this exchange, he's free to come forward with his side of things (and other parties with theirs) if a public debate needs to occur. Until then, this is a lot of people being roped into proxy battle of "SJWs" vs "un-PC enthusiasts" that frankly has little to do with an actual set of circumstances.
There's a mountain of tweets discussing this in replies to Nodevember's announcement, but nothing directly affirmed by Nodevember as the basis for their decision. Seems like a lot of outrage being stirred up over bullshit no one can substantiate.
[1]: http://atom-morgan.github.io/in-defense-of-douglas-crockford
[2]: http://nodevember.org/statement.html
[3]: https://medium.com/@nodebotanist/why-i-won-t-be-speaking-at-...
No, what I'm saying is you (and everyone else) aren't owed an explanation, and furthermore there are good reasons for not offering a public explanation.
> And that Crockford doesn't respond to this hypocritical nonsense is a kind of confession of guilt?
No, I'm acknowledging that this is a lot of frenzy being whipped up when the party directly affected by the decision hasn't released a statement questioning their motivation. He may have been given an explanation and has chosen not to make an issue of it for all you know. The only people seemingly angered by this decision are people who aren't affected.
Like the one yesterday, it attempts to provide its own explanation for why Crockford was disinvited as the keynote speaker. However, none of these details are mentioned in the actual statement given by Nodevember[2], and there are no sources given for how this explanation was arrived at. The Nodevember statement claims he was disinvited based on some public and private feedback from speakers who felt uncomfortable attending with him as the keynote speaker. Why should we take it on face value that the circumstances documented here are actual set of circumstances that lead to him being disinvited? Where is this information even coming from?
One of the alleged sources is a blog post from Kas Perch[3]. The alleged slut shaming incident seems to be a minor note in a post otherwise focused on his negative and dismissive attitude towards some of the other speakers. Perhaps this is the more salient part of why this person felt uncomfortable attending with him? Perhaps there's more feedback being kept private to protect the witch hunt from targeting those coming forward?
Notably I haven't even seen a statement from Crockford crying foul over this incident. It seems to be coming entirely from other Node developers upset about the fact that he was removed as the keynote speaker. It seems like this is an argument being played out as a proxy over political stances about diversity and inclusion tactics, without a lot of consideration for the actual circumstances. Furthermore I find it very disappointing that this poorly substantiated argument is being used to demonize inclusivity and diversity overall, which is something badly needed in developer communities.
[1]: http://atom-morgan.github.io/in-defense-of-douglas-crockford
[2]: http://nodevember.org/statement.html
[3]: https://medium.com/@nodebotanist/why-i-won-t-be-speaking-at-...
As for the above story, that engineer was sort of on his way out at the time, so I used the above method to provide query hints as a short term fix. After he left, I was able to restructure the event data schema to make more use of columns. Some of the ancillary attributes that weren't used for row selection stayed as jsonb, but things like timestamp, event name, user id, etc. were moved to columns.
Protip: Use the planner config settings[1] (one of which is mentioned in this article) with SET LOCAL in a transaction if you're really sure the query planner is giving you guff. On more structured data that Postgres can calculate statistics on, let it do its magic.
[1]: https://www.postgresql.org/docs/current/static/runtime-confi...
Huh? I'm not actually arguing that assembly is a scalable language. I'm invoking a counter-example to the idea that a smaller cardinality of concepts is inherently a good thing. Assembly has a smaller number of concepts than Go, so by the espoused benefits of having a language with less features, assembly should be favored. But obviously this is not true, so I debate that Gophers actually ascribe to this version of "simplicity".
My point here is that Gophers need to examine their rhetoric a little more and get better at honing their definition of "simplicity", since its clearly not just having less "stuff" as Rob Pike seems to claim in every Go presentation.
> Sorry, that doesn't follow. Is the logic here just because I mention Smalltalk, that I'm advocating late binding and the only type being Object for Go? Sorry, but that doesn't follow. The argument is that Go doesn't need a more complicated type system to avoid problems with heterogeneous collections -- because practice shows that even a simpler one can suffice.
Your original question was how does unsafe casting introduce cost. It adds cost in exactly the same way that every other means of circumventing a type system or not having a type system introduces cost: it allows runtime errors to occur at points where data is illegally used.
Type systems are effectively proof solvers. Just like making an improper assumption in a logical proof can lead to a faulty conclusion, forcing a type system to assume a type for a value that it cannot prove can lead to a buggy program. This is why programmers who strong believers in static type checking take issue with casting: its a way of circumventing the protection that a type checker gives you, when instead you can add power to the type system for expressing your constraints or add means of showing the equivalency of different types.
> Sounds like you're invoking freshman level false "common knowledge."
There's no need to get personal here. I'm making a factual point: it's true that any code path calling into the point where a type bug occurs is potentially responsible. Nothing in my comment is invoking "common knowledge". Also you should keep in mind that invoking your "personal experience working on X large scale system in industry" is not a compelling argument. It's not even a comparative argument about an untyped language vs a statically typed language.
> Have you ever worked in an "untyped" language in a real project? What if a project simply used runtime asserts? Then a type error in a heterogeneous collection would be caught in unit testing. If it got out to production, it could be easily caught and logged. In 15 years of Smalltalk industry work I never encountered the kind of heterogeneous collection type error you're referring to in production. The closest thing I can recall involved the heterogeneous typed reuse of a local variable.
Yes, I have. I've worked in Python and Javascript for a couple large projects. I'm not going to get into my feelings about this, because I don't think it forms the basis of a compelling argument.
However, I take issue with the claim that these kinds of bugs are always trivially caught in unit tests. One thing to note about untyped languages is that they allow an infinite number of values to be passed to a function by virtue of being untyped, so there's no way to write an exhaustive unit test. This isn't unique to untyped languages (for instance, I can't write an exhaustive unit test in haskell for a function that accepts strings), but a sufficiently expressive typed language always gives me the ability to reduce the scope of my tests by writing more constrained types (for instance, with sized collection types using Data Kinds in haskell). Similarly, languages that disallow parametric types cannot express constraints about contained values in a type, which allows exactly the same sorts of bugs that an untyped language can have.
Unit tests are great, but they are better suited for probing the edges of acceptable inputs based on assumptions about the code under test, and are generally poorly matched to providing the guarantees of a type system. They are not perfect: they can suffer from laziness, code rot, faulty assumptions, etc. I've seen bugs in test code far more frequently than I've seen bugs in a type checker (in fact I don't ever think I've seen a bug in a type checker).
My argument here boils down to the fact that you can trivially show there's potential for human error here that a type system can protect against. The point of contention you have is that these kinds of bugs don't manifest in practice. In my experience they do, and they occur more frequently in larger scale systems where there's more invariants to juggle that a type system doesn't ensure for you. I'd also argue that this largely explains the resurgence of typed languages with more expressive type systems (like scala, rust, swift, idris, hack, etc.). Ultimately I think we just have to agree to disagree here.
> (P.S. Technically speaking, Smalltalk is strongly typed with message passing semantics for methods implemented through late binding. It's not "untyped.")
Untyped is commonly used in academic literature to refer to "dynamically typed" languages[1]. The strong/weak typing distinction is arguably imprecise or a useless distinction, especially for dynamically typed languages. For example, how does smalltalk prevent "type punning" when functions do not declare the types of values they may be called on? Perhaps you can make the argument that dynamic languages like these can justify their claim to "strong typing" by having builtin operators that do not make implicit conversions of the values they work on, but this guarantee doesn't hold in general in user defined code, so it seems like a useless distinction.
[1]: http://stackoverflow.com/questions/9154388/does-untyped-also...
Node came of age about a decade after epoll was introduced, when not having access to nonblocking IO was considered a big liability for a couple of dominant web programming languages, and they built their concurrency model around the semantics of epoll.
However, there are languages like Haskell, Erlang, and Go that IMO did the right thing by building a synchronous programming model for concurrency and offering preemptable lightweight processes to avoid the overhead associated with OS thread per connection concurrency models. These languages offer concurrent semantics to programmers, yet still are able to use nonblocking IO underneath the covers by parking processes waiting on I/O events. It's not the right tradeoff for every language, particularly I think lower level languages like Rust are better off not inheriting all the extra baggage of a runtime like this, but for higher level languages I think its probably the most convenient model to programmers.
GHC 8 also introduces the Strict and StrictData pragmas[1] which allow you to make a module (or its types) fully strictly evaluated.
The entire build chain with Go is probably one of the most frustratingly limited build tools I've ever used, which is probably why nearly every Go developer I've met has switched to using one of the third party options.
> The argument is that simpler is better at scale. Airplanes can move freely in 3 dimensions, but airliners are constrained to fly in particular ways around busy airports and cross country.
For one, I just debate the premise the simplicity has anything to do with cardinality of features/concepts. But let's take that argument at face value: then why _not_ assembly if this is the case? Why not a language with the absolute minimum number of concepts? I think if you interrogate this premise you'll find it doesn't hold a lot of water and that Go doesn't really aspire to this goal anyways. I think we have some amount of working memory for being able to intuit programming with a certain number of concepts. There's a valid argument that some languages suffer by breaking that barrier (though I personally think Go underestimates where that barrier is), but it seems incorrect that language designers should be optimizing for a minimal number of features.
I think complexity at scale has more to do with features that interact poorly (or cause poor interactions more frequently with a larger number of people). Specifically its about composition. For instance, there's a valid argument to be made that asynchronous exceptions (i.e. the ability to interrupt another thread with an exception) and locks poorly compose. Mutable state is a common example of a feature that's a detriment to composition. But parametric polymorphism, if anything, gives us a much greater ability to compose. It allows us to define functions that work on data arbitrarily parameterized by other types, which makes them conducive to composition. And likewise, we don't suffer ability to reason about composition at scale with parametric types. A parametric function does not gain complexity as more team members are added, more code is written, more deadcode accumulates, etc. Parametricity changes nothing at scale.
> In your experience, what kind of "cost" has there been in unsafe casting to use collections? Even in environments like Smalltalk, where all use of collections amounts to "unsafe casting," I've rarely seen situations where a mistake of this type wasn't found trivially.
That's an argument for Go to not have types. But Go does have types, and type safety is often espoused as a benefit of Go. If you're going to have types, it makes zero sense to me why you should not have parametric polymorphism, since this is the only way to have things like typed collections without opening yourself up to the possibility of casting errors. Frankly I find it bizarre that people claim that they have found type errors to be trivially fixable, because the scope of where a type error can be introduced is enormous in an untyped language... its literally every location that potentially calls into the code where the error occurs.
> Does your frustration come from having to abandon the "assured safety" the type system would give you, or does it come from an experience of the costs?
Yes, type safety is an enormous advantage to writing correct code in my opinion. It's one of the best mechanisms a programming language can give you for enforcing invariants about data. The curry-howard correspondence is a huge advantage to writing correct code. Every place a type checker isn't being used to delimit acceptable data is a potential source of a huge number of bugs. It's also a frustration because casting introduces conversation and type checking boilerplate that a type checker could ultimately take care of for you.
This is a narrative without specifics, and unfortunately always where the conversation seems to end with gophers. We accept some amount of features that can be abused because they offer utility that outweighs their potential for misuse. So how exactly are generics a worse offender than these other features or a worse tradeoff for their utility? Because from my perspective, being able to define parametric data types and functions is a huge win for safety and terseness of code without a lot of downside.
"The key point here is our programmers are Googlers [...] They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt."
I'll concede there's the possibility for some weird tongue-in-cheekness here, but it definitely seems to be the canonical view among gophers that Go's paucity of features is about accessibility for programmers that don't understand them or find them cumbersome to work with.
I think this idea that "paucity = good" is so easily abusable that whenever this comes up from gophers I wish they would concede that this is an unhelpful simplification of what they must actually believe. Assembly language has possibly the highest paucity of concepts given it offers no ability to introduce language-level abstractions (other than say conventions about calling, etc.), but Go is nothing like this.
The argument can't be that paucity is good as a general condition, its that there are forms of abstraction and programming language features that Gophers find unhelpful or difficult to understand. The problem I have with this when applied to parametric polymorphism, is that Gophers already work with these concepts daily, so it can't be that use of them is complicated.
I also have a hard time believing that the ability to define parametric types and functions costs you anything. It's almost always self-evident when to use parametric types or functions, things that are "wrappers" or "collections" probably account for 80% of their use. I also don't think I've ever experienced ambiguity of choice with the feature. For instance I don't think I've ever been in the situation where I had to trade off implementing a generic definition vs. N specialized definitions. The frustration of using Go is actually that I now have to consider the later as a possibility or trade off type safety by using unsafe casting.
If there's a place where parametricity truly introduces complexity, I'd love to hear about it from a Gopher instead of a blanket statement about how "programmers don't understand it", "it decreases readability", or "Go is simpler without it".
Teach them to require consent and communication, and tell them they'll always have your support. Beyond that, don't get involved in what may or may not be virtuous behavior to you. Women inherit a lot of puritanical guilt over nonsense that parents project onto them.
I understand the desire to protect your children, and particularly your daughter in a world that generally values them less. But misogyny is a power dynamic that is reinforced by socializing women to normalize it and especially by socializing men to perceive their existence and agency as more valuable.