zeeboo

I don't understand your points about the NSA letters, which makes me think that my point was missed. I am saying that the NSA letter claiming that only some information was logged is fully independent of the open source code of the server. Assuming the NSA letter reflects the truth, there could be more information or less information that what appears to be collected from the open source server code because, once again, the server does not have to be running the open source code, and even if it were, that does not preclude other systems from running against the same information the server has access to. Hence, open sourcing the server does not affect the security of the system at all. If the system is insecure without knowledge of how the server works, then the system is insecure. Period.

I think you're trying to argue that open source is good, and I agree with you. Open sourcing the server has many benefits. The only point I have consistently been trying to make is that open sourcing it does not help with determining the security of the system, whatsoever.

edit:

> If you didn't have the NSA letter, would you be able to verify the source code?

No, but even if the code was open sourced, you would not be able to verify the code that is running.

> If another project got an NSA letter and responded to it, would it tell you anything about the source code?

It would tell you something about the code they are running, yes, but nothing about the code they open sourced.

> This is simple: Having the source code means you get to learn more from the other signals, no pun intended, of how that source code is used.

This is equally simple: the source code that is open may have nothing to do with the source code that is running, and you must assume that they are not equal when auditing the security of the system.

Did you notice how your proof rests entirely on the NSA letter, and not the source code of the server at all? Isn't a world conceivable where they open sourced the server, with no logging, and then sent an NSA letter that contained information that wasn't logged in the open source code? If this is somehow impossible, please explain how.

You can know exactly what the attacker can learn because you can see ALL of the information that your client passes to the server by auditing ONLY the client. Your argument applies equally well to every single router or middle box on the internet, and it's just as wrong there.

You prove that it doesn't matter by assuming that keybase is running the most malicious code possible, auditing your client, and deciding that the system is still secure. This is what auditing the client means.

Additionally, to bring up this fact again because it has only been hand-waved away: Even if the server was open source, there is no guarantee they are running that code. Thus, there is no benefit to security until systems exist (somehow?) to prove the server is running the code you expect.

Double additionally: even IF you can prove that the server is running what you expect, how do you know that some box, after https is peeled off, but before the request makes it to the server, is not sending the same request off to some other, malicious, server?

And yet, all of the routers are not audited, and I presume you believe in the security of some applications that use them. In other words, the trusted base of the system does not depend on all of the components in the system (you start with the assumption that any closed components are hostile). The keybase servers are exactly the same.

The answer is that they do have things to hide: anti-spam/anti-scam systems, for example. The question is if they are hiding something that matters for security. You can determine this by auditing only the client.

Sure, open source software is great, and has many uses. In this case, it has no use in ensuring that keybase is secure. Somehow, you don't have to trust a great many components in the secure software you use on a daily basis, and yet you have to trust keybase's servers because.. reasons? And somehow this trust is important even though you'd still be blindly trusting that they're running what you hope.

I won't argue that some people would benefit from the server being open source, but to argue that open sourcing it has anything to do with security is just inane and FUD.

You can construct a client such that the server IS outside of the scope. You can determine if a client has such a property from the client alone. I would only want to use a client such that the server is outside of the scope. It's strictly better if the client has this property even if the server is open and magically trusted to be running the code you expect. The keybase client is such a client. If it isn't please inform someone with details on why.

edit: Do you audit all of the software running on all of the routers between you and keybase's servers? Why or why not? If not, why does this reasoning not extend to the servers? Why would the routers not part of the whole system, top to bottom?

So audit the client assuming that all of the traffic it sends is in the clear (i.e. not over https to keybase's servers). If that's not sufficient, then don't use keybase.

Regardless, until you have a way to ensure the server is running the code you expect it to be running (can this even exist? what about hardware level attacks on the servers keybase is running?) the server code is useless from a security perspective.

That's just false. You can get all of the trust information necessary from the client. It's exactly the same amount of trust.

edit: To be clear, even if they open sourced the server right now, I would not even look at the code to determine if running the client was safe. The only time I would care to look at the server code is if the client's correct operation depends on the server running specific code. If it doesn't do that, then the server code doesn't matter. And if it does do that, I wouldn't trust the system, anyway.

How does open sourcing the server help you audit what their servers are running? There's no way to know if what's open source matches their running code, and if the security of the system depends on the server being open, it's not secure.

Do you agree both sets are countable? If so, any two countable sets can be put into bijection by composing their bijections to the naturals.

I’m confused. First, there’s no notion of sub-typing in Go. Second, the closest thing I can think of what you mean does work: https://play.golang.org/p/IE9sg4CwCd_C

Usually those proofs start off by saying "pick an N as large as you wish", and then showing that the statement is true. Thus, you're certain that it doesn't break at big scales because you allowed any big scale you wished at the start.

The easy way: compress it with whatever you want (xz? gzip?) and compare the sizes. If it doesn't get significantly smaller, it's probably encrypted or already compressed.

It's clear that a SAT solver is strictly stronger than the MVS approach. In other words, any MVS selection can be encoded in a SAT solver. The argument is for a reduction in power. Thus, in order to convince someone that a SAT solver is preferred over MVS, you must show examples where it succeeds when MVS fails, and the extra power is necessary. It's trivial to contrive these situations, but finding them in practice seems harder. Evidence of that happening would help change my mind, and I'd hope would help change Russ's mind.

The empirical data from Go's package ecosystem is drawn from a corpus with sophisticated package management: dep. The argument is that dep is unnecessarily powerful and that a simpler approach will suffice. The evidence supports that argument. Note that this is not an argument about Cargo, or Bundler, or any thing else. Right now, the ecosystem is using dep, and there is evidence that it can be done simpler.

To stick with your analogy, I think it's fair to conclude that the only vehicles users need on bike trails are bikes.

Additionally, I have done an analysis of two Rust projects that have been brought up in my discussions on this issue. Specifically, LALRPOP and exa. In both cases, throughout the entire history of the project (hundreds of changes over 4-5 years), Cargo only had to select the largest semver compatible version [1]. Again, I would love to find examples of projects where this strategy was not sufficient.

[1] There is one complication: in Cargo, a ^ constraint (the default kind) on a v0 dependency is only good up to the minor version. In other words, ^0.1.0 means >=0.1.0 and <0.2.0, where ^1.0.0 means >=1.0.0 and <2.0.0. Selecting the largest semver compatible version is meant in this way because of the community norms around breakage in v0. In an MVS world, any breaking change is a major version bump, and would have the same properties, but with different version strings.

I think there's empirical evidence that the SAT-solver approach is not necessary. I have done an analysis on as many Gopkg.{lock,toml} files as I could find, and in no instances did it ever do any non-trivial version selection: the maximal available version at the time was always selected. Additionally, Russ has stated that ~93% of the top 1000 Go packages in the wild build successfully with no changes. I appreciate that they may be convinced, but I think they need to ask themselves what evidence would change their mind. They have had months coming up on a year to figure this out.

https://github.com/zeebo/dep-analysis

For what it's worth, I analyzed as many Gopkg.{lock,toml} files I could find in the wild, and found that vgo's algorithms would service all of them. I've done similar analysis for a smaller number of Rust projects using Cargo and had the same findings. I've found it hard to find projects in the wild that actually do any non-trivial version selection. I think this is some good evidence that it will work out in practice.

I think it's less that Go has significantly different needs, but it's more that people overestimate what their actual needs are.

https://github.com/zeebo/dep-analysis

Additionally, dep is, and always was, meant as an experiment to learn from. Russ gave multiple concrete examples of things he learned from dep. That the dep authors don't take this as a win is telling. Pitching dep as the eventual Go dependency management system and having that not be the case is a problem they invented for themselves.

His "attack" is literally a brute force of 32 bits of state. It would not scale to PCG with 128 bits of internal state.

If I start at 0 and successively add 1, do you agree that I eventually hit any positive integer you could pick after a finite number of steps? Does that not prove to you that I hit every positive integers? Which one do I not hit?

You don't have to use a finitely definable function. In fact, Cantor's argument defines the function as an infinite list of real numbers.

You can leave the inner type out...

https://play.golang.org/p/m3bLmneArB

In some sense, yes. A proper scoring rule would give that forecaster less points than one that forecasts a 90% chance of winning when they win all of them.

Funny because Theo didn't say either of the quotes your parent was talking about.

An interface being equal to nil just because the value it contains is nil would be even more confusing. For example, imagine if you had two interface values, one holding a nil * T and one holding a nil * S. They would both be equal to nil, but not equal to each other, losing transitivity of equality.

Also, sometimes nil is a perfectly valid value of something and can implement interfaces. You can call methods on nil receivers because method calls are just curried function applications over the value. It would be odd if the interface compared equal to nil just because I happened to implement it with a value that's valid with nil.

The article paraphrased. The court document showed that the person attempted for months to get the plates and never did. Also the money was to pay for court fees which the government required the person spend in the first place to get the plates.

It can also switch on memory allocations and function calls as well. I'd expect a large fraction of the context switches come from those events and don't go through the kernel.

I hope your optimism that a single kind of thread pool will service all applications is well founded. It seems like people would want to specialize them much like they want to specialize their memory allocators. The Rust team has a really great track record of innovation and technical excellence so I look forward to the design that will accomodate that and hope the ecosystem buys off on it.

Go does limit the number of threads and will crash the process if it goes over. It's also very rare to have CGo call back into Go multiple times versus libraries juggling adapters between async and sync in my experience. It's also easy to have your library have a limiter on the number of CGo calls you make, but less easy to limit the number of tasks you throw into a thread pool because you don't have the option to block. (edit: I think you can just store the closure on the thread pool and schedule it eventually at the cost of writing your own scheduler and perhaps requiring allocations?) I have a feeling that a similar crashing solution won't work in the Rust community, and what to do when the limits are hit will be punted upstream. My main point is that there are many subtle details in solving the "colored function" problem.

I don't think all M:N systems have the debuggability problem becausd the runtime has a single canonical representation of what is running: the stack traces. Since the entire ecosystem bought into the runtime, you don't have any fracturing of representations. If you're optimisitc that the entire ecosystem will buy into whatever mechanism you have to do async work, then this can be solved, but I believe that's already not the case (people hand code state machines) and is unlikely to stay the case as time goes on.

Easy and error prone.

Sometimes libraries pretend to be async and accidentally are sync. Consider some library that in the normal case just does some pure computation, but logs to syslog or something on some error condition. If you use that library in an async context, it could work fine most of the time, until you hit some unexpected situation where it happens to make a network request to some syslog daemon and blocks up your worker thread. The same thing can happen with mutexes, or many other common blocking operations.

It's also the case that often async libraries depend on some sync library and so they have their own worker pool. You can easily have many libraries with their own worker pools all using more resources then they need.

You also have to worry about if your functions do any of these transformations under the hood. For example, if you have some async worker that delegates some sync task to the worker pool, and that sync task happens to use some async function and blocks on it, and that async function ALSO has a sync task and attempts to delegate it to a worker pool, and that worker pool is bounded, then you have just opened yourself up to a deadlock under high load that you probably won't find under normal operation.

On top of all that, debugging is usually much harder in these environments because you have to inspect the runtime memory state that depends on the specific internal details of the libraries being used instead of a bunch of stacks. It's extremely hard to diagnose deadlocks or stalls in these environments. It's non-trivial to provide a good debugging experience that doesn't cause extra load in production environments.

These issues are all real things that I have hit in production with Twisted. A static type system could help all these things, but I think it requires buy in from every library you might use, transitively.

My criticism is that the design limits the places where they are valid. I'm not trying to use a hammer where a screwdriver is required, I'm saying that if the hammer was designed differently, we'd be able to use it in more situations appropriately.

It's as if someone created a gun that fired backwards and I said "hey, it might be better if the gun fired forwards. we'd be able to use it in more situations." and people responded with "you shouldn't use a gun that fires backwards if you want to fire forwards." I totally agree, but it's missing the point.

It's great that panics happen when you violate those contracts. That is a deliberate design decision and I agree with it. However, the contracts that they enforce cause real problems evidenced by the article. Small additions might make those contracts more general and make channels more applicable. In my opinion, you should be able to attempt to send on a channel that could be closed in the same way that you are allowed to check if an interface contains a specific concrete type without panicing. In my experience, this would allow for a number of useful patterns that are very hard to express right now.

Nil channels blocking is definitely a deliberate design decision and has valid use cases. I use them frequently when I have a channel based design. It also isn't the thing that most people first expect since they have the opposite analog for using anything else that is nil: panics. The article which I assume you read makes only this point.

I never attempted to lift statements that are obviously opinion based (anything that has a judgement of something good or bad) as objective fact.

Here's a proposal I worked on with a coworker to make channels better that might give you more of an idea of why I'm suggesting that the current design has flaws: https://github.com/golang/go/issues/14601

Given how much weight channels are in the language specification and memory model, it would be nice if they were more generally applicable and easier to use for more concurrency situations.

The api definitely is badly implemented and makes them hard to use. That's the point of the post. There are design decisions around channels (sends panicing, close panicing, nil channels blocking) that make them hard to understand, follow, and compose concurrent solutions.