I use Elm, which is as statically-typed as possible for a client webapp, but I know I can never guarantee that JSON data I receive from remote servers is of the expected type.
Edit: also, versioning, cross language sharing, use in generative testing, api client code generation and other metaprogramming uses. And documentation generation, eg swagger
If you are using eg Clojure's spec, you can reference the same stuff downstream in the control flow too, for eg fn argument validation.
Types can also be very useful for security, to keep track of which runtime safety checks need to be done. For example, you can have a separate type for HTML strings that can be rendered without escaping.
I've worked on a lot of distributed systems, and I don't see what this has to do with anything. Yes, static typing doesn't prevent you from doing any of the things you mentioned like changing your API out from under someone, but it's not supposed to. It Does however prevent certain classes of mistake in the individual service or executable, which is still a laudable goal in and of itself. It's not a silver bullet, but it's no less useful in distributed systems than it is anywhere else.
Also, are we talking about strong typing or static typing? The OP said one thing and this reply says another. Either way, both are nice in distributed systems too.
If I'm dealing with some complex long-lived distributed system, I don't have time for the bullshit errors no static type checking allows. I'm already waist-deep with real problems!
Also if you control the client and the server, do not just blindly follow Postel's law and except crappy data. Be rigid and lo, and behold, one side quickly excises all the bugs in the other.
> Also if you control the client and the server, do not just blindly follow Postel's law and except crappy data.
But in real world, we do not livd in a bubble. Very often, we do not have the comfort of controlling the server and the client. We have keep edge cases in mind. Unexpected errors / cases happen.
How?
So whenever something was expected to arrive from the wire, it does not parse correctly, depending on the parsing method, once you have that null, all bets are off. Type system gave up.
It all depends.
Another example is Java interfaces. Merely checking for an interface type does not prevent errors down the line. It does limit the amount of errors.
They are opt in of course, but have the property that the more specifications you add and the more precise they are, the more type errors they'd discover. After it run it tells you if there is a type discrepancy. If it can't decide it doesn't say anything.
The technical term for that is "success typing" http://www.it.uu.se/research/group/hipe/papers/succ_types.pd...
Erlang is not in my opinion a “dynamic language gets us going fast” language. It's a language specifically designed to handle large scale reliably for long periods of time.
You will get a lot of milage out the Erlang ecosystem.
IIRC message passing greatly complicates things.
Types are great at stating and preserving global invariants. Invariants sometimes do vary as systems evolve, though. You may need to handle polymorphism in your data flow mid-transition. And runtime polymorphism is just another phrase for dynamic typing.
Yes, you can add enough expressive power to the typing system to make this work (e.g. dependent types), but then it stops being pragmatically viable, as far as we know in 2019.
- No type inference, since that's not decidable for dependent types.
- In most industrial software engineering, getting the specifications is a big problem, whence agile methods. Without detailed specifications early on in a software project's lifecycle, what's the point of paying the price for type dependency when you don't even use it.
- Relatedly, dependent types don't solve the oracle problem in software engineering: i.e. where do you get the specs from and how do you know that your specs (in this case types) are correct rather than false?
- Rewriting & refactoring code becomes much more expensive, because you now also have to rewrite / refactor the specifications. (This is already a reason why post-Java, exception specifications have been abandoned)
- No widely used programming language offers full dependent types (yes I'm aware of Scala's path dependent types and Haskell's forays into dependency), and that despite dependent types are at least half a century old [1]. So it's not like programming language designers don't know about them.
- Dependent types have really only been worked out beyond research prototypes for functional languages, not e.g. for message passing concurrency.
My experience with them so far
has been more than great.
[1] N. de Bruijn, Automath, a language for mathematics.
You can still have type inference, it just won't be able to infer the type of all valid terms, in which case you just have to add a manual annotation. This is not a strange concept, this is what haskell does for example with higher ranked types.
> getting the specifications is a big problem
You do not need to specify what the whole program will do (in fact you can use a language with dependent types without having to specify anything at all) - that being said, people usually specify the behavior of specific functions in the program.
> - Relatedly, dependent types don't solve the oracle problem in software engineering: i.e. where do you get the specs from and how do you know that your specs (in this case types) are correct rather than false?
I thought that the point of erlang having dependent types would be to make data-dependent code easier, not to specify the behavior of functions. But yes, this is correct, and this holds true with every proof assistant that I am aware, not only with the ones that use dependent types.
> because you now also have to rewrite / refactor the specifications
This is a great thing actually - I would consider it a bug if my function after a refactoring did something that went against its specification, with dependent types I can have the compiler warn me about it.
But again, you are not forced to write a specification. Dependent types only add possibilities without removing anything.
> So it's not like programming language designers don't know about them.
I would argue that most programming language designers are not familiar with type theory. There are thousands of languages, one does not need to be a genius to make one. Heck, two of the most popular languages (C and Go) do not even have parametric polymorphism, not to mention that generics were added in Java only in 2004. In addition to that pretty much no popular language has proper support for higher order types either.
> industrial code, 20k LoCs, at least 3 programmers involved, specification changed over time. Specification was not fully available at the start of the project.
In that case, none. Though I would argue that your standards are too high, after all just the fact that I have not worked in the software industry excludes anything that I have made.
As for LoC, I would not use it to measure the scale of a project. With Java for example even the most trivial program can easily reach thousands of lines.
most programming language designers
are not familiar with type theory.
parametric polymorphism
I have not worked in the software industry
> Though I would argue that your standards are too high
Ahaha what? 20kLoCs and 3 programmers is a college-level one-off project. Industrial code (and industrial projects) routinely involve orders of magnitude more code and orders of magnitude more programmers.
So no, "anything that you have made" doesn't count until yes, it's not just you working on your own project that you know inside out.
So yes, when you write "My experience with them so far has been more than great" and you can't provide evidence that they work even for a measly 20k-LoC-project with three people, your experience is entirely irrelevant.
Dynamic doesn't really cause problems in its own. Only abuse of it which can be said about most things.
I guess there is an argument that f forces you to have good habits but that comes at a pretty big cost if your team is half decent.
I said don't abuse language features. That is a far cry from writing perfect code.
Non perfect code is expected early in a startup, I'm just saying don't confuse your non perfect code with a language failure.
I'd also like to say if you've never seen "hire perfect developers" scale passed one you've been optimising for the wrong thing.
I've seen a team go from technical debt hell and dread to a powerhouse team where everyone commits daily.
It doesn't take much, and it doesn't take perfect developers, and it certainly doesn't have anything to do with the language.
It had everything to do with honest and healthy code review. It had everything to do with people feeling comfortable enough to say, your solution is a hack, why are you taking this shortcut.
That isn't write perfect code, or hire the right people. That is hire pretty much whoever will take the job. That is prioritising and optimising for quality.
We still had a legacy code base, it still made us money, and it gave us the time opportunity to do things right, but you still need to take the time.
Note I said "now let's scale" -- it's pretty easy to rely on good practices and a steady hand when it's a dozen engineers in a room. When it's a thousand engineers in three timezones it's a fair bit harder.
Edit: typo
You can move slow, choosing a static language only imposes that on you, it doesn't fix the underlying problem (developers being lazy.)
The nice thing about erlang though is that the nature of the message passing interface and pattern matching means that you can get relatively understandable interfaces.
It's done by writing your handlers to accept a closure to use for future requests. It's not inherently different from doing this in Java:
if (req.isUpdate()) {
handler = Class.forName(req.getUpdateName()).newInstance();
} else {
handler.handle(req);
}
- Given a distributed Erlang system with two nodes, where both nodes are running the same release;
- Given the same process (Erlang thread) on both nodes, running the same code (i.e. a GenServer backed by the same module)
- Given a new version of the release is being applied
You can't assume that the two processes will be upgraded at the same time. This means that messages sent between those processes may violate the types expected by one version or the other. Furthermore, in Erlang, any process can send a message to another process, so there is no way to enforce that the data a particular `receive` expression gets will even be a type defined in the code it is running. Furthermore, even on the same node, during an upgrade, some parts of the system are still running old code, while some parts are running new code, so it isn't even specifically about distribution.
There is a whole area of type system research around session types, which are designed for more or less this use case, but from what I've seen, none of them handle the case of arbitrary messages, or the case where system upgrades are being rolled out and old code may receive messages from new code.
I still think there is a place for a type system in Erlang/Elixir, but it would have to deliberately ignore process messaging at the very least, at least until a type theoretic solution is available.
I think that this ideas generalizes pretty well. Just because a language doesn't have explicit types doesn't mean types don't emerge organically. Assumptions will still be made about certain fields existing, there's just no guarantee that the assumptions still hold!