Are dynamic languages going to replace static languages? (2003)
artima.com
artima.com
I'm the same with strong static typing. At its most basic, why would I give up getting the computer to automatically check for me at compile time that I'm passing the right number of parameters to a function, that a parameter is an array and not a string, a field can never be null, and that I haven't made a typo in a variable name? Coding without these trivial checks of correctness feels archaic and backwards to me.
To make up for it in a dynamic language, you end up having to reimplement what a type checker would do for you with no guarantee you didn't miss something or make a mistake, with the added cost of e.g. polluting your projects with noisy defensive code that double checks the types of parameters, adding extra tests, and bolting on linting, analysis and IDE tools to help.
I'm with the other posters too about the article content, if you haven't coded in a language like OCaml, Haskell, ML etc. you really don't know enough about what strong static types offer to discount them. Java, C and C++ have much weaker checks of correctness and lack type inference + pattern matching that make strong types pleasant to work with. The article says more about e.g. Java vs Python.
You want strong static types + tests, they're not mutually exclusive at all but complementary.
Same argument could be made the other way, if you haven't used Smalltalk, Clojure (and other languages that are strongly dynamic), you shouldn't discount them.
Que?
class A(random.choice([B, C]):
... class Rectangle:
var w,h
class Square(Rectangle if w == h):
...
The last thing: being able to edit program code, add new types/members/functionality and so-on while it's running... that's not really a language limitation nor typing-system limitation either: that's just good tooling that is able to hot-swap executable code after the process has already started. Visual Studio has had "Edit & Continue" in debugging sessions for almost 3 decades now, and the runtime build system used by ASP.NET means that source code and raw aspx/cshtml views can still be FTP'd up to production and just work, because .NET supports runtime compilation.
------------
For ordinary business software, where the software's focus is on data-modelling and data-transforming-processes, so all these programs basically just pass giant aggregates of business data around, so strong-typing is essential to ensure we don't lose any properties/fields/values as these aggregate objects travel through the software-proper. In this situation there is absolutely no "expressiveness" that's could be gained by simply giving up on type-correctness and instead rely on unit tests (and hope you have 100% test coverage).
There is also ways of dealing with this in dynamic languages. For example, Clojure has clojure.spec that achieves the same results, but without relying on strong typing.
In the end, it's all tradeoffs. There is no "one solution" that will be perfect for everything.
It's more complex than that. The language absolutely has to define these operations for them to make any sense. Modifying some code inside a function is the easy case, and supported by many environments. Modifying types is much more difficult, and usually not supported (.NET and JVM do not support any kind of type modification - neither field nor function). I'm not sure if the C++ debugger can handle this either. Common Lisp can do it, and it actually defines how existing objects behave if their type changes.
By contrast, most other languages only have eval() - which takes a string and parses and executes it as code. In Lisp, eval() takes a Lisp list object and executes that.
For eg.
>> code: load "foreach n [1 2 3 4] [print n * 2]"
== [foreach n [1 2 3 4] [print n * 2]
]
>> type? code
== block!
>> do code
2
4
6
8https://vvvvalvalval.github.io/posts/what-makes-a-good-repl....
> If you're not familiar with Clojure, you may be surprised that I describe the REPL as Clojure's most differentiating feature: after all, most industrial programming languages come with REPLs or 'shells' these days (including Python, Ruby, Javascript, PHP, Scala, Haskell, ...). However, I've never managed to reproduced the productive REPL workflow I had in Clojure with those languages; the truth is that not all REPLs are created equal.
Basically, the "REPL" you usually call a REPL is in fact a shell, and not a REPL as normally used in lisp languages.
Python 3.9.0 (default, Oct 10 2020, 11:40:52)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+"1"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +:
'int' and 'str'
Welcome to Node.js v16.0.0.
Type ".help" for more information.
> 1 + "1"
'11'JavaScript's more forgiving, and often surprising, implicit conversions are still rigidly-defined .
Remember that implicit conversions should never (ostensibly) throw any exception or cause any errors because implicit conversion should always succeed because it should only be used to represent an obvious and safe narrowing-conversion. A narrowing-conversion necessarily involves data-loss, but it's safe because all of the data needed by the rest of the program is being provably retained.
29.5 Types Versus Safety
"Programming and Programming Languages"
https://papl.cs.brown.edu/2020/safety-soundness.html#%28part...
I think it's more germane to refer to that as weak implicit type casting.
This doesn't feel like a very useful definition. What languages would fall under the category of "weakly typed" by your definition? I can only think of C, and even that is only true for a subset of the language (when you're messing about with pointers).
> Remember that implicit conversions should never (ostensibly) throw any exception or cause any errors because implicit conversion should always succeed because it should only be used to represent an obvious and safe narrowing-conversion.
> const fun = (x, y) => (x + y) / y
> fun(1, 2)
1.5
> fun("1", 2)
6
I'm, not sure if Smalltalk and Clojure do this better, but since I've gotten into static languages, I basically haven't looked back, and I haven't missed the dynamism much at all.
Yes, languages that are truly dynamic (ships with a REPL, has a instant change->run this snippet workflow, can redefine anything in the runtime AT runtime) does work a lot better than Python which is basically comparing the type systems of Java and Haskell.
Give it a try. Worst that can happen is that you now have tried and learned something different, I'm sure it'll give you some knowledge that'll be helpful in any programming language :)
Everything Is An Object in Smalltalk, and of course this can't be true in Objective-C due to its mixed heritage.
Pharo is a nice implementation of Smalltalk and they have a nice page that describes the top features of the language. Take a look and see if it's different from the developer experience you had with Objective-C: https://archive.is/uAWX5 (linking to a archive via archive.is as some images couldn't load)
Clojure has some cases where it's insane how elegant you can make the solution, but frankly static languages with good type systems and tooling come close enough but don't have the scaling downsides.
Yeah, agree. But this is hardly surprising. Learn enough about ANY language on this planet and you'll come to the same conclusion.
The beauty of Clojure is that it mostly provides you with tooling and other facilities that leads to elegant and more importantly simple code.
Java can put you off with all the boiler plate. That's not a requirement of static typing.
F# skirts-around some of the limitations with just-in-time design-time type generation ( https://docs.microsoft.com/en-us/dotnet/fsharp/tutorials/typ... ) though Microsoft pitches it as a "better T4" for F# specifically, I think it's far more powerful.
The way things are looking, C# is set up get some form of higher-kinded-types in the next 3-5 years (hurrah!) - but from what Mads and the design-team have said it seems like they'll be implemented without significant changes to the CLR's type-system - so be prepared for everything to be implemented as glorified structs (and don't expect struct inheritance either...) with excessive amounts of implicit-conversion.
The feature you state (F# unions) - that simplification has some positives as well. I can exhaustive pattern match for example. I can always use composition/pattern matching/active patterns to mix in other assemblies types cases. Is it really limiting that it doesn't do this? Would changing this mean more complexity and make other language features harder to deliver? All features introduce some additional complexity, and it more than linear typically. F# also has some interesting patterns that were either introduced first there en masse (e.g Async) or are easier to use there.
I'm a fan of static checking, and conciseness with performance close to the platform it is hosted on and is easy to reason about. I'm also a fan of easy to read and clean code that isn't "scary" to look at. F# seems to strike that balance well compared to other JIT functional languages, at least from where I’m sitting. Too much complexity and you start scaring people away and/or too much abstraction in the codebase starts to occur trading off maintainability.
TypeScript, by design, does not produce nor introduce any "noisy defensive code": the point of TypeScript was that type-safety in JavaScript can be achieved without any runtime cost.
The only significant code-gen that TypeScript does do by itself is stuff relating to ECMAScript modules (e.g. wrapping and repackaging your TypeScript module for other module systems).
...with the exception of when you take a modern TypeScript compiler and specifically instruct it to target ES3 (lol) or perhaps some older variants of ES5 - at which point the code `tsc` generates is still so minimal you'll still need to bring-your-own polyfill/monkeypatch libraries. This is stuff like reimplementing the spread-operator or `for(of)`, and so on.
the web runs on JavaScript because it's extremely forgiving and therefore stable. there can be lots of errors and things keep chugging along.
...and when types are within TypeScript's sphere-of-influence the compiler is largely able to deduce types without needing assertions or type-checks.
...or so you think! But there's two problems:
1. TypeScript's type system is unsound: so there are times when the compiler will let you do some (for example) contravariant array mutation, even though that's almost essentially guaranteed to crash at runtime if the rest of the system makes incorrect assumptions about the type-variance of the array/collection. So having the type-guard function available to assert to tsc that whatever array operation you performed is safe, you wouldn't need to do that in pure JS.
2. Type-guards are essential for when you're working with values that pass through an interface (because that necessarily means some loss of typing information has occurred), so the type-guard function is needed to recover that static information - and while this approach does also apply to JS, in some situations where if your code is the factory of subclasses that pass through an interface (e.g. so `MyComplicatedSubclassForRockAndRoll` appears as `MusicSubclass`) then in JS a blind "reinterpret-cast" of the object is fine - but TypeScript really doesn't want you doing that and again, having a type-guard function is just to keep the compiler happy - and even if it is possible to mathematically prove that the `object` type-cast is correct (because theoretically static analysis can see on both sides of an interface) but TypeScript isn't perfect and there are cases where we really do know better than the compiler, but we don't want to just make it unsafe and untyped - instead we want to keep the typing and we want to keep the compiler happy, but doing it this way (a type-guard) is not only easier than suppressing warnings and ignoring your sense of guilt but they're invariably genuinely useful a few months down the line when someone made a change that the type-system spotted and prevented from compiling (and so going into production), whereas it would be very easy to (for example) unintentionally introduce vulnerabilities because even with near-100% test coverage, because adding brand new functionality to a project (even if it said functionality doesn't work) it won't cause any tests to fail (at least, until tests for the new functionality are completed).
If it's an external API, do the same thing except it probably needs a more manual approach.
This is not something that's in anyway unique to typescript. The difference is JavaScript is designed to fail gracefully using coercion whereas most other languages and runtimes abort and throw exceptions.
I don't understand. All languages, without exception, must compile down to a "worthless untyped jargon". Only assembly can execute.
Javascript is a garbage layer of broken standards, a worst of breed language that shouldn't have to exist between a strict typed higher level language and lower level bytecode, but everything on the web now needs to be built on top of it, compile to it first, and run through one of a few JS engines. You can never build a full fledged language on top of something that has to go through JS... so TS, coffescript, et al are always going to be pseudo typed but not truly solid or permanent solutions.
Ultimately I think ECMA standards will finally move toward strict types in JS, and in a decade more we'll have something that for all intents and purposes is exactly what Flash was in 2009.
Sure, but if your typed language is sound you shouldn't have type errors at runtime. Not having type at runtime is not a problem, type erasure is nothing new and works really well in OCaml.
> You can never build a full fledged language on top of something that has to go through JS... so TS, coffescript, et al are always going to be pseudo typed but not truly solid or permanent solutions.
Rescript and Elm are really solid, OCaml can compile to JS too, Scala and F# too. Granted, only Elm has been built on JS at first, others were adapted (Rescript is like half and half), but they're all solid.
> Ultimately I think ECMA standards will finally move toward strict types in JS, and in a decade more we'll have something that for all intents and purposes is exactly what Flash was in 2009.
I think the idea here is to offer WASM as a compilation target. Trying to gradually type JS leads to really complex systems (TS types) because people did all sorts of crazy things with JS. I wouldn't like this to be the base on which we build other languages, or the web in general.
In those cases, the difference between a compile-time check and a run-time check is much smaller.
I also find 'phantom types' incredibly useful. For example, we might have a database or key/value store, where the keys are represented as strings and the values can be strings, integers, floats, booleans, etc. We can 'label' each key with a "phantom type" corresponding to the values. This way, the compiler knows exactly what types are expected, we only have to handle the expected type, we don't need any boilerplate to wrap/unwrap the database results, etc.:
-- Haskell syntax
data DBKey t = DBKey String
dbLookup :: DBKey t -> Maybe t
dbLookup (DBKey key) = ...
name = DBKey @String "name"
age = DBKey @Int "age"
// Scala syntax
final case class DBKey[T](override val toString: String)
def dbLookup[T](key: DBKey[T]): Try[T] = ...
val name = DBKey[String]("name")
val age = DBKey[Int ]("age" )
Of course, we can combine such 'labels' with more meaningful types; so that 'name' looks up a 'Username'; 'age' looks up an 'Age', or a 'Nat', or a 'Duration'; or whatever's convenient. It can also be combined with other type system features like implicits/ad-hoc-/subtype-polymorphism, e.g. to decode JSON as its read from the DB, etc.
I've used the example above in real Scala applications when accessing a key/value store. I've also used phantom types (again in Scala) to help me manipulate Array[Byte] values: in that case the bytes could either be 'Plaintext' or 'Encrypted', they could represent either a 'Payload' or a 'Key', and each 'Key' could be intended for either 'Encryption' or 'Signing'. Under the hood these were all just arrays of bytes, and the runtime would treat them as such (using Scala's 'AnyVal' to avoid actually wrapping values up), but these stronger types made things much clearer and safer.
For example, we can decrypt and verify a message which carries its own 'data keys' (encrypted with a 'master key'):
def decrypt[T](
key : Bytes[Plaintext, Key[Encryption]],
data: Bytes[Encrypted, T],
): Try[Bytes[Plaintext, T]] = ...
def validated[T](
key : Bytes[Plaintext, Key[Signing]],
data: Bytes[Plaintext, T],
): Try[Bytes[Plaintext, T]] = ...
def decryptMessage[T](
master : Bytes[Plaintext, Key[Encryption]],
dataEnc: Bytes[Encrypted, Key[Encryption]],
dataSig: Bytes[Encrypted, Key[Signing ]],
message: Bytes[Encrypted, T ],
): Try[Bytes[Plaintext, T]] = for {
encKey <- decrypt(master, dataEnc)
sigKey <- decrypt(master, dataSig)
raw <- decrypt(encKey, message)
valid <- validated(sigKey, raw)
} yield valid
As you say, with ADT's and null safety, you can be a lot more judicious about testing to get to the same level of confidence.
When working with Rust or Swift, I'm honestly usually getting away with some integration tests, and whichever unit tests came about organically when a particular section was particularly well suited to TDD. But "code coverage" is a much less pressing concern.
I should also note, I've attempted OOP in TS and it wasn't pretty. It is much more suited to a functional style IME, although I will concede I have seen some open source projects that use it in a Java-esque OOP style presumably to good effect. Personally it wasn't my cup of tea.
Expand please?
If you are curious about this, you might be interested in checking out Haxe [0]. It has static typing, type inference, pattern matching [1] (maybe not on par with ML languages, but still good), and can transpile to multiple targets [2] including JS.
[0]: https://haxe.org [1]: https://haxe.org/manual/lf-pattern-matching.html [2]: https://haxe.org/documentation/introduction/compiler-targets...
[1] https://rescript-lang.org/
[2] https://github.com/melange-re/melange
Anybody who last looked at TypeScript a few years ago and dismissed it needs to have another look.
See what I did there?
In contrast, a compiles-to-JS language can produce code that behaves unexpectedly in its non-functional requirements, and in the worst cases crashes where it shouldn't. That will be less of an issue as compilers get better, and WASM should help a lot, but in the current ecosystem TypeScript produces more predictably good results.
It's a very complex type system though.
From my perspective, I feel user-facing complexity is quite low in TypeScript in some key ways. You can write your type declarations anywhere, you can import and export them in a familiar JS-module way, you don't need to explicitly type anything in particular so you can be anywhere on the spectrum from very rigorous types to no types at all / `any` everywhere, and finally it has really good error messages so you rarely are stuck with a type mismatch that makes no sense.
Even when using modern stuff correctly on a mature codebase fuzz testing will find bugs. So if you don't fuzz test and someone evil has your binary then they'll fuzz test for you...
Testing can't really substitute for types. They're two diametrically opposed approaches to impeding bugs.
They obviously work best when "combined with test coverage" but then again, I dont consider test coverage to be "optional" in any language. This idea that "if it compiles it works" that some haskellers seem to believe, for instance, is complete horseshit.
If somebody argues that static typing works great because without tests it catches more bugs than dynamic typing does when you also dont write tests, it's notionally true, yes, but theyre telling you more about themselves than they are about types. Namely that they're not in the habit of writing tests.
If you've got enough of the right kind of testing coverage to validate an app written in a statically typed language you will have enough to trigger invariants in a dynamically typed langauge.
I have successfully had invariants usefully triggered in prod and by manual testers in big balls of mud without test coverage (failing fast isnt just for test environments) but I considered that to be a form of emergency programming CPR and damage limitation that isnt a substitute for fixing your testing "diet".
(... I once had a professor who believed that phrase as nearly a point of religion, and his project work was maddening. No documentation on any functions either, because "the types are self-documenting." Then you'd get hit with a 'division' function that takes (number,number) but he put the denominator as the first argument. Ugh.)
Clojure does have spec (https://clojure.org/about/spec) which is pretty cool and declarative. And, of course, your IDE can help you when writing code that obviously violates a spec declaration. But that's not fundamentally different from an IDE parsing the type-hints of Python, JavaScript, or PHP. It is more powerful than the aforementioned type-hints, though.
At the end of the day, it blurs the line between static typing and dynamic typing. IMO it should be considered static typing because it serves that same purpose, works at "write time" to help the programmers, and can probably absolve you of writing your own type-checking tests.
In some sense, it is all just asserts. Even statically typed languages have unsafe type casting, which can trigger an "assert" at runtime.
In a dynamic language you end up writing a number of tests that would be covered by a static type system and writing a number of tests that would not be covered by that. Going from dynamic to static typing is a replacement for some of the former, and none of the latter.
Not once have I done this. I've written plenty of invariants that would have been covered by a static type system but no tests.
As far as I'm concerned if you had to write an extra test to "cover for a dynamic type system" that test is a bad smell that should be deleted. A test validates that a method raises an exception when being fed a null is pointless if it's just testing a one line assertion that does the same thing. It's a waste of keystrokes. It doesnt catch bugs.
There may be some exceptions to this rule but they would be under very obscure circumstances, I think.
If you have that “pointless” test, you’ll find out about it. If not, you’ll probably only encounter it at runtime with a program that is subtly wrong.
Whether it's a good or a bad decision is another matter and not something the test will help with.
Tests that exist to check for the presence of a single line of code are deadweight.
Failure to sanity-check a null almost never stemmed from someone deleting the non-null assertion intentionally. It stemmed from someone rewiring the control flow in such a way as to bypass the non-null assertion without intending to (such as early-returning because they think they can shortcut the computation, but the shortcut isn't actually valid if the argument in question is null).
The tests have value precisely because they aren't in the regular control flow of the unit under test and will fail when the code flow is changed and the author didn't realize the change modified the expected behavior of the function.
Unit tests that test the contract of a method (if you put in this argument, then you’ll get this return or side effect) are a good idea in both dynamic and static languages.
That reason will be one of two things: (1) They replaced it with something they thought would handle the same functionality (along with possibly some other functionality) in a beter way, or (2) The requirement changed.
> That reason will make them remove the test that makes it fail as well.
If it was #2, sure (and in a test-first workflow, they would take out the test first). In case #1, they won’t take out the test, and if their implementation is wrong, the test will let them know.
> Tests that exist to check for the presence of a single line of code are deadweight.
Unless you have a very weird framework, tests don’t check the existence of lines of code, they check the behavior of units. The fact that the first-pass implementation in the same iteration in which the test was added is trivial doesn’t mean the function isn’t part of the contract that needs verified, and doesn’t mean that no one will ever change the intended implementation to a less direct one.
Do these invariants have some language-level support?
> There may be some exceptions to this rule but they would be under very obscure circumstances, I think.
Maybe it depends highly on the specific dynamic and static languages under comparison. My main experience with dynamic languages has been with Groovy (all inherited codebases), and it's an annoyingly frequent occurrence that I've had to deal with PROD issues for things that would have been compile errors in Java.
I believe my favorite is when someone used a String for the counter in a for loop, and it worked...until one day the counter had to go past 9. IIRC, Groovy would convert the String to a char, increment it, then convert that into an Integer. That actually works for 0-9, but incrementing "9" in this way gives you ":" which would then get a NumberFormatException in the last conversion.
It catches the bug that gets created when someone does a later refactor, replaces the assertion and some other code with a call to another function that they think throws an exception on null as well as doing some computation needed in the current function, but they are wrong.
Tests validating the contract of the unit are useful even when the trivially pass in the first-pass implementation, because implementations evolve, and the thing people will forget to add tests for with changes are the pre-existing, unmodified expected behavior.
(Also, for workflows where tests are written before the first-pass implementation.)
A common misconception.
In tests, you test that values are correct, the types take care of themselves.
Types being correct does not imply values being correct, so you will have to write those tests anyhow, although you maybe be fooled by the safyness of static typing[1] that you don't have to.
Values being correct does imply that the types are correct, so if your tests pass, then your types are also obviously correct. (If a==3 then I don't have to check if a is of type pink elephant, it obviously isn't).
So that's the in-principle argument. And it turns out that this is completely supported in practice as well: neither the defensive code nor the extra type-based tests that you propose must be the result actually occur in dynamic code-bases with any frequency.
That said, static types certainly have their uses, particularly in being machine-checked documentation and also for producing faster code without going through crazy amounts of extra effort.
[1] https://blog.metaobject.com/2014/06/the-safyness-of-static-t...
If I generate values within a function that are arguments to a constructor call, and the make the constructor call and return the result with no further manipulation, I can—without exhaustive, or even any—testing be quite confident that the function will return the type for which the constructor so called is the constructor any time it returns, and fail otherwise.
But when you start having loops, conditions, arrays, nullable references etc. you can't be so confident any more unlike with proper type checking where you can be fully confident.
What you have, after manual inspection of the code, is some sort of informal and not written down correctness proof. A machine tested proof would be better and types help with that.
> In tests, you test that values are correct, the types take care of themselves.
How can you be sure your function e.g. returns an integer for all values and never undefined for some values? Unit tests cannot check this exhaustively unlike type checkers.
> Types being correct does not imply values being correct, so you will have to write those tests anyhow, although you maybe be fooled by the safyness of static typing[1] that you don't have to.
As I said, types compliment tests, they don't replace them but types do reduce some of the burden on writing tests for you.
To nitpick: type checking can prove values are correct when the type system is powerful enough (i.e. dependent types) but this isn't practical enough for mainstream yet.
In particular, we can use dependent types to place much tighter constraints on critical pieces of code, but 'cut short' a full correctness proof by simply punting to an error case when it's convenient.
For example, say we need to manipulate the Nth element of a collection. We can express indices with a type 'Fin X', whose values are equivalent to 1, 2, ..., X, e.g.
f1 : (c: List Foo) -> (n: Fin (length c)) -> BAR
Yet this places a burden on anyone who calls this 'f1' function: they need to prove that the index appears within the list (and, in doing so, prove that the list is non-empty). This might be very difficult in the general case, yet a full correctness proof would require it.
However, we don't need a full correctness proof; so we can just check if we're in bounds, and return an error if not:
f2 : List Foo -> Int -> Either String BAR
f2 c i = case mkFin (length c) i of
Just i -> Right (f1 c i)
Nothing -> Left ("Index " + show i + " not in list of length " + show (length c))
where mkFin : (n: Nat) -> (i: Int) -> Maybe (Fin n)
mkFin Z _ = Nothing
mkFin _ i | i < 0 = Nothing
mkFin (S n) 0 = Just fZ
mkFin (S n) i = map fS (mkFin n (i - 1))
Dependent types are really useful for propagating error conditions 'backwards' through the code like this. When practical, I much prefer this to propagating error conditions 'forwards', by returning Maybe, etc. Most applications start with some sort of parsing step, which is a great place to perform such checks, if we can manage to propagate them back that far. Otherwise, we just handle these error cases just like we would any others (dependent types or not).
Right, which is actually an argument against the claim that you have to write more tests with dynamic typing.
Urgh, that reference is such a shallow, straw-man facade of static typing. This is especially evident with its link to https://www.sans.org/top25-software-errors/ and claim that those "don't look like type errors to me", a list which includes:
- 16 even says 'type' in the title: "Unrestricted Upload of File with Dangerous Type"! Phrased as a type error, this would be "savePicture: Required argument of type 'Either[PNG, JPEG]', given 'Array[Byte]' instead"
- 2, 3, 6, 11, 18 are all injection attacks, which are classic type errors (AKA "stringly typed programming"). Phrased as a type error, these would be "+: Cannot append 'Foo' with 'FormInput'", where 'Foo' might be 'SQL', 'ShellCommand', etc.
- NULL pointer dereference is Tony Hoare's 'billion dollar mistake', which as type error would be "Expected 'Foo', given 'null'"
- integer over/underflow, buffer over/underflow, out-of-bounds access, etc. can all be mitigated with types like 'Maybe', 'Try', etc. at the cost of a bounds/flag check. If we don't want runtime checks, we can instead track memory usage bounds in the types, and have the compiler check that we're allocating enough.
So many of these problems are avoidable with types, even before we go into fancy stuff like dependent types, linear types, etc. In particular, most of the burden to getting this right is on languages and libraries, not application developers.
For example, if a database library can only be queried by an 'SQL' type, and the only way to construct that type is via literals and a 'withParam[T]: SQL -> T -> SQL' function, then application developers are unable to introduce injection attacks; the library forces them to do the right thing. Instead we get 'dbQuery: String -> ...', 'runShell: String -> ...', 'response.withBody: String -> ...', etc. and draw the conclusion that types can't help :(
The SQL case is much more complex. SQL is such a non-standardized language in practice, and such a complex one, that trying to offer a generic SQL library that presents an AST as the API for even the most popular RDBMSs quickly gets complex. So, you end up with convenient string->SQL converters, and then people stick untrusted input in those strings. Not to mention, your fancy AST library still needs to spit out SQL text to talk to most DBs, so you end up needing to sanitize identifiers in your own SQL->string conversion.
The problem with stringly-typing is the model, which is SQL as strings. Statically type-checking that all your strings are, in fact, strings does not buy you anything whatsoever in this scenario.
On the other hand, if you have an actual model of the SQL, then it also doesn't matter whether that model is checked at runtime (dynamic typing) or at compile-time (static typing). In either case it will not allow injection attacks, if implemented correctly.
And of course the actual place where nastiness happens is the user input, and I haven't yet seen a place that can statically typecheck users. ;-). So, as you point out, you will need to do dynamic checking and sanitising of input.
User input has type `ByteString`. Not only can we check that statically, we absolutely should; and we should enforce that type, to reject any code which assumes otherwise.
Nastiness happens when developers treat user input as something other than ByteString; e.g. they might try appending it to a fragment of HTML (XSS); or wrap it in quotation marks and send it to a database (SQL injection).
We don't need to 'statically typecheck users'; we do need to statically check that `myApp` has type `ByteString -> Foo`, so we can avoid ever executing implementations which actually have type `HTMLFragment -> Foo` (XSS), or `SQLFragment -> Foo` (SQL injection), or whatever.
It's possible to implement anything correctly with dynamic types but the point is static typing makes it easier to do so. Static types will exhaustively check for you that certain errors aren't possible (vs tests that check a finite number of cases) + compile-time (vs at runtime when the error actually occurs on particular inputs).
> And of course the actual place where nastiness happens is the user input, and I haven't yet seen a place that can statically typecheck users. ;-). So, as you point out, you will need to do dynamic checking and sanitising of input.
It sounds like you're talking about dynamic (type) checking here when you really mean regular runtime behaviour? You absolutely can at compile-time make sure your program is going to respond in a sensible way to problematic user inputs at runtime e.g. the compiler verifies that either the user input string value will get turned into a `valid email` value at runtime or the user is asked to try again. I wouldn't call the condition that checks if the string is a valid email a dynamic type check.
I don't see how user input is different or special from any other value either e.g. it's not like you can predict what exact values are going to come from files, random number generators or from complex calculations during runtime, you just care that they're in the range you expect.
No. Quite the opposite. People here confuse "static typing" with "checking stuff", even such obviously dynamic things as "checking outside input".
> You absolutely can at compile-time make sure your program is going to respond in a sensible way to problematic user inputs at runtime
Right. Sort of. But that is just normal code. Code that absolutely, definitely has to be executed at runtime and at no other time. Because that is when the actual users of your system are going to input the actual data. Not while you're compiling stuff. Your users (and the rest of the world you interact with) aren't there while you're compiling. And your compiler isn't around when your users are inputting data.
> the compiler verifies that either the user input string value will get turned into a `valid email` value at runtime
No, the compiler cannot verify user input, because the compiler isn't running when the user is inputting data. You can implement a model that has the concept of "valid e-mail", and you can implement your system such that it converts input data such as strings into your structured model at the edges and then only deals with "valid e-mail" objects internally. But implementing such a model has nothing whatsoever to do with whether you use a statically type-checked language or a dynamically type-checked language.
It also turns out that such systems tend to be really, really bad. What you actually want to do is keep the user input exactly as it was input, for auditing purposes if nothing else, and build your structures as an enrichment on top of that basic data. Because your idea of what constitutes a "valid" e-mail is almost certainly wrong, and it's better if the system can at least represent data even if it doesn't completely understand it, rather than destroy user data.
> No, the compiler cannot verify user input, because the compiler isn't running when the user is inputting data.
I'm not following why you thought I meant that. It's like you're arguing it's impossible to make any compile-time guarantees because you don't know what exact literal inputs your program will be dealing with at runtime when the field of https://en.wikipedia.org/wiki/Formal_verification exists. E.g. at compile time you can prove a sorting function gives the correct output for all possible inputs without having to run the code - you don't know in advance what the input is going to be (which is normal when coding), but you still know the output will be correct with respect to the function specification.
1. Define SQL in your type system, and force programmer to specify for any piece of user input they want to use in a query what its semantics are supposed to be. This is the SQL DSL approach, such as the short lived LINQ-to-SQL or Haskell's Selda. ORMs also do something similar.
2. Enforce that any string sent to the DB passes through some kind of checker that enures certain properties hold for that SQL. The checker will have to understand all of the semantics of SQL, just like in 1.
There are many libraries that go through path 1, but don't support the full capabilities of SQL (usually they support a tiny subset), even for a single DB.
This is where the argument became circular and we can basically stop.
The parent claim (by chriswarbo) was that, for example, SQL injection attacks were, in fact, incontrovertible proof that SQL injection attacks "are classic type errors". That simply isn't true, at least not in the sense of static type checking (and only made true, as I mentioned above, by conflating "static type", "type" and "model" into one incoherent gooey mush).
chriswarbo's incorrect claim was in response to my referenced article, The Safyness of Static Typing [1], where I (as a prelude to my actual point) look at the evidence for the claim that "static types make it easier to implement something correctly".
There isn't any.
Or to be precise, there isn't any that passes any statistical or other standards. What little evidence there is goes both ways and has, at best, tiny effect sizes.
And yes, that article is somewhat old, but the evidentiary situation has not changed, despite further attempts to make the claim that static typing is provably better. Claims that were resoundingly debunked.
So that's the background. With this background we have the idea that SQL injection attacks are somehow evidence for the problems with dynamic typing, which they are not. I can have a statically checked string type and have exactly the same SQL injection attacks, and in principle, checking code needs to run at runtime against the dynamic input it is represented with. Which I think we agree on:
> It's possible to implement anything correctly with dynamic types
but apparently chriswarbo does not. So that brings us to the circularity of the argument:
> but the point is static typing makes it easier to do so
This hasn't been shown. It was claimed. And it was claimed that SQL injection somehow proves this. Which it doesn't, because we agree that what prevents the attack is the code that runs, the model that executes. It may be that such code is easier to write with static types, but that hasn't been shown, it is just claimed and the claim repeated in support of the claim. Circle.
[1] https://blog.metaobject.com/2014/06/the-safyness-of-static-t...
You mean the link from [1] to "An experiment about static and dynamic type systems" based on "an empirical study with 49 [undergraduate] subjects that studies the impact of a static type system for the development of a parser over 27 hours working time.". I think studies like this are more distracting here than useful (low skill level, not large scale enough, too much noise, toy example). The article here sums up a lot of my thoughts on this: https://news.ycombinator.com/item?id=27892615
And do you really need a study that proves e.g. (to pick a simpler example to summarise) a language that makes it impossible for null deferences to happen at compile-time is going to have less null dereference errors than one that lets those errors happen at runtime? It's like insisting for an empirical study that 2 + 2 = 4.
> It may be that such code is easier to write with static types, but that hasn't been shown,
The typed SQL example might look something like:
function getUserInput(): string
function createSanitisedString(s: string): SanitisedString
function createSafeSqlQuery(s: SanitisedString): SqlQuery
1. At best, the code will fail only after the user input is received by createSafeSqlQuery during runtime and you won't know where the input originated from.
2. At worst, the coder forgets to add a check like `typeof s === SanitisedString` or a call to `createSanitisedString` in `createSafeSqlQuery` and creates an injection attack.
The static type checker clearly wins here for me for safety and ease of implementing correctly. I don't need a study to know that compile-time errors are better than production runtime errors, that automated and exhaustive checks of correctness are better than the coder having to remember to add manual checks, and that it's better to know the exact line the unsanitised input came from over only knowing it comes from somewhere.
What languages have you used that have strong static type systems that you've written large programs in? Have you tried OCaml or Haskell for example?
The problem with the whole argument is that these functions are not actually enough to work with SQL, since they don't allow us to create dynamic SQL from safe strings.
Here are some ideas of why we can' use the functions you proposed:
user_filter = raw_user_input_col_name ++ " LIKE ?"
createSafeSQLQuery("SELECT * FROM my_table WHERE " ++ user_filter ++ ";") //type error
sanitizedQuery = createSanitisedString("SELECT * FROM my_table WHERE" ++ user_filter ++ ";") //should return error or quote everything
createSafeSQLQuery("SELECT * FROM my_table WHERE" ++ createSanitisedString(user_filter) ++ ";") //filter will be wrong, type error
function asSanitizedString(s: string) : SanitizedString {
return s
}
user_filter = createSanitizedString(raw_user_input_col_name ) ++ asSanitizedString(" LIKE ?")
query = asSanitizedString("SELECT * FROM my_table WHERE") ++ user_filter ++ asSanitizedString(";")
createSafeSqlQuery(query) //works, nice
createSafeSqlQuery(asSanitizedString("SELECT * FROM my_table WHERE " ++ user_col_name ++ "LIKE ?;")) //oops, this also works
If you don't believe me, go looking for a library that allows arbitrary strings as SQL, but statically ensures user input can't be used to construct an SQL query. I don't know of any one.
My bad for being unclear but I meant you would be sanitising something like the user entering "tea cups" into a shop search input form and searching your shop product database with that. I didn't mean the user would be entering SQL queries.
> But either way, you can't fix it without modeling the entirety of SQL into your type system
Using the type system to check correctness sounds good to me.
This is getting way too into the weeds about SQL anyway. Null dereference checks are a less distracting example to focus on for instance. As long as you can encode it into the type system, all my points are still relevant.
No, I understood that. But my point is that the programmer is going to be composing SQL queries, and they will be doing it based on user input. The SQL API will have a very hard time distinguishing which parts it receives are trusted programmer input and which parts are untrusted user input.
> Using the type system to check correctness sounds good to me.
Sure, but SQL is a huge language with significant differences between any 2 major DB implementations (and that depends on SQL server configurations - e.g. in MySQL you can specify via config file whether "a" is a string or an identifier). I have never seen a full SQL DSL or AST used in any SQL client library, in any language: it's just too much work.
No. I mean, yes, that is one study, but there are a lot more. They all come out essentially the same.
Danluu did a an overview article a while ago:
https://danluu.com/empirical-pl/
Note that the A large scale study of programming languages and code quality in github paper, the one that makes some of the strongest arguments for safety of the bunch, was the one that was later completely taken apart at OOPSLA 2019. Its findings, which also had very small effect sizes and statistics significance, are completely invalid.
> do you really need a study that proves ... less null references errors
1. Languages don't have null errors, programs do.
2. I don't need a study to show that a program has less than or equal null errors in a language with null safety, because the program in a language without might still have zero. If you're going to make a logical claim, then let's stick to what's logically provable. If you're going to hand-wave...well you might as well use a dynamically typed language ;-)
3. I do need a study to show that such differences matter at all. All software has bugs, if this a significant source of bugs, then a mechanism to make me not have them might matter.
4. I do need a study to show that the net effect is positive. For example, the famous Caper Jones study showed that dynamic languages such as Smalltalk and Objective-C were significantly more productive than statically typed languages, including Haskell, C++ and Java. Studies also have long shown that bugs scale linearly with code size, a very strong correlation. So let's say null-references are 1% of you total bugs, but you now write twice the amount of code, that means a move to a null-checked language will significantly increase your total bug count, despite eliminating a certain class of bugs.
(In fact other, older studies showed that type errors accounted for 10% of bugs, so if you can save just 10% of code using a dynamically typed language, you're ahead in terms of bugs).
Of course, I can also not require such studies and make an engineering judgement. And this is fine, we do it all the time because very little in software has been demonstrated much at all. But you then need to be aware that this is a subjective judgement call, and others may reasonably come to a different conclusion.
And being aware of this makes for much, much better engineering judgement, IMHO.
> 2. I don't need a study to show that a program has less than or equal null errors in a language with null safety, because the program in a language without might still have zero. If you're going to make a logical claim, then let's stick to what's logically provable. If you're going to hand-wave...well you might as well use a dynamically typed language ;-)
I think now you're just nitpicking and being uncharitable for the sake of it instead of responding to the overall point of my message I took care to write.
> Studies also have long shown that bugs scale linearly with code size, a very strong correlation. So let's say null-references are 1% of you total bugs, but you now write twice the amount of code, that means a move to a null-checked language will significantly increase your total bug count, despite eliminating a certain class of bugs.
> (In fact other, older studies showed that type errors accounted for 10% of bugs, so if you can save just 10% of code using a dynamically typed language, you're ahead in terms of bugs).
Most software engineering studies have so many confounding factors even in isolation that combing results from multiple studies like this is nonsensical and misleading. This reads like brilliant satire if I'm honest.
> And being aware of this makes for much, much better engineering judgement, IMHO.
You didn't reply to the question about if you've written large projects with strong static type systems like OCaml or Haskell. If the answer is no, they're worth learning for more awareness instead of relying on likely very limited and/or flawed empirical studies in my opinion.
Have you ever considered the validity of that study: which purports to compare programming languages, apparently without considering obvious differences in available programming tools?
Even back in the late '80s, those Smalltalk implementations provided a very integrated development environment — writing Smalltalk in a simple text editor really isn't the same ;-)
We could take some JavaScript programs —
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
— and try to transliterate them into Dart —
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
— and see how much or how little boiler plate is required when a language supports null safety, local type inference, implicit-dynamic: false ?
> This is where the argument became circular and we can basically stop.
I think any appeal to "easier" is inherently subjective. I've certainly tried to avoid making any such claims. Instead, I've mostly tried to argue that static types can forbid certain programs from ever running, and we can use that to forbid things like vulnerable SQL queries.
> > It's possible to implement anything correctly with dynamic types
> but apparently chriswarbo does not.
I never said any such thing; besides which, it's trivially the case that anything implemented with static types could also be implemented with dynamic types, since static types only exist at compile time (they are "erased" during compilation), and hence every running program is dynamically typed.
In fact, I don't think I've said anything about correct implementations at all. My point is that static types can forbid incorrect implementations. When it comes to security, that is much more important; i.e. I would much rather bang my head against a screen filled with compiler errors, than expose some insecure program to the harsh reality of the Internet.
> The parent claim (by chriswarbo) was that, for example, SQL injection attacks were, in fact, incontrovertible proof that SQL injection attacks "are classic type errors".
I've not claimed that (in fact, I'm stuggling to parse what that sentence might even mean). I claim that SQL injection attacks are "classic type errors" in the sense that:
- It's a widespread problem, easily encountered, commonly warned about, and most developers have probably done it at some point.
- It's a class of error that can be entirely ruled out at the language/API level, through use of static types. Similar to how segfaults can be entirely ruled out at the language level, using managed memory like in Python/JS/etc.
- Due to the above, it's a common example used to illustrate the usefulness of static types in blog posts, articles, etc. One which sticks in my mind is "A type-based solution to the “strings problem”: a fitting end to XSS and SQL-injection holes?" https://blog.moertel.com/posts/2006-10-18-a-type-based-solut...
- Due to the above, it's considered "folk wisdom" (or a "folk theorem") in the relevant fields (e.g. Programming Language Theory, Formal Methods, etc.)
Other examples of "classic type errors" might include:
- Mixing up integers with strings-of-digits, e.g. 'print("You were born around " + (time.now.year - input("How old are you?"))'
- Forgetting to 'unwrap' some list element, e.g. 'open(listdir("config/")).read()' instead of 'listdir("config/").map(f => open(f).read())'
- Mixing up units of measure, e.g. 'distance = speed + time' instead of 'distance = speed * time'
Basically any common mix-up between values/components in a program, where the computer could help up to spot the mix-up (perhaps entirely automatically, if type inference is involved). In the case of SQL injection, the mix-up is between 'string of bytes' and 'SQL query'.
> And yes, that article is somewhat old, but the evidentiary situation has not changed, despite further attempts to make the claim that static typing is provably better. Claims that were resoundingly debunked.
I very much appreciate when researchers attempt to ground things in a bit more empiricism! Unfortunately those particular studies just aren't looking at the sorts of things that I find relevant; in particular, those studies (and that linked blog post, and many of the comments here), seem overly-preoccupied with mundane trivialities like "string", or "int", or "SQLColumnName".
Personally, I'm much more interested in how types can help me:
- Avoid leaking secret information https://link.springer.com/chapter/10.1007/978-3-030-17138-4_...
- Guarantee big-O resource usage https://twanvl.nl/blog/agda/sorting#a-monad-for-keeping-trac...
- Guarantee the correct order of operations https://docs.idris-lang.org/en/latest/st/machines.html
- Prevent AI agents from losing capabilities http://chriswarbo.net/projects/powerplay
> It's a class of error that can be entirely ruled out at the language/API level, through use of static types.
Repeating this claim doesn't make it true.
Once again: this has nothing to do with static vs. dynamic types, and everything to do with modelling.
To make this clear, let's compare a dynamically and a statically type-checked version of this with the model of SQL as just strings.
Both the dynamically and the statically type-checked version of this program will be susceptible to SQL injection attacks. The statically type-checked version will verify at compile time that all the values are indeed strings.
Now let's compare a dynamically and a statically checked program with a proper model for the SQL, not strings.
Both of these will not be susceptible to SQL injection attacks.
It has nothing to do with static vs. dynamic, and everything with how you model the problem.
My gut tells me that's not quite right (for some reasonable definition of 'usable'), since we can always build elimination forms into the language (after all, 'null' must be built in, and few take objection to building in elimination forms like if/then/else).
> it also requires some kind of pattern matching or monad comprehensions - otherwise, code using Optional is neither type safe nor readable.
I really like comprehensions in Python, but hardly ever use them elsewhere. In fact, when I do use them in Haskell and Scala, I usually find myself having to refactor them into calls to `map`/`join`/etc. soon after, to have more fine-grained scoping or somesuch.
> trying to offer a generic SQL library that presents an AST as the API for even the most popular RDBMSs quickly gets complex
That's orthogonal to anything I said. Represent SQL using arrays of bytes in memory if you like; offer a string-like interface for constructing and manipulating them if you like; just make sure to distinguish them from other string-like types, in a way that's statically enforceable, and where the only API to convert a 'String' to an 'SQL' is the escaping function.
Note that I don't particularly care if an SQL value is valid; it would be nice to statically enforce that, but you're right that (vendor-supplied) SQL is pretty complex in its own right. What's much more important to get right is that SQL is not user-generated.
True, you could go the Go array route and have Optional be a special type that is generic, without otherwise supporting generic data structures.
> I really like comprehensions in Python, but hardly ever use them elsewhere.
I think code like possiblyMissing.map(value => code) introduces lots of unpleasant imbrication, and breaking out functions for every possibly-missing value also seems to me to lead to bad readability. I suppose this may be a matter of taste, or it may be related to other language features.
> just make sure to distinguish them from other string-like types, in a way that's statically enforceable, and where the only API to convert a 'String' to an 'SQL' is the escaping function.
I think this is the part that doesn't really work, because, in order to escape a piece of user input, you need to understand what it's meant to be. For example, a common (unsafe) idiom is:
sqlFormatString = "SELECT %s FROM %s WHERE %s;";
filterFormatString = "%s LIKE ?";
filterString = sprintf(filterFormatString, userInputColumnName);
sqlQuery = sprintf(sqlFormatString, columnName, tableName, filterString);
preparedQuery = dbImpl.prepare(sqlQuery, userInputValue);
Now, you could replace this with a type safe dbImpl.prepare(), which only takes a NotUserGeneratedSQL type. But now you need to have some way to build a NotUserGeneratedSQL.
One option is to go all in on an SQL AST library, where you could build the query above like
query = selectQuery();
query.columnNames = [ escapeColName(columnName) ];
query.from = escapeTableName(tableName);
queryFilter = equalityComparison();
queryFilter.left = escapeColName(userInputColumn);
queryFilter.right = preparedArg();
query.filter = queryFilter;
preparedQuery = dbImpl.prepare(query, userInputValue)
However, I don't think there is any alternative that can actually enforce validation of user input in constructed queries. You can help users think about it by forcing them to use some kind of string -> NotUserGeneratedSQL function, but this function can't actually be written to reject the example I gave initially. A function that creates an empty minimal NotUserGeneratedSQL and offers other functions to build it up to a useful query will either accept strings (making it possible to introduce SQL injection) or have to accept structured SQL, making it fall into the AST problem.
Of course, the problem of NotUserGeneratedSQL is easy if you don't want to support any kind of dynamic query, other than prepared statements. But if you want to dynamically generate queries based on user input (e.g. user chooses which columns they want to see, which columns they want to filter by etc.), then I don't think it's possible to statically ensure that SQL injection is not possible without an ORM or SQL AST API.
Edit: if you believe otherwise, please show me a type-safe, SQL-injection-safe library in any language that isn't an ORM/EDSL or a straight up SQL AST manipulator. I am quite certain none exists, but I would be happy to be proven wrong.
allowedColumns: Map[UserInput, SQL] = {
"name": "name",
"age": "age",
...
}
strict: SQL = strictComparison? "=" : "LIKE"
query: Maybe[SQL] = allowedColumns
.get(userCol)
.map(col => "SELECT " + col + " FROM tbl WHERE tbl.foo " + strict + addParameter(":?", userFoo))
- We can write literals which look like strings but have type SQL
- We can append fragments of SQL together (potentially making it invalid; oh well)
- We can include user input via parameterised queries, in locations where arbitrary strings/ints/etc. are allowed
- Anything 'structural', like identifiers, choice of comparison operations, building up sub-expressions, etc. must be done programatically, using the above features. In this case we select the column name (written as a static, literal SQL value) by looking up the user's input in a map. We also allow choosing between the type of comparison to use (again, both are SQL literals).
It seems to me that requirements like 'user chooses which columns they want to see, which columns they want to filter by etc.' is fundamentally incompatible with a safe, string-like representation. Instead, the options are:
- Fully representing the structure of the language. This results in an AST approach, which allows safe dynamic queries. I think any alternative, like tracking the offset of each delimiter in a string, etc. will turn out to be equivalent to maintaining an AST.
- A "flat", string-like representation, whose dynamism is limited to choosing between some combination of pre-supplied fragments. This is what I've shown above. This is safe, but the 'dynamism' is inherently limited up-front (i.e. it's overly conservative).
- A "flat", string-like representation, which has unrestricted dynamism, but hence is also inherently unsafe (i.e. it's overly liberal).
What I'm not clear is: what prevents me from accidentally/stupidly doing:
filter : SQL = "WHERE " + userInputCol + " = ?"
Do you know of any library that implements this?
Ah, maybe I should have made it clearer that I was overloading the double-quote syntax, so we can write:
"foo": String
"bar": SQL
"baz": UserInput
"quux": Shell
etc.
+: String -> String -> String
+: SQL -> SQL -> SQL
+: Int -> Int -> Int
+: Float -> Float -> Float
+: List[T] -> List[T] -> List[T]
etc.
// Code as written
filter : SQL = "WHERE " + userInputCol + " = ?"
// Right-hand-side must have type SQL, to match left-hand-side
"WHERE " + userInputCol + " = ?": SQL
// Resolving order-of-operations of the two '+' operations
("WHERE " + userInputCol) + " = ?" : SQL
// Arguments to outer '+' have same type as return value, which is SQL
"WHERE " + userInputCol : SQL
" = ?" : SQL
// Arguments to inner '+' have same type as return value, which is SQL
"WHERE " : SQL
userInputCol : SQL
> The main downside is that you need to work entirely with compile-time constructs - e.g. you can't use something like printf to take a compile-time format string and turn it runtime into a query; and you can't take queries from a separate file, they must be in source code.
Yep, although macros could help with that sort of thing, e.g. Haskell's quasiquotation https://wiki.haskell.org/Quasiquotation
A couple of Google hits for 'haskell quasiquote sql':
https://hackage.haskell.org/package/postgresql-simple-0.6.2/...
https://hackage.haskell.org/package/postgresql-query
> Do you know of any library that implements this?
Not completely. De-coupling double-quoted literal syntax from a single String type can be done with Haskell's OverloadedStrings feature, but that relies on a function 'fromString : String -> t', which is what we're trying to avoid https://hackage.haskell.org/package/base-4.6.0.1/docs/Data-S...
Scala's custom interpolators are similar, but they rely on a function from 'StringContext -> t', and StringContext is easily created from a String ( https://www.scala-lang.org/api/current/scala/StringContext.h... )
To ensure safety, we would need some way to encapsulate the underlying fromString/StringContext implementation to prevent it being called by anything other than the literal-expansion at compile-time.
Of course, if we're willing to use macros then it's pretty easy, like those quasiquote examples above.
Haskell's module system is famously mediocre, so it might be possible to do this overloading + encapsulation with Idris https://idris2.readthedocs.io/en/latest/reference/overloaded...
(Of course, Idris also has an incredibly expressive type system, and a very powerful macro system, AKA "elaborator reflection", so it can definitely be done; but I haven't figured out the cleanest way)
You are confusing "type" with "static type". This is user input. You are not going to statically typecheck your users.
(See other comment for stringly-typing).
You are confusing "type" with "not a function type".
Firstly, I absolutely want to check, statically, that my user input is `ByteString`, or `String`, or some more distinguished newtype/AnyVal wrapper around those. If that doesn't pass the type checker, I would be very worried about ever starting such an executable!
Other than that, I want to check, statically, that my functions conform to their type signatures. When I write `parsePNG userInput: Maybe PNG`, not only do I want to make damn sure that `userInput: ByteString` is definitely a `ByteString`; I also want to make damn sure that `parsePNG: ByteString -> Maybe PNG` is definitely a (partial) function from `ByteString` to `Maybe PNG`. If I can't guarantee, without executing the program, that `parsePNG` can only return `Maybe PNG` values, then all bets are off regarding downstream invariants, and that's not a good situation to be in w.r.t. security vulnerabilities.
My condolences, you must have horribly unreliable user widget/user input libraries. My TextFields always deliver a "String" to me (unless I've set them up to deliver numbers). I don't need to check that statically.
> You are confusing "type" with "not a function type".
No I am not.
You don't seem to understand what I meant with "You are not going to statically type check your users".
The libraries seem pretty reliable; but I'm not (exactly the same as in the SQL example, BTW).
For example, I don't think I've ever managed to write a Python3 program which hasn't crashed at some point due to me mixing-up string/bytes; of course, testing doesn't usually find such problems, even with Hypothesis, since I make exactly the same mistakes when I'm writing the tests. I've also written a whole bunch of Python3 programs which spend ages crunching numbers, only to output '<map object at 0x10199c2e0>' instead of printing a list.
I make exactly the same mistakes when writing Haskell and Scala; but the computer tells me that I've made a mistake, and how to fix it, before the program ever gets a chance to run.
> You don't seem to understand what I meant with "You are not going to statically type check your users".
That might be the case. I can't elaborate on what you meant (since I may have misunderstood)
Not sure what that even is, but I'd point out that dynamically-typed ≠ Python. In particular, the "hash languages" (Python, Ruby, JavaScript) seem to make it particularly easy to make a hash of things, but not due to the dynamic typing.
I've never had that problem with NSString and NSData. They are quite distinct.
> Python3 ... crunching numbers ... map instead of list
??
Why would you output a map when you are crunching numbers?
To get a "list", for example, I'd use an NSArray. It seems tricky to get an NSDictionary instead, since you have to explicitly ask for an NSDictionary in order to get one.
So what you do? I find code that uses excessive type checks too verbose, you end up doing things only to make the compiler happy, even if you know that a particular condition will never happen (e.g. a variable that can be null but given a particular condition you know that it cannot be because it was initialized earlier, but the compiler is not smart enough to know).
So to me using static types makes sense where types are used by the compiler to produce faster code (embedded contexts, basically, of course I don't use Python on a microcontroller) and as a linting of the code (Python type annotations or Typescript).
"SafeStrings: Representing Strings as Structured Data"
https://arxiv.org/abs/1904.11254
Bosque
https://github.com/microsoft/BosqueLanguage/blob/master/docs...
This sentence betrays a confusion about what types are, and how to use them. In particular, you seem to be mixing up two possible scenarios.
In the first scenario, we have a method which "wants a string" (i.e. its input type is 'String'), and we can "pass a string" (i.e. call it with an argument of type 'String'). This code passes the type-checker, since it implements the specification we gave: accept 'String'. Perhaps there are higher-level concepts like "formats" involved, and the resulting application doesn't work as intended; but that's not the fault of the types. More specifically: typecheckers verify that code implements its specification; but they do not validate that our specification captures the behaviour we want.
In the second scenario, we have a method which "wants a string in a particular format" (i.e. its input type is 'Format[A]'), and we "pass a string in another format" (i.e. call it with an argument of type 'Format[B]'). That is a type error, and the typechecker will spot that for us; perhaps pointing us to the exact source of the problem, with a helpful message about what's wrong and possible solutions. We do not get a runnable executable, since there is no meaningful way to interpret the inconsistent statements we have written.
tl;dr if you want a typechecker to spot problems, don't tell it that everything's fine!
If you want the safety and machine-assistance of such 'Format' types, but also want the convenience of writing them as strings, there are several ways to do it, e.g.
- Polymorphism http://okmij.org/ftp/typed-formatting/FPrintScan.html#print-...
- Macros https://www.cse.iitk.ac.in/users/ppk/teaching/cs653/notes/le...
- Dependent types https://gist.github.com/chrisdone/672efcd784528b7d0b7e17ad9c...
> A method that wants an integer in a range. An array of maximum N elements. And we can go on forever.
You're making the same mix-up with all of these. Fundamentatlly: does your method accept a 'Foo' argument (in which case, why complain that it accepts 'Foo' arguments??!) or does it accept a 'FooWithRestriction[R]' argument (in which case, use that type; what's the problem?)
Again, there are multiple ways to actually write these sorts of types, depending on preferences and other constraints (e.g. the language we're using). Some examples:
- https://hackage.haskell.org/package/fin
- https://ucsd-progsys.github.io/liquidhaskell-blog
Your "integer in a range" is usually called 'Fin n', an array with exactly N elements is usually called 'Vector n t', etc. I don't think there's an "array of maximum N elements" type in any stdlib I've come across, but it would be trivial enough to define, e.g.
data ArrayOfAtMost Nat (t: Type) where
Nil : {n: Nat} -> ArrayOfAtMost n t
Cons : {n: Nat} -> (x: t) -> (a: ArrayOfAtMost n t) -> ArrayOfAtMost (S n) t
myArray: ArrayOfAtMost 8 Foo
myArray = Cons a (Cons b (Cons c (Cons d (Cons e (Nil 3)))))
I actually wrote a blog post many years ago which goes through various types which have a similar 'Nil and Cons' structure http://chriswarbo.net/blog/2014-12-04-Nat_like_types.html
Also note that we don't actually need to write such constraints directly, since we can often read them in from some external specification instead (known as a "type provider"), e.g. a database schema https://studylib.net/doc/18619182
> you end up doing things only to make the compiler happy, even if you know that a particular condition will never happen
You might know it, but the compiler doesn't; you can either prove it to the compiler, which is what types are for; or you can just tell the compiler to assume it, via an "escape hatch". For example, in any language which allows non-termination (including Idris, which lets us turn off the termination checker on a case-by-case basis), we can write an infinite loop which has a completely polymorphic type, e.g.
-- Haskell version
anything: a
anything = anything
-- Dependent type version
anything: (t: Type) -> t
anything t = anything t
> So to me using static types makes sense where types are used by the compiler to produce faster code (embedded contexts, basically, of course I don't use Python on a microcontroller) and as a linting of the code (Python type annotations or Typescript).
Static types certainly give a compiler more information to work with; although whether the resulting language is faster or not is largely orthogonal (e.g. V8 is a very fast JS interpreter; Typed Racket not so much).
As for merely using a type system for linting, that's ignoring most of the benefits. No wonder your code is full of inappropriate types like "string" and "int"! I highly recommend you expand your assumptions about what's possible, and take a look at something like Type Driven Development (e.g. https://www.idris-lang.org )
Dynamic languages can be very productive for small teams.
I definitely liked the productivity of F#, but the ecosystem was lacking. Often in dynamic languages the ecosystem more than makes up for the lack of compile time checks.
It is confusing that people talk about C in terms of being a typed language. The types are an annotation so the compiler knows how much memory to allocate / read. All the side benefits of having to spell that out are incidental.
That sort of typing is a distraction. A programmer does not care that they are passing 4 bytes as the first argument. That isn't really trying to squeeze anything useful out of a type system. They care that they are passing something recognised as an ID, not that it has 4 bytes.
Using C/C++ as an example of a typed language doesn't say much. The guarantees are more a culture of how to use the language than in the language itself.
struct ID { uint32_t val; };
void check_id(struct ID id);I never understood this, how is it possible that a developer calling a function doesn't test if they are calling it right. Once they test it, how does the additional compiler test matter?
Unless of course your calling function is mutating the value of a variable across so many types that it's impossible to realistically test it. It would then make sense to use a type to prevent the variable from being something it shouldn't.
But most people, don't have that situation most of the time. Even in a big application.
May be that explains why something like Python has such wide adoption.
Maybe I am just more absent minded than most dynamic programmers, but live IDE diagnostics and autocomplete are very important for me.
Of the all the constraints I've faced while developing software, reusing variables for many purposes in one code block isn't one of them.
What I found problematic is that system and integration tests is generally relegated to be done outside of the programming process. It is instead the administrators job to write an extensive test system to verify that the program actually do the job that it was written for. You start to wish for a very different set of tool design when the programmer and administrator is the same person.
What about the decision whether a function argument is an integer or a string? A floating-point value or an optional floating-point value? How could the compiler ever decide that?
The size of an integer is only a very, very small part of a modern static type system.
If a number is an int, float, large or what have you, the question I would ask is how many decimal points should be printed or if you divide 2 with 2, do you get 1 or 0.99999. A good language/compile should have sane defaults that works almost every time, while giving me options to override it if in testing it get an unexpected result where a human would disagree with the computer.
The source of tests should be the customer requirements, and types is just an set of automatic tests being applied on the code which has no direct connection to the customer requirements.
I have no idea what you're going on about. The customer doesn't care about the particulars of your unit or integration tests, instead they care about whether your software works properly. A modern, powerful static type system is one tool that helps a developer achieve that goal of properly working software, and the developer absolutely should care about whether that function argument is an integer or a string.
I don't understand this. This makes a lot of sense when you make complex enterprise products depending on unstable 3rd-party libraries but for personal project this just inflates the time you need to get the thing done. I just code exactly what I need and check if it works and can't imagine dedicating time to writing additional layer of code to automate this checking. Even when I code pure functions which are a lot easier to test.
A solid type system helps a lot in making refactoring a less fearsome endeavour, but a reasonable test coverage on top of static checking makes it even more so.
Also, automatic perf tests that keep track of regressions and improvements are just awesome.
A lot of times designing your interfaces to support mock objects more easily improves them in other ways; if you pass in a file object instead of a filename, for example, not only does it become easier to pass in a mock object that raises an error, but it also becomes easier to pass in a GzipFile object or an HTTPResponse object, and presto, your function now handles compressed data and data stored on a web server.
Also, though, monkeypatching stuff in tests isn't hard in dynamic languages. Here's an example from https://echohack.medium.com/python-unit-testing-injecting-ex...:
with patch.object(requests, "get") as get_mock:
with nose.tools.assert_raises(socket.error):
get_mock.side_effect = socket.error
inject_exception.call_api("http://example.com")If your functions have lots of hidden dependencies and side-effects, it’s hard to test.
If you split concerns properly and keep your glue/IO code separated from the decision-making/business-rules/logic, mocking is quite trivial, and there are advantages other than just testability.
Check out this talk to see examples of that in action: https://www.destroyallsoftware.com/talks/boundaries
I hate tests with a passion, but sometimes I'll write hundreds even for a tiny personal project - when that tiny personal project is going to be at the core of another thing and I want to make absolutely sure that it is working correctly in every way.
Like that time I wrote a process manager for node.js applications because I realized PM2 is a buggy inconsistent mess with race conditions all the way down. Gotta make sure mine works better than that...
def ok(a, b): assert a == b, (a, b)
from .util import ok
ok(Parse(b'.25').do(0, real), (3, ('real', 0.25)))
ok(Parse(b'1.5').do(0, real), (3, ('real', 1.5)))
ok(Parse(b'-1.5').do(0, real), (4, ('real', -1.5)))
ok(Parse(b'+1.5').do(0, real), (4, ('real', +1.5)))
I mean, I do do some manual testing. Sometimes it's easier to bang on a function and decide whether the output looks right than to figure out what the results should be from first principles. But it doesn't take very many repetitions of a manual test of something like the above before it's the manual testing inflating the time needed, not adding another ok() line.
This is especially true for debugging. Adding automated tests is often a faster way to debug than to add print statements or to step through things in a debugger. Sometimes I take the tests out afterwards, but more commonly I leave them in. And with things like Hypothesis and QuickCheck, tests can have a much higher strength-to-weight ratio than simple tests like the above.
It's also a breath of fresh air when I'm trying to understand what some code is supposed to do, or how to call it, and I come across a one-line test like the above.
Speaking of how to call things, sometimes I do test-driven development so that my internal APIs don't suck. And sometimes I don't and wish I had. There are a couple of lines of code in the same project that say this:
array = Thunk(lambda: drop_ws(b'[') + Any(pdf_obj) + drop_ws(b']'))
array.xform = lambda d: ('array', d[0][1])
It would have been less work to fix it at design time, the way my friend Aaron taught me: first, write some application code as if the ideal library existed to write it with; then, write the library code that makes it run. And that's just so much easier when that application code is just a unit test.
However, as with most things, I disagree with Robert Martin's point of view on this. I can easily conceive of writing software without TDD, or even without tests. In fact, I do it frequently. Tests are often valuable, and TDD is sometimes valuable, even in weekend personal projects. But they aren't always worth the cost.
Tests have advantages, but they also hinder your agility by increasing the costs of the initial version and of making major changes.
Hard to update? Good tests! Well understood problem sphere and product? Good tests! High cost of failure? Good tests! MVP with no actual current users? No TDD. Exploring the product space and update cost is low? No TDD.
To give an example from Haskell:
You can stick everything into IO, or you can be more careful with your effects.
Or to give a more main stream example:
Even in a language like Go, you can do lots of runtime casting (which is essentially dynamic typing), or you can try to cajole the type system into recognizing what your are doing statically. You can do the latter effort incrementally.
Less opinionated/experienced with strong type systems though, I do appreciate the flexibility of being able to hack things together to see if they work - then improving upon the solution. strong types seem to make that harder.
I often build systems, then break them up to take the working parts out, then put a subset back together.
The type system is like magnetic Lego that lets you snap pieces together.
With a dynamic system it is really difficult to break things apart and then put them back together (without ever running the code, and with 100% accuracy and immediate feedback).
Writing correct, maintainable software is hard. You need both tests and typechecking.
Those are all functional languages, so this argument seems to be the "functional is better than OO" argument, typing is not the primary factor (but of course to the degree that FP allows static typing to be more intuitive, that's a plus). What examples of static typing with OO or procedural languages that are only a "pleasure to work with" can we look at ?
I've never been able to make heads or tails of Girard's paper but http://pauillac.inria.fr/~fpottier/slides/fpottier-2007-05-l... is an introductory divulgation of some of the pre-Rust history of this stuff. It makes me think I should read Wadler's 01990 paper.
So I think functional languages kind of had a head start, but also in a sense they have an easier problem to solve. ML's type system doesn't even have subtyping, much less mutation and resource leak safety.
OCaml and Rust would fit the bill. You can do procedural and OO in both. You could add Scala to that. I think the conclusion is not that "functional is better than OO" but "they both have their strength and are not mutually exclusive".
This is entirely dependent on the type of application that you're dealing with.
A web application backed by a database, especially CRUD style, acts as little more than a pass through layer. You're getting everything as a string from the browser, storing it in the database with a SQL string, usually transformed by an ORM layer. In that style of application, the types are maintained in the database itself.
Having a language with strong static typing in that middle layer between the browser and the database only serves to add a huge amount of headaches and extra friction, especially when you're not even using each variable that comes through on its way to the database.
The only thing you're doing in that situation is adding an extra set of type checks that you have to keep in sync between the database and the application layer for virtually no benefit.
In your example, the database and the application layer must be kept in sync regardless of whether you use static typing or not. The benefit of static typing is that it can be enforced before you push to production or write a test for congruency.
This is handled automatically by a large number of ORM tools in dynamic languages (Ruby, PHP, etc). It's a non-issue.
I've spent significantly more time in my career debugging issues in statically typed languages used as an app server in front of a database than I have in dynamically typed languages.
You're doing some processing work in the language? Calculations, ML, etc? Static typing is the obvious way to go.
If you're just sticking things in a database you're going to end up wasting a lot of people's time (and subsequently, the money that goes to pay them). In 20 years, I've never seen it be a good choice but I've seen a lot of people married to the delusion that it is while other people are getting real work done.
This may be one of the differences of requirements that causes disagreement here. I can see how this might work if your app:
* Loads data from the client
* Later displays that data, unchanged, to the client
In that case, treating everything as a string is great, because you never need to use it as anything else.
In my case, I've never actually worked on an app that didn't need to do more than just display the data at some point. In those cases, I don't want to get everything as a string from the browser. My data is often not merely a string, and treating it as such means I have to deal with stupid questions like whether "0" == 0 == "null".
In my work, I have spent many, many hours repairing issues that could have been avoided if the client and server were both statically typed (and on the flip side, in services that are statically typed, rarely run into those same problems). I'm sure that this can be remedied by just "doing it better", but I'd rather operate from a foundation where doing it right is easier than doing it wrong.
Instead of strings, I want my apps to have a statically typed contract between the client and the server that specifies what endpoints are available and the data structures they accept and return. Ideally this is in a shared class/type definition file, but two definitions kept in sync can work as well if they don't share a language.
The common argument against statically typing your network requests at each end is that you're not resilient against unexpected values and/or version differences, but:
a) It's trivial to tell any JSON parser to ignore unknown values, so adding new optional fields to the request is never a problem.
b) If your app ever uses the data received from the client, version differences will eventually break your dynamic code, too. You'll just only be told so at runtime, potentially only after loading the bad data from the db.
If you ever use your app's data for anything other than rendering a template, you have to make breaking changes to your data structures very carefully, and using a strong layer of static types on both client and server helps to ensure that you consider all the breaking changes and either avoid them altogether or somehow mitigate them.
I agree that this is a different scenario. If you're able to have the client control the input to ensure only the correct types are passed in there may be some additional value there to keep things more sane.
However, how much of the data that you're dealing with in the client on the server and in the database are you really working with in the server itself? 1% of the variables? 2%?
Generally any heavy work will be handled in either the database itself or a background job just to reduce wait time for the client.
And if that's the case, you're spending the time defining types for the client, server and database, keeping them in sync, having to change them in each place and creating this exercise for the other 98-99% of variables passed in and out of your server just to gain the static benefit for the 1-2% where you're actually working with it in the server on the web request.
It's significantly more work with questionable benefits IMO.
And if you have, you probably know that its awesome but for most purposes doesn’t compensate for the weak ecosystems of the languages that have it, compared either to poorer (higher cost to satisfy for lower benefits) statically typed languages or dynamically typed languages with or without available optional static type checking.
Its type system was so lame that it gave rise to most of the "Gang of Four" patterns, most of which are just workarounds for Java- not some first order principles of programming.
So you had these awkward and verbose "patterns" and polymorphism and naming a bazillion new types that were almost the same as each other, and STILL had runtime type errors and crashes. What's the point?
It's no wonder at all that every desktop app I installed on my Linux machine in 2008 was written in Python.
It's no wonder someone thought it was actually a good idea to run *JavaScript*, of all things, on the backend *and* the desktop. Holy crap- how bad do statically-typed languages have to be to make people want JavaScript?! (I'll tell you: they have to be Java and C++98 bad- that's how bad.)
Thank goodness for Swift, Rust, Go, etc in the post-2010 world.
For the last few years people are looking down their noses at dynamically typed languages, but I think statically-typed languages almost died thanks to Java and C++.
Java wasn't released until after the first edition of Gang of Four. C++ is probably what you should be pointing the finger at.
Those two are just minor syntactic sugar. They literally make miniscule difference practically.
I think you forgot C++11 :-)
If I may ask, what ideas are static languages incorporating from dynamic languages? Could you help clarify?
It's handy if you need to add interfaces onto code you don't control (say, library code) as you can avoid a whole layer of adapter types / functions that you need in true statically typed languages.
But I'm curious if its a great idea in the long run to have interfaces being "accidentally" implemented by objects. Doesn't seem like it would stand up well to refactoring or various other scenarios.
(that's a joke)
∃a.P(a)
----------------- Remove-Exist
∀b.(∀a. P(a)→b)→b
Something that quacks like an iterable cannot be used as an iterable, unless the trait is explicitly stated — unlike python, where it simply has to quack.
[1] https://channel9.msdn.com/posts/MDCC-TechTalk-Classes-Jim-bu...
I don't know much about C# but how is this dynamic? Are those methods often added at runtime?
This was discussed in 2011 in a StackOverflow question [0], including a link to a blog post by one of the language designers [1].
[0] https://stackoverflow.com/questions/6368967/duck-typing-in-t...
[1] https://web.archive.org/web/20120126033827/http://blogs.msdn...
Also, over the past 20 years what you've seen is big advances in the 'semi-static' world like the JVM, where it runs statically typed languages but they have the ability to eval code, redefine their own code on the fly, reflect themselves, they're garbage collected etc. This is kind of a middle ground. It's worth remembering that when Python and Ruby were new, there weren't really any great options if you wanted lightweight syntax with garbage collection. Nowadays there is Kotlin and you can write code that looks very similar to say Ruby, but which often has the performance of entirely statically typed languages.
Lisps and MLs offered this before Ruby and Python got popular. Unless you don't consider them great options?
I suspect one issue is that Lisp never seemed to be well supported on Windows and never came out of the box on Linux, except perhaps for Guile, but Guile never reached any kind of critical mass despite being promoted by the GNU project. Maybe one reason is the lack of learning materials. Even today, although Guile has an initially pretty and appealing website, clicking "tutorials" reveals a complete lack of interest in growing that community - there is only one single tutorial, which is about how to embed Guile as a scripting language into a C program!
https://www.gnu.org/software/guile/learn/#tutorials
I remember learning Python in the 1990s. The learning materials were excellent. Java was also famous for extensive tutorials and learning materials (they've lost that in recent years, the modern Java docsites are just piles of specifications, but when Java was interested in growth they had it). In the end these things matter more than the exact nature of a runtime or type system.
No ML was a realistic workable option back then at all. Haskell is fine now, but was barely introduced in 1993. OCaml has become a workable option almost entirely due to the gargantuan effort of Jane Street, but again, it wasn't then. Standard ML remains terrible. Library support is sparse, there is almost no community outside of academics, no consistent implementation of the standard basis, compilers are wildly different from each other. The REPLs tend to be great, but it's very difficult to get from a set of source files to a portable executable, whereas with Python and Ruby, just writing the source files already gets you that.
> OCaml has become a workable option almost entirely due to the gargantuan effort of Jane Street, but again, it wasn't then.
What do you mean by this? I'm aware of dune and opam, but people were using C and C++ without equivalents before without problems. Python's package managment and building is also not that good, even today. I don't have a strong grasp of the history of OCaml so maybe they released Core, Base and Async really early compared to batteries and Lwt? But outside of that, basic OCaml with a makefile doesn't sound worse than C/C++.
> Standard ML remains terrible. Library support is sparse, there is almost no community outside of academics, no consistent implementation of the standard basis, compilers are wildly different from each other.
That's fair, my point is more that I don't really understand why no big company ever picked it. Considering how much companies invested in their tooling (Google with Java, Go, Dart, Python, C++ ; Facebook with PHP and C++ ; etc), they could have made something great.
> it's very difficult to get from a set of source files to a portable executable, whereas with Python and Ruby, just writing the source files already gets you that.
Depends on your definition of portable executable. I've always found deploying and distributing Python and Ruby painful, at least to end users. The best in class experience here for me is Go, it's great for end users, great for servers, cross compilation works well.
That isn’t the case for F#, and I would say it’s questionable for Python for anything larger than a couple source files.
After working with more strongly typed languages, it actually seems crazy to go back to JS or Python and not have type information in function signatures. Like how are people supposed to read code and understand what this function is allowed to take?
Before that large companies like Google got around it by annotating in the comments. It may be an inferior solution but huge codebases serving hundreds of millions of users have worked this way.
f(x: a, y: b): c = ...
f :: a -> b -> c
f x y = ...
It also comes down to the compiler. If you're "annotating" your code with type "sugestions," but don't get any ironclad guarantees from it, then it's reasonable to question whether it's even worth the effort. I may be misremembering things, but "typed" versions of Python and Scheme aren't particularly rigorous.
Do I care what the different functions in the library return? No. The field could exist in everything from an XML object, a dict, a string, a byte array, or a custom object made for easy interaction. What I need to know is where in the flow I can best inject my new code, and in order to do so I need to know when and where the information can be accessed. In an optimal world I would have written a shell script, and in that case every function has a single return type which is string.
A good typed language will allow you to be more or less strict about what you accept (eg. a specific literal string, a set of strings, any string, any string like object, any object...)
People argue that they need types to very that you do not add in programming errors, and in order to easier read what the code does. In the environment I operate in and the jobs I do, types do not achieve those things. All they do is add unnecessary information and restrictions that need to be worked around.
Shell is an interesting environment in that there really is only one type: string. There are no int, no floats, barely any arrays (rare enough that I don't ever use them). You run a program with string inputs and you may get an string output in return. The majority time writing a shell script is spent on manuals, understanding how different programs interact, and data flows. This remain mostly true for dynamic languages. For static typed languages some of the time get redirected on dealing with types, declaring types ahead, and making sure the type declarations and castings are correct.
I would encourage you to try typescript for a while, it's very approachable and has excellent tooling.
I have worked with 10+ different languages, multiple different IDE's, some types and some dynamic, some like shell which doesn't really have the concept of types at all.
If you are encouraging me to try typescript because you think its great, then I would encourage you in return to test python in combination with system and integration tests. It makes for a very safe environment where you know your code is working.
I know how to make python safe with TDD. It doesn't even sort of compare.
I don't want to speculate why you feel so attached to static languages. I worked with python in 15 years, you in 17. I have done low level embedded C code. I done web development, and shell scripts. Tried lisp, and programmed in Ada. If you have also worked with similar amount of languages and reach a different conclusion then either your work is very different from mine, or you as an other person have a different experience than mine.
When it comes to keeping simple thing being simple, static languages has never delivered for me. In embedded system I live with that aspect, but that's the nature of such environments. The primary question when working with dynamic language is: "is there already a library that does what I want, and how do I modify it to work in my use case". Instead of thinking about structure and types, I am thinking about interfaces.
I will conceit one example where a static typed language is much better than a dynamic typed, and it illustrate the environment where such language aspect shine. SQL. SQL with dynamic types would be horrible and any database which treat types some-what dynamic is a horror to work with. SQL does not make simple things simple, and database exceptions are quite harsh in every aspect. There is also few if any interfaces, so its types or nothing.
If there are types that allow you to not care about the type, then why have a type?
Otherwise I don't know why you would particularly care more about types with st than otherwise You'd just follow the same pattern as you did when extracting the other fields. E.g. if the fields to extract are listed in a list like structure, you're just going to add the extra field to the list. And in case of static typing, you'll get soonish feedback if you messed something up.
The return type thing was just an example to show that you care about return types regardless of static typing, not that you care about return types during every act of programming.
If we both agree that you do not need to care about types during every act of programming, then lets follow that thread and see where it leads. When do you need to care about types, and how big portion of a programmers time is spent on thinking about types?
My own expensive provides the answer "rarely" and "as little as possible". The few times I do care about types is when interacting with third-party functions, and in those cases I read the manual to figure out what their functions are returning and what I can expect to use. I do not however write test that verify that a function that is documented to return a string actually do return a string (or expect my compiler to do it for me). If the documentation is incorrect then I treat that as a bug, just as if it was a logic bug.
It would however be interesting to hear how much others spend time on it, and in what kind of acts of programming.
For instance, if I calculate the time that has elapsed between two points in time: - what does my input look like (e.g. two objects that look like XY) - in what form should I return the difference? Time in seconds (i.e. number), again an object that looks like XY ? ...
I can't imagine saying I would care about types rarely.
Another nice benefit of st langs is that you have to read manuals less often. IDEs so a way better job showing you the type of every variable (and result of an operation) as well as the options you have at your hand for a given type (including their documentation and example code). Yes I know dt languages and their tools try to catch up.
> or expect my compiler to do it for me). If the documentation is incorrect then I treat that as a bug, just as if it was a logic bug.
Well, I don't expect that with Dr languages as well, but I am glad that the compiler does this (or in DT languages at least the IDE due to type hints). I mean, it saves a lot of time (money) later on ... even if you notice it right away the first time you use the code
I would first ask, should I write a function or maybe have this as an property of the object that has the original data. I want the intended interface to influence my design.
What methods does my input has? Does it already implement what I want or have a helper function that makes writing the transformation easier? Is there built in functions or third-party libraries that do time differences calculations.
Within python I would apply some minor assumptions about types. If something is a time it is either a string or an object with a time interface, and so I might need to cast it to a time-like object. Preferable this already occurred as close to the edge of the program as possible so in the rest of the program I don't need to put any thought about the type. If its a date, it should be a an object with a time interface. If time strings are floating around in the data flow, just as with byte strings, I should try to fix that as early and close to the edge of the program.
There seems to be a similar view about unicode. You don't want to have every function check if the input is of the type unicode. Instead what people do is when they read a byte string, they turn it into unicode as everything else assume the unicode interface.
Dynamic code requires the programmer to think about the same things; what static typing requires is communicating that thought to the type-checker through a constrained language (in the best cases, with robust type inference, this can be no additional cost for substantial fractions of a code base, though). On the other hand, it also provides fairly immediate feedback on the correctness of the information so communicated, and cam leverage it in dev tooling, so there is a benefit with the cost.
Typed codebases are far better to return to than untyped ones. Anyone who doesn't think so has probably not tried a good, typed language with a good IDE.
I have provided examples in other comments where types is about as far away from my mind when writing complex programs. In terms of guarantee of safety, system and integration tests do that. Types are not based on customer requirements, nor do they verify that the program do what I get paid to do. A type is simply a limited set of automatic restrictions and tests to very that those restrictions are followed.
You have to be thinking about types, because the computer can't. Since your programs work (I assume), and computers can't be doing this for you, by process of elimination, you're the one thinking about types.
What is actually happening is either that you've so deeply internalized it you don't realize it any more than you think about individual muscle contractions while walking, or you don't recognize what "thinking about types" means. For instance, I imagine an obvious riposte to my first paragraph would be "But if the code expects a URL and I pass it an LoggedInUser, I can set it up so the LoggedInUser can yield or function as a URL", which is in fact precisely "thinking about types" because you just changed the type of your LoggedInUser.
(You want to see someone who is truly "not thinking about types", go help someone learn programming for the first time, who truly innocently tries to pass a User to something expecting a URL and is truly surprised when that doesn't work because they truly, deeply thought the program would just figure out how to turn that into some URL they had in their head for the user, but never explained to the code. That's "not thinking about types". It is nonfunctional.)
As others have said, dynamic code requires you to do more thinking about types, not less, for two reasons: One, the compiler isn't doing it for you, and two, you have a lot more things to juggle in a dynamic language than in most static ones precisely because they give you more power. You shouldn't be fighting this idea, you should be embracing it; it is precisely the fact that they make you think more about types and leave more of the decisions to you that gives you the additional power in dynamic types you are enjoying.
But that additional power irreducibly comes with more responsibility. You can't claim you've obtained the power yet somehow dodged the responsibility of managing it. You are holding a lot more state in your head that has no existence in your source code than you would be in an equivalent static program. You can't claim you've got that state in your head but you're somehow not "thinking" about it.
You're thinking about types all the time. Your complaint is more than you like a programming style that involves very complicated type ideas that are difficult to express in a static language. (Or, possibly, that you haven't taken the time to learn to express in a static language. There are some static languages that work better with this style than others.) Or that you don't want the constraint of having to express them. So you stick to an environment that loads the work on to you instead in return for the power of those more complicated things you never have to lay out for the compiler. But the work is getting done somewhere. It has to be, or your program wouldn't work.
If the code only expect the interface of an URL, the type of the object being sent in is irrelevant, and thus the programmer do not need to think about the type. Instead we are thinking about interfaces.
Programming in a dynamic language do allow a programmer to not think about types, but they do need to think about interfaces. Programming in a static language require the programmer to think about types, but they also need to think about interfaces. Having types does not eliminate the obligations to think about interfaces regardless of programming style.
If we want to use TypeScript as an example, a type is a data type of either Any Type, Built-In Type, and User-Defined Type. The Type System in TypeScript is responsible for checking the data type of any value taken before it can be provided as an input to the program.
Interface in TypeScript: An Interface in TypeScript is a syntactical obligation that all entities must follow. It can only contain the declaration of the members and is responsible for defining the properties, methods, and events. In a way, it is responsible for defining a standard structure that the derived classes will have to follow.
(above taken from https://www.geeksforgeeks.org/what-is-the-difference-between...)
Is a type, as defined above, required for all programming? No. Interfaces? Yes. Interfaces are also more similar to customer requirements in that they define behavior. Do the object has an absolute path method. Do it has an UUID property. Can it be used to call open on. Those questions are not type questions, and so I do not think about type when answering them.
...until you accidentally pass the ClientsTable object to the delete_supplier() function, which expects a Supplier object. Unfortunately, the delete_supplier() function calls the delete() function of the passed object, which is helpfully provided by ClientsTable. Bummer. Eh, just restore it from backups, right?
No, it won't prevent every conceivable bug or error, but nobody claimed it does. Automatically ruling out a large class of bugs at compile-time is still very valuable. Don't let the perfect be the enemy of the good.
The closest I would ever have gotten to the above scenario would be the time when I wrote an array of function pointers to be called in a parallel process. A bit of rather complex C code, and if I recall a bunch of void pointers.
If you intend to provide examples or bugs being caught with type checking, it would be useful if those examples actually occurred and were caught by the type checking.
> You can pass a LoggedInUser to a thing expecting a URL in a dynamic language if the LoggedInUser has the same methods as defined in the interface of an URL. It called ducktyping. In the same line of thought, you should even actively avoid checking if LoggedInUser is of the type URL because a different type sharing the same interface as URL should be equally accepted as URL.
You can do that in statically typed languages as well (though not in every). See C++. Rust actively decided against it, and makes it the users responsibility to say "yes I indeed want to give this type to this function, because it satisfies the trait like this".
Is a type, as defined above, required for all programming? No. Interfaces? Yes. Interfaces are also more similar to customer requirements in that they define behavior. Do the object has an absolute path method. Do it has an UUID property. Can it be used to call open on. Those questions are not type questions, and so I do not think about type when answering them.
Interfaces in TypeScript just describe sets of types, so yes you are still thinking about types (about types of types). I'd like to know why you think either one is (syntactically) required for programming (as we know ASM doesn't have either of those) Further, I'd like to know how you don't think about basic types in JS. Finally, I'd like to know why you couldn't fall back to (mostly) duck typing in statically typed langauges.
Taking the concept of interface, it would be difficult to program something with objects if you did not know the methods, properties, events. Just knowing the type name without any of the knowledge about the interface of the object would make programming close to impossible. The opposite however, knowing the methods and properties but not the type, is enough information to write a program.
In one python program I wrote I created a proxy object for a third-party library. The library only supported a single process, and I needed multiprocessing. The proxy object allowed me to take every call to the object from process B to be pickled and forwarded to process A, with the return value being sent back to B. No function needed to be made aware of the proxy object, the third-party library behaved just as it was a single process program and everything just worked. The proxy object did not need to have any information about the call signatures or method names of the third-party library objects.
It would be interesting to see such proxy object being written in a statically typed language that maintain the type checks when the proxy object get used inside a third-party library. Requirements would be that the proxy object should be independent implemented without being effect by the call signatures and method names in the third-party library. It sounds a bit fun trying to get the compiler to resolve what the type and signature should exist at compile time, through that might just be implementing a dynamic-like language through macros and compiler tricks.
Perhaps you could parse the source code to the third-party library and generate matching proxy objects from that.
No number of compiler tricks would allow you to define a single object that can be a proxy for anything using a statically-typed language.
"But it turns out that strong typing does not eliminate the need for careful testing. And I have found in my work that the sorts of errors that strong type checking finds are not the errors I worry about."And that space is where your data itself is highly dynamic. When you just want to represent data as maps, do generic operations on maps, then spit out more maps back to something else. This description applies to many many real world systems, particular in the more business-type domains.
Another space where dynamic really shines is dealing with relational data in a relational way. Relations are just sets of maps, the most natural thing in the world to work with dynamically. In a dynamic language I can just write code that peaks into the map, transforms certain fields, then passes the whole thing on down the line. I don't care what combination of selects and joins got me that relation, and in reality there will be many, I want to reuse my functions regardless. Although this is where more structural type systems can help ease the pain over nominal type systems.
I feel like this is a great blog post talking about the downsides of static typing in a real world system: https://lispcast.com/user-wizard-scenario/. I appreciate that the author has real world experience with both Haskell and Clojure and comes from that perspective.
Ultimately I'd say that if a program can be written in a static language, it should be, however some programs deal with information that just defies fitting into your type system without herculean efforts. Just like static languages will look down on dynamic languages for reimplementing poor ad-hoc type checking, dynamic languages can also look down on static languages for poorly reimpelmenting dynamic behavior with a bunch of functions mapping Any to Any.
> I also realized that the flexibility of dynamically typed langauges makes writing code significantly easier. Modules are easier to write, and easier to change. There are no build time issues at all. Life in a dynamically typed world is fundamentally simpler.
Isn't he mixing static and compiled languages here? I'm also not convinced at all that dynamically typed code is easier to change. Easier to write maybe, but I thought his whole thing was that writing is the easy part and after that software rots and you have to change it.
Edit:
> I thought an experiment was in order. So I tried writing some applications in Python, and then Ruby (well known dynamically typed languages). I was not entirely surprised when I found that type issues simply never arose.
I don't think understanding code you just wrote and not having issues in it is a great experiment with a lot of meaning.
Yeah this is something I take huge issue with. There's always a tradeoff: less comptime issues means more runtime issues.
Comptime issues are predictable and deterministic. Runtime issues may depend on inputs, or external factors like memory constraints, so they're way harder to identify and solve.
Static types are like eating your vegetables. You might get more instant gratification from eating cakes, but it's going to come back to bite you later.
But my sense is that we're gradually swinging back around to static typing via gradual typing + mainstreaming of ML style features like generics, unions, records that are more expressive than the C/Java model.
In part, I think the move to mainstream more ML features is to finally try and address this.
Meanwhile Julia gives dynamic languages some of the same feel as static ones.
Both dynamic and static languages have benefits, so I don’t find it weird that they borrow from each other.
Personally I prefer opting in to use static type checking over opting out.
The error messages are not as nice as if the types were specified at the top level, but there's usually still enough static type checking to know whether the code will run or not before it runs, which is one of the big benefits of static type checking. The other benefit of static types is performance due to compile time optimisations, which Julia can benefit from if the functions are type-stable.
Do you have type constraints for generic arguments in Julia? I.e. when you're looking at the signature, does it ensure that the operations performed on values inside are going to work?
If I write
f(x :: Number) = x + 1
f(x :: String) = x * x
f(x :: Int) = x - 1
julia> f(1.0)
2.0
julia> f("hi ")
"hi hi "
julia> f(2)
1
> when you're looking at the signature, does it ensure that the operations performed on values inside are going to work?
I guess the answer depends on what you mean. Julia's interfaces are somewhat implicit, and we do not do ahead of time checks by default, so if I define my own type that is a subtype of Number, but that type does not have methods for addition, then f will error on that type.
However, the static checking package JET.jl will easily detect that this will happen statically at compile time.
In Python, you can, if you work at it, find a variable in some other process by name, and store into it from another thread. While this is unlikely, you cannot definitively tell at compile time it won't happen. This prevents most compiler optimizations.
Yes, you can do awful things involving creating an optimized loop, detecting at run time that someone broke it from the outside, and switching from compiled to interpreted mode. I think PyPy does stuff like that. It's a huge effort in the compiler and in the runtime to handle such cases.
It's also part of why Python sucks at threading, with the Global Interpreter Lock. There's not enough separation between threads.
The advantages of a dynamic type system are two-fold:
1. Dynamic typing and a meta-object protocol allow for certain design patterns. Arguably makes it a bit easier to write DSL's.
2. You don't have to spend 500 milliseconds typing the word `string` or `int`.
However, in reality:
1. Virtually no one in these threads are actually writing DSL's, or applying any of those design patterns. They're writing line-of-business CRUD apps, and simply don't feel like reading or typing `string` or `int`.
2. Most static languages are getting more sophisticated with type inference, such that you don't have to type `string` or `int` nearly as often (although we can debate whether this actually makes your code more readable or less).
So in sum, most people aren't even getting much if any purported advantage, and in exchange they are giving up a slew of compile-time checks that must instead be manually implemented in (unreliable) tests.
It's just madness. You don't want to type `string` or `int`. It really is as simple as that.
I prefer to write "string" or "int", but it can be borderline impossible to write the type of a complex query, the kind of thing returned by LINQ or the like.
What's the type of a class or struct that has fifty non-optional (not null) fields of various types? Easy! It's just the name of the class or struct.
What's the name of the same class after applying the "select" or "project" operator, with one of the fifty or so fields removed? Umm...
Okay, next: How about the name of the type that is a projection that removes one arbitrary column based on a dynamic, runtime input? Can your static language express this at all?
This is the most basic query operation possible, affecting one column or field, and your fancy static typing has fallen flat on its face.
This is not about typing speed. It's about solving trivial problems with fit-for-purpose tools instead of fighting the compiler.
This is why SQL is still a thing, and why a lot of people massage data with Python or R instead of C++.
Instead of developing normally, writing code, then compiling, then testing, then raising a CR, you work with a REPL that helps you fill out parts of the code as you go. So what you can do is just type
`> :t what_is_this_type?`
where what_is_this_type is a hole for the thing you don't know how to type yet. So with this REPL based development you don't even have to try to figure out the type.
So IMHO, it fundamentally comes down to the power of the statically typed language. If you're working with Java, C++, C#, etc, type inference can help a lot but ultimately can't give you everything you want. Precisely because of that dynamic, runtime input. But if you have a functional language, like Haskell, Idris, or Clojure, you actually can precisely type that input and have something reasonable looking come out at the other end.
I like to use the select/project operator as an example precisely because it is so basic, yet so revealing of the gaps in modern programming languages (static typed or dynamic).
It really is very fundamental to not just verification of correctness, but also performance, data security, API surface design, and a whole host of things that are causing a lot of grief for a lot of programmers. The "ORM impedance mismatch" is in no small part caused by this issue.
Imagine a not-so-hypothetical scenario of wanting to make a Web API for something like "get cloud virtual machine". A virtual machine might have hundreds of properties! Everything from creation time, licensing mode, SKU, status, etc, etc...
So of course, for efficiency, you want the API to only return the relevant columns, right? Currently, there is no way to do this in most programming languages without reams and reams of dynamic typing of some sort, somewhere. One way or another you will be explicitly treating the returned rows as hashtables, arrays of "object", or something. More than likely, you'll have to dynamically generate the SQL/NoSQL query as text. You'll have to write nested loops to handle the rows and columns. Then all of this will have to be repeated on the client side, but likely with different languages. There's a decent chance that the trivial operation of "select {set of columns} from table" will end up executing a hundred megabytes of code in the end-to-end path and take multiple seconds to return a couple of kilobytes of data.
It's madness.
With duck typing you just stuff the data in there and pick it out again.
I did not get it either until I had a couple of years of Python experience under my belt. And yes, you can do that in something like Java too, I'm sure. It's just not as convenient, and the culture will not tolerate it. And yes, there are downsides. (But there a downsides to the spec'ing too, one of them being proliferation of concepts, increasing the mental load of understanding the system.)
I've never seen a Python codebase larger than 10,000 lines, that didn't require exhaustive archeology for newcomers to figure out the contracts. I'm sure that the original authors (who are always long gone) had fun, though.
Correct! I don't want to type it and, more importantly, I don't want to force others reviewing my code to have to read it. One of the very few empirically demonstrated facts about software engineering is that the number of defects is proportional to size of code, usually measured as the number of lines of code. Source: https://softwareengineering.stackexchange.com/questions/1856...
There's a lot of bullshit in software engineering, cargo cults, buzz words, bla bla oriented programming, etc. All with zero empirical evidence. But that the number of bugs per line of code remain roughly the same independent of language has actually been demonstrated.
Thus, I pick Python over more verbose statically typed languages.
Having it written there makes code review easier instead of harder. Because you see information instantly instead of having to guess it.
I think you're taking that too literally. It doesn't make sense applied to types.
If you have the same piece of code, having types doesn't increase the complexity in any way. Its just more information shown and more validation.
By that logic we should all use the shortest variable/function names possible so that we can minimize the length/lines of code
> By that logic we should all use the shortest variable/function names possible so that we can minimize the length/lines of code
Perhaps we should since there is no research to suggest that short names negatively affect software quality.
To then turn around, and claim that type declarations inappropriately inflate the line count, is absurd.
Snorted out loud at that one.
This post seems to assign value to static typing: That it aids in avoiding bugs. It then makes the case that TDD will do a much better job at it, and thus, obsoletes the need for this (having 2 systems would be redundant!).
The problem is, that's just one of the many things static typing does.
static typing also deals with the general problem of discoverability: When I'm writing code, how do I know which features and functions are available, right now?
"Just read a ton of docs and be a walking dictionary of API calls" is a really bad answer to the question. So is: "Welp, re-read the intro page of the docs over and over".
A much better answer would be: Well, at the touch of a single keyboard shortcut, or even 'as you type', you get a curated list of the most likely things you'd want to do right now so you can just eyeball that as you program.
static typing makes that way, way easier to do vs. dynamic.
But, I think even as far back as 2003, Bob "TDD!" Martin was already reframing all coding issues in terms of TDD so this isn't exactly surprising.
1. The type system is not the only consideration, nor is it the most important one for most cases.
2. Language choice often boils down to cultural things: what is already written, who can write it already, the lowest common denominator. This is not always a good thing, but often it is necessary.
3. The tools we use shape the way we think.
4. Tools should be invisible, like an extension of ourselves.
5. Static vs. Dynamic is a false dichotomy. There is a gradualism to it and it includes more than typing.
6. Constraints are important, because they facilitate engineering, confidence and systems thinking.
7. Interactivity is powerful, because it facilitates understanding, expression and domain thinking.
8. There are cases where either of these things are more beneficial.
9. Some languages are half-assing either of those to be more familiar.
10. We should give ourselves more credit to be able to use the right tool for the job.
My suggested answers: 1. We now know, 20 years later, that dynamic-typing languages have not replaced static-typing languages. All of the languages from that '03 blog post are still heavily used. If we look at more recently introduced languages, some are statically typed and some are dynamically typed. Evidently programmers still see value in both of those ideas, and neither looks like it's replacing the other. 2. We now know that it's possible to add type deduction to statically typed languages and type annotation to dynamically typed languages, even older languages like C++ and Java and Python.
Abstractions are very powerful, but when many people work on a codebase, keeping extra checks via static typing imo is good. Unless your Dev team is _highly_ experienced, then you can put away with the training wheels
Nah, they don't have to be done. Most programmers using dynamic languages just don't think about types until they need to be thought. That implies there's more resources/energy available to think about the actual problem, which often has almost nothing to do with types. Then if you're using something like Common Lisp/SBCL, you can define the types afterwards where they are most critical.
Doing all this well in a dynamic language requires more discipline obviously, since you cannot even release software in a stricter static language if it fails to pass the type checking phase. But it is faster, sometimes significantly faster. Sometimes it may even happen that the coder could not have finished the task in a stricter language, which would imply that the the dynamic language was infinitely faster.
https://danluu.com/empirical-pl/
The evidence behind strong claims about static vs. dynamic languages - https://news.ycombinator.com/item?id=16287083 - Feb 2018 (1 comment)
The Empirical Evidence That Types Affect Productivity and Correctness - https://news.ycombinator.com/item?id=8618887 - Nov 2014 (17 comments)
The empirical evidence that types affect productivity and correctness - https://news.ycombinator.com/item?id=8594769 - Nov 2014 (9 comments)
And as someone else said, static types aren't training wheels. You don't stop using them when you think you're sufficiently skilled.
No, dynamic and static languages are going to merge, coming at optional, selective static typing via opt-in static typecheckers on the dynamic side and opt-outs via dynamic escape hatches and/or use of broad (e.g., Any) types from the static side.
My experience with dynamic language rot is that a single developer or a small team can create a clean bug-free dynamic application but as soon as you start adding features, refactoring, or adding developers you get all kinds of runtime-only bugs that creep in. Older unsafe static languages like C and C++ have the same problem to a lesser degree in that the compiler will catch egregious type bugs but things like memory errors, type conversion gotchas, etc. will creep in over time unless you have some really diligent testing and auditing in place.
Dynamic languages are suitable for prototyping, glue, and throwaway code, which is why we used to call them scripting languages.
I do have to add though that today's best static languages like Go and Rust are vastly superior to 1990s-2000s era C or the carpal tunnel inducing "enterprise" nightmare that Java became. Even C++ has improved if you use more modern constructions like lambdas, and nearly all languages have better libraries and better ecosystems than they did back then.
If that’s how you feel about auto, wait until you see something like rust’s post-facto type inference.
This point is mostly relevant for mathematics. When you define a variable in a program the reader can easily infer the type by the value it was initialised with. Sure, if you just see "10" you don't know if it's a byte or a long int, but that doesn't matter because only the compiler needs to infer those details. For reading the code it's enough to know it's an integer. I agree about complex types, though. In Python and pure JS it gets really complicated when you have to find out about a dictionary's/object's fields by looking at code inserting them one by one.
This breaks exactly when you're dealing with more than one function. Then you have to guess the types of the whole chain of functions.
And maybe that function is used in multiple places, now you need to guess the types of all the places where this function is used.
Considering the use of features like auto, I personally prefer writing unspecified variables while programming and getting the statically inferred types live from the IDE, which for example aids when composing a complicated expression whose type I don't fully know while writing.
Clojure programs have a pulse a spark of life that I can mold at runtime with a bit of static analysis via clj-kondo it's a crazy productive creation environment
I am not entirely convinced of the validity of the latter argument.
But a powerful structural type system built on a flexible object model is really a great mix - type as much as you need, ignore types when they get in the way.
[1]: https://blog.cleancoder.com/uncle-bob/2019/08/22/WhyClojure....
> It is just simpler, and easier, and less occluding to write expressive code in Clojure. It requires fewer lines. It require fewer characters. It require fewer hours. It requires fewer mental gymnastics.
This paragraph is not only about "faster to write". It's about faster to understand, easier to maintain (less code to understand), easier to change and a lot more. I urge you to re-read the post again and think about what's being written,
> I think it's time to stop following what he things.
I think this is true for many "leaders" in programming. We should never follow people because they are (self-described/community-described) as "thought leaders", it simply leads to too much cargo culting. You see this in Clojure, VueJS and so many other ecosystems.
Ideas are good if they are good ideas, not good because of who said them.
> Ideas are good if they are good ideas, not good because of who said them.
You're right about that, I always insist a bit more with Robert Martin because there is a lot of cargo culting around what he says. Clean code and TDD are other good examples of that.
I have used dynamic languages a lot and I enjoy them in many ways, especially Python. But I've been using Go for a few years now and I'm hooked. For me, Go is pleasant enough that I don't mind its warts and I loooove the compile time checking. It has some other great features, tooling-wise as well.
I still love Python and I will continue to use it for the foreseeable future but I prefer Go when I have the option. If anything, Go has sort of rekindled my appreciation for static typing. I'm kind of hunting for a reason to learn Rust sometime soon.
Runtime is funtime with dynamic languages and I'm kind of burnt out on that. :)
Dynamic langs can be static (but without the guarantee that it always will be), static languages can be dynamic but with difficulties.
If guarantees are the issue, standards, language features/structure and constraints are the crux. Take a dynamic language, and explicitly label what is dynamic, and what is static, and you can restore static guarantees, while still having dynamic features if need be. Hybrid langs, with better reflectivity will win - you can have "both".
That said, software development models that don't trust their devs are always going to choose locked down languages, and claim this is the only "viable" way - much like those companies that claiming they cannot seem to hire employees, but don't mention "at the price we want".
I'm not shipping the tests. The fact that they work is very nice, and it can prove that the code under test is working, not that all attempts to use the code in something larger aren't broken.
So all that's left in this article is that dynamic languages make for easier to refactor code. That might be true, but it doesn't have to be so bad in practice when you keep your interfaces very small and focused.
- Most things you want to express can be expressed in a static language
- Tooling is much easier to build over static languages, and I think automated checkers are the future of rapid development of robust programs
I don't think that OOP languages as they are now (particularly Java 8) are the future, but something with less ceremony like an ML. Java and C# might flourish in the long term by becoming ML languages, but only if they are prepared to remove the cruft.
wtf argument
Stack Overflow's annual survey puts JavaScript, SQL, and Python at the top, with C at 12th [0]. RedMonk's Top 20 puts JavaScript, Python, and Java at the top, with C in 10th [1]. ITJobsWatch puts JavaScript, SQL, and C# at top, with C at 9th [2]. Dice puts SQL, Java, and Python as its top 3, with C not even ranking in their top 12 [3]. All of those put JavaScript, Java, and Python in the top 5.
[0] https://insights.stackoverflow.com/survey/2021#most-popular-...
[1] https://redmonk.com/kfitzpatrick/2021/03/02/redmonk-top-20-l...
[2] https://www.itjobswatch.co.uk/default.aspx?ql=&ll=&id=900&p=...
[3] https://insights.dice.com/2021/01/05/top-12-programming-lang...
On personal projects I only want to deal with builtin browser technologies, there is no npm, webpack, tsc, yarn, whatever, just plain <script/> tags.
I think at least for UI work there is a lot of space for dynamic plumbing together of well typed components. Components in this case meaning actual UI components as well as modules of business logic or state management.
Also the scale of the software matters, types are more important in larger systems. Single developer systems tend to be small.
For a compiler that can compile even with errors in the code, look at ECJ.
If you take HotSpot and attach a debugger in your IDE, then you have some hotswap capabilities [1]. You can redefine the contents of methods, add methods and add classes (e.g. lambdas). This is OK as far as it goes. But this is no inherent limitation of Java. In fact the debugger protocol lets you request arbitrary changes, it's just not all JVMs support them.
One that does is called DCEVM [2]. It fleshes out the hotswap capabilities far more extensively and lets you do things like delete methods, change prototypes, alter the inheritance hierarchies etc, all on the fly whilst the app is loaded and running.
DCEVM is an open source patchset onto HotSpot, but it's not really a pro level tool. JRebel [3] is a company that makes hotswap tools as a commercial product. Many companies use it.
These are a neat capabilities and I've used them a few times, but even with that level of power, it's not always complete for every use case. The problem is that most apps have initialized state. Web servers often do not, because they throw away state at the end of every request, and there it works better, but if you're working on e.g. a GUI or a CLI tool with a shell, or really anything where the program has to maintain state for longer than a split second, changing code becomes of more limited use because you are going to end up changing initialization code that won't re-run. You can add various ad-hoc hooks to your app to let you force re-init whilst running, but it's not totally smooth. Note: this isn't a limitation of the language or runtime. It's inherent to the concept of hotswapping code.
There is work to resolve even this problem. The Espresso JVM has full hotswap support, and a callback interface that lets the app learn when its own code is being redefined. It can use those callbacks to trigger re-initialization of any state that's required. [4] Espresso is still new but this sort of thing lights the way to apps that perhaps never need to be shut down at all, not even for upgrades.
[1] https://www.jetbrains.com/help/idea/altering-the-program-s-e...
[3] https://www.jrebel.com/products/jrebel
[4] https://www.graalvm.org/reference-manual/java-on-truffle/hot...
Whereas with Java majority of the time things work first try because any trivial type issue have been sorted already by the compiler.
I often wonder how much time (== money) I waste debugging existing code which has a logic error that would have been caught by any type system, or trying to piece together the type of some variable when writing new code. I expect it’s significant.
No. And neither will screws replace nails.
Amen.
One could also make the opposite and not contradictory point that type-systems make unit-tests less needed.
They both help
Will this article disprove the rule that whenever the headline is a question, the answer is always no?
Also no.
I recently wanted to make a simple enough PR to a large JavaScript codebase to fix a bug. After a few hours I found where the call was happening and what changes I needed to make.
At a simple level the code update was "measure the length of each of these things and store them, for each of the stored things align them relative to the longest length of stored items and add them to the state".
It took about 3 times longer to get to working code because of the lack of types. "Oh, you've used `i` in some context, you must just mean a global of indeterminate type, better keep running. Oh you forgot whether the method name on type of string is SubString or Substring? Sorry can't help you, don't know what a string is. You don't know/remember what you can do with this variable? Neither do I!" Sure you can have linters or whatever to catch these stupid mistakes but I'd rather just have the bloody thing work first time.
I'd rather have my tools assist me than feel like an alpha-chad because I've managed to memorise the entire API of something. I have more useful things to keep in my brain. Sure it might let you go quickly in your 50 line script but woe-betide any developer who comes later.
The raise of unit testing in the early 2003-2006, coincided with the raise of popularity of the Python web frameworks and Ruby on Rails. Suddenly Unit testing became a necessity, while previously it was a 'nice to have', as static typing covered a lot of the errors that dynamic languages introduced. Before 2005, in most places, 'integration testing' was the norm, but not unit testing.
In the other hand, Objective-C took a very interesting turn, where it is a static typed language, but it is message oriented, and you can use it in a semi dynamic typing way. You can send a message (method call) to any object, which allows for interesting compositions (eg. [(id)myObject anyMethod] ). Calling a method on a nil object, doesn't raise exceptions, etc...
I thought it made an awesome compromise. Unfortunately, Swift went the opposite way, which I didn't think was the right call. (as Java already covered that area well). Hence Swift hasn't taken off, appart from iOS as it didn't really introduce anything new, just little soup of paradigms that don't fundamentally make things much better. It was a missed opportunity.
I think size becomes a problem when you don’t properly modularize and test.
Also I think often it is more about the specific language in use rather than dynamic vs static typing. I would rather use Go than JavaScript. But I would use Julia over almost anything else when possible.
The theory that unit tests started happening around the rise of Python and Ruby is vague, unsubstantiated and intellectually lazy.
I actually remember the rise of TDD being intimately related to the enterprise world and Java but I’m certainly not going to pass that as scientific fact.