In theory weak dynamic typing works great. In practice humans suck. Hence checklists.
[1] https://www.newyorker.com/magazine/2007/12/10/the-checklist
It is absolutely critical in a discussion like this to understand that defects in software are not necessarily defects in data. This conversation is about data, and checks for processing data are already present in various forms. For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables?
Well, I don't know about the person you're replying to, but I certainly complain about these all the time.
Interestingly and counter to your point, I think, XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA), and SQL actually has a reasonably strong type system for most implementations albeit dynamically checked at query planning time. (SQLite seems to be the outlier here.)
JSON is actually the outlier here, but even with JSON there's been attempts at a sort-of type system with JSON Schema. Why? Well, because beyond a certain size the "just chuck some data somewhere in there" ceases to be workable when you want stable and maintainable interfaces between software components.
Ultimately type systems are invented and used for almost entirely pragmatic reasons.
The type system came from both XML Schema and XPath and was codified with XSLT 2, then expanded when XQuery was defined. It's not been a type system "of sorts" since XSLT 2, which filled in a lot of gaps and undefined behaviours.
>> You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language.
Correct, but in a statically typed language defensive programming is not required on your part to the same extent (because the compiler enforces a whole set of defenses on your behalf). You'll also forget sometimes, mistype sometimes, check the wrong things sometimes, forget to update some defenses. The compiler won't.
>> That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect.
The type system enforces you checked the data (by for instance, providing a conversion function `fn convert(input: String) -> Result<StructuredData, Error>` which then forces you to handle your error there, and then you don't need to check anything else later.
>> If a dose of common sense is still not enough... then program in TypeScript.
That's half a solution because unless your entire ecosystem is typescript those types are at best aspirational. An improvement to be sure, but still aspirational.
Which, IMO, can all be summed up by "we have automated checks so humans don't have to write them and make mistakes doing so as they go". Outsource the checks to an automated system.
There is not much of a difference, theoretically speaking, when data is passed between functions inside an app, or between apps via some network api. What you describe is actually a real issue in typing. Types are lost across systems. JSON is not strictly typed, neither is XML or SQL. So if I have a typed application that spits out data to these formats I lose type checking. If you think about it, even jumping across abstractions from one typed language (say rust on the backend) to another typed language (say typescript on the frontend) you will lose type safety due to incompatibilities between languages.
I don't know if there's research into or thought going into solving this issue. But a universal type language/syntax that can be translated into respective type systems across api abstractions could lend a lot to safety of entire ecosystems. There is no type safety across http services and designing a system to overcome this issue is very possible and to my knowledge not something people have thought about yet. New idea?
Imagine a language/framework that compiles into web apps and microservices on some cloud provider. The language has syntax that allows me to decorate functions to specify which microservice it runs on. Data transfer between microservices can now be placed in some binary format or maybe the language has syntax that allows me to specify human readability. Either way the language abstracts away the devops and webapp layer and combines it into a single source code... this would allow type checking of the shape of data to not happen just within an app, but across services.
Is there such a framework/language that does this? I'm positive it can be done. I would totally love to work on an idea like this. The microservice fad talked up modularity between services but in practice people realized that microservices are too complicated. Perhaps an entire language framework that combines the web app layer and dev ops layer and adds type checking across services would solve this issue.
XML, well, there's the schema language for that, but that's fairly "outdated" and I haven't touched it in ages, but if you've ever looked at an HTML parser, it's borderline sentient because of weak typing.
JSON? That's a huge source of issues. I've run into them professionally and personally, and they cause outages all the time. Thats why protobufs are great (schema) and GraphQL with generated interface is great.
type Person { name: String }
function parsePerson(json: String): Person { ... }
function usePerson(person: Person) { ... }
Whether the language runtime also uses type checks while running depends entirely on the language and the runtime.
IIRC, the JVM (to take an example) more or less throws away a lot of the static type information -- which is often too generic to be useful in guiding optimization -- in favor of runtime profiling, and several of its key optimizations involve inserting runtime type checks.
In practice, with parsing, this usually means that runtime type metadata is used. Effectively, types become your schema, and you can validate against them as needed.
Then again, if both ends use schema, then comparing schemas is easy and gives you fair chance to fingers the place that differ between systems.
If there is one place or lib as a source of schema on both ends, then mismatch is just a result of different version being imported by them - something easy to check.
The difference is that in a statically-typed system, you're validating data at your boundaries such that they comply with your nominal types that are then compiler-guaranteed to be internally consistent.