Fear, trust and JavaScript: When types and functional programming fail
reaktor.com
reaktor.com
It's really great to have typed Records (often nested), and I expose the getters as regular properties so that I don't have to call "get()" all the time. I agree that it's not perfect, but it's miles ahead of anything I've worked with in the past (at least in terms of JavaScript.)
JSON schemas are awesome, and I like the idea of validating types at runtime on the front-end. I already have a JSON schema that defines my API (I use Swagger), so I could even go one step further and auto-generate my typed Records directly from the schema. I'll get immediate feedback about any type errors in my JS codebase as soon as I make changes to the API. I generate my Swagger API specification automatically from some RSpec tests (using rswag [2]), and I also run Flow during CI, so this would result in a broken build. I could also make this work with the data I'm inlining in the HTML to hydrate the Redux store.
Random aside: I also have a one set of data that just gets dumped into a jsonb database column, so the schema for this specific data is only being defined by the front-end code. In my backend I'm always careful to provide a default value, but it's not really critical data. Anyway, I just realized that I should be defining this jsonb column as an actual model, but instead of ActiveRecord, I could use ActiveModel and a plain Ruby class. That's a neat idea. It would give me a schema and validations on the backend, while still having the flexibility and performance of storing everything in a single json column.
[1] https://gist.github.com/glenjamin/75a96b45f4bb5c6ac221815d28...
>Now we compare the runtime performance:
BuckleScript Immutable Map: 1186ms
Facebook Immutable Map: 3415ms
BuckleScript (Prod mode): 899 Bytes
Facebook Immutable: 55.3K Bytes
https://github.com/BuckleScript/bucklescript/wiki/Why-buckle...
https://medium.com/@javierwchavarri/performance-of-records-i...
It seems like half of the time you’re just using the escape hatch to plain JS to stitch all of this stuff together, no?
It is the least leaky abstraction I have ever encountered, which is amazing given the fact that it is also blazingly faster than every other framework from React to Vue. Elm is a way higher level abstraction then jsx or Vue yet you don't need to understand the lower level details to produce more efficient applications.
The most leaky abstraction I have ever encountered is probably SQL. Optimizing SQL involves understanding what it compiles into; Elm there is no such understanding required.
SQL is a leaky abstraction not because of scale. Scale is a requirement and the chosen abstraction for scale is actually not the wisest choice. SQL is an expression based language used to execute algorithms like binary search that are inherently imperative. Thus it is a bad abstraction and the scale of the requirements magnifies this. The need for the EXPLAIN ANALYZE key words exemplifies this flaw. A good abstraction for databases would be a language that explicitly declares the algorithm used for the query. This would make users able to explicitly optimize the query without needing to cross the barriers of abstraction. Like googles V8 implementation of javascript, ubiquity and tons of resources invested into creating an incredibly efficient implementation of an interpreter have made SQL and javascript the best choice for their respective fields even though they wouldn't have been the best design choice if people were given the choice to start over.
Rust is a better design then C++ but again ubiquity, early adoption and random forces have made languages with questionable design choices the dominant paradigm.
- JS Interop is harder in elm.
- Inferno and some others are a lot faster than elm.
Never heard of inferno... I'll check it out.
Then I wanted a fast lookup table, turns out all dictionary lookups in Elm are nlogn because it's backed by an immutable tree structure.
So, I disagree with you and went back to JavaScript.
Doesn't that prove my point, that there's an impedance mismatch of Elm trying to present a purely-functional world on top of a very side-effect-full environment of the browser+DOM etc.?
> but the practicality won't be apparent until you create a huge 10 million LOC monolithic SPA with elm
It sounds like you're arguing that for apps smaller than "absurdly large", plain JavaScript can be better.
Calling bs on anyone having anything approaching 10 million LOC in Elm. You're just making things up.
Edit: for reference, the Linux kernel has about 20 million LOC (according to Wikipedia), so you're talking about half a Linux kernel written in Elm. Absurd.
Your reply doesn't even address my point. That's all I'm saying. All I said is that I disagree with you.
>It sounds like you're arguing that for apps smaller than "absurdly large", plain JavaScript can be better.
>Calling bs on anyone having anything approaching 10 million LOC in Elm. You're just making things up.
>Edit: for reference, the Linux kernel has about 20 million LOC (according to Wikipedia), so you're talking about half a Linux kernel written in Elm. Absurd.
Don't ever call what I say absurd or BS that's plain offensive and against the principles here on HN. If you want me to continue replying to you, I'll need an apology otherwise I will ignore you.
(Unfortunately, the JS runtime object model does not allow for such runtime type checks to be performed with reasonable performance.)
It's disturbing how often I encounter this in TypeScript. Or its inverse: the types are correct and the compiler is wrong. Or a third common problem: the types for a library are incorrect. The unsoundness of TypeScript is not merely theoretical. The compiler is frequently just wrong. For that reason, I am mystified by the amount of enthusiasm for TypeScript that I encounter online.
I have been using Flow (mostly) and TS (occasionally, including counter-checking problems in Flow to see what TS does in a similar case). Overall I would not go back to the time without a type checker, however, I too am mystified by the enthusiasm. Anyone who wants to check the state of static type checking for Javascript should take some time and read through
- https://github.com/Microsoft/TypeScript/issues
- https://github.com/facebook/flow/issues
I do that quite a lot myself and have also contributed a tiny bit to both projects (no core code, things like definitions, small doc improvements, quite a bit of answering to issues and often checking the posted code for myself, often in both Flow and TS).
Ignore the issues posted by people who really would just need a forum to ask questions, there are plenty of real issues left. Worst part: Many of them won't be solved (too hard, too much work, too many issues overall). You have to change your coding style and write in a way that the type checker can actually help you with. Also, it's easily possible to end up with types that are far more complicated than the code they are supposed to describe.
I spent more time working on the types than on the actual code. What makes it worth it is that one, I get some control over the code other people write using my library, if I insist they too use the type system I can prevent them from misusing the API to a degree. Two, those other people also includes myself in future incarnations. Three, refactoring can be significantly easier, if you have good types the checker will tell you all the places you missed changing.
So overall, at least for my situation, mostly for writing a library with few external dependencies (so I don't need the more or less unreliable external type definitions) that the business heavily relies on in many products, adding the very considerable additional effort is worth it. Still, I very much disagree with all the enthusiasm.
The type checkers are software trying to understand software, and that software that it's attempting to check is not just an already complex dynamic language, but in addition on top of it is people's code that comes in a million styles. Those type checkers can be valuable, but they are far (very far) from perfect, and they come at a considerable price (mostly in the time it takes to create and maintain the types, I think the additional step to remove type annotations for production code is pretty insignificant in comparison).
That's interesting. I very strongly don't, when writing TypeScript (but I don't then also mess around with a second type system on top of it). Better Intellisense and fail-up-front checking means I write code much, much faster.
I didn't have to change my coding style because this is already how I wrote code; I now have the tools to actually do it well.
TypeScript is not a "complex dynamic" language unless you step outside its bounds. Which, sure, sometimes you need to; it's not perfect, and one of its advantages is being able to opt out when absolutely necessary. Then you fence off that type-unsafe code by strongly testing before you hand something back into TypeScript because you're a competent programmer who understands the limitations of your tools. But that happens so very, very rarely that optimizing for that corner case seems foolish.
> TypeScript is not a "complex dynamic" language
I don't know what you read into my comment, but if you just stick to what I wrote, Javascript certainly is, and TypeScript is just Javascript (the type annotations are a separate thing). I'm not sure what your point is overall I have to admit, it's a bit on the defensive side for no reason that I can see. For whatever reason you seem to feel personally attacked ("I didn't have to change my coding style")? I refer back to what I wrote, point for point. Would it please be allowed to write down my observations? Especially when it is base don years of practice in all the relevant technologies (JS, TS, Flow) and I'm not just making stuff up without having data (i.e. actual experience). Plenty of other people wrote similar comments here.
JavaScript is quite complex and quite dynamic, but the reason TypeScript got me back into doing web stuff was, by and large, because it removes that except for in clearly delineated places (at least, once you turn on strict mode).
You're reading in some stuff that isn't intended.
What a coincidence, same thing I said to you! Why do you make a stupid reply when you know it's stupid?
It's still a million times better than JS without TS/flow.
While the alternatives might be in some ways a better start a greenfield project in, the ability to gradually add types to an existing codebase with developers who are just learning the language with the ability make the system sounder over time is a killer feature for me. Also, the big-company backing helps corporate slaves like me trying to sell a new technology to management.
The flags prevent two behaviors that are a major source of type failures: defaulting some types to "any", and allowing null/undefined to be valid values for ANY type (!). Enabling these flags for an existing codebase is likely to uncover hundreds of type errors that are papered over with the default compiler settings.
W.r.t. upstream types for libraries being wrong, I find the silver lining here is that libraries will usually be happy to accept patches for their types, often with significantly less fuss than a patch to the actual code.
edit: Even with those in place, I also notice that Typescript types are not 100% trustworthy. This often comes down to unsanitized input, use of the "as" keyword, or hidden type violations. The conclusion I've come to is that it's just very hard to type Javascript code, partly because the values themselves are so prone to being mutated. So I agree that Typescript doesn't feel like the perfect solution to me. (I usually use Clojurescript these days).
To be clear: I'm not complaining about these third party types. I really appreciate that people took the time to write and share these. My gratefulness is definitely not contingent on perfection. I fully expect human work to be imperfect. Type systems exist to guard against this imperfection. A type system in which type annotations are maintained separately from the implementations, and are routinely debugged is highly suspect.
I will have a quick skim through the @types/ramda issues, thanks.
There is a nice little library [1] that fixes this issue.
So a general redux setup looks as follows:
const FETCH_USERS_BEGIN = "@@FOO/FETCH_USERS_BEGIN"
const FETCH_USERS_SUCCESS = "@@FOO/FETCH_USERS_SUCCESS"
const FETCH_USERS_ERROR = "@@FOO/FETCH_USERS_ERROR"
const fetchUsersBegin = () => action(FETCH_USERS_BEGIN)
const fetchUsersSuccess = (users: IUser[]) => action(FETCH_USERS_SUCCESS, users)
const fetchUsersError = () => action(FETCH_USERS_ERROR)
type IActions =
| ReturnType<typeof fetchUsersBegin>
| ReturnType<typeof fetchUsersSuccess>
| ReturnType<typeof fetchUsersError>
interface IUser {
id: number
name: string
}
interface IState {
readonly isLoading: boolean
readonly isErrorLoading: boolean
readonly allIds: number[]
readonly byId: {
readonly [key: number]: IUser
}
}
const defaultState = {
isLoading: false,
isErrorLoading: false,
allIds: [],
byId: {}
}
const reducer = (state: IState = defaultState, action: IActions) => {
// narrowing on types
switch (action.type) {
case FETCH_USERS_BEGIN:
return { ...state, isLoading: true, isErrorLoading: false }
case FETCH_USERS_SUCCESS:
return {
...state,
byId: action.payload.reduce(
(acc, u) => ({ ...acc, [u.id]: u }),
{}
),
allIds: action.payload.map(u => u.id),
isLoading: false,
isErrorLoading: false
}
case FETCH_USERS_ERROR:
return {
...state,
isLoading: false,
isErrorLoading: true
}
default:
return state
}
}
However, those of us that are in love with TypeScript often want to refactor existing JS into TS, where allowing implicit `any` and allowing `null` make it a bit easier to convert larger codebases.
So yeah, go use a proper functional language (many of which compile to JS!). Complaining that an imperative language isn't a soundly-typed functional language is tautologically true I guess, but who cares?
For example, the idea that you can't trust Typescript because it might call untyped code but you CAN trust Purescript is just wrong. It's not even just wrong, it's utterly misrepresentative of what actually happens in Elm and Purescript. It's misinformation.
All pure/statically typed languages have escape hatches. It has been leveraged as a counter-argument to using them. But but but, they will say, Haskell has unsafePerformIO!!! Well, yeah, sure it does. You can subvert the borrow checker of Rust, too.
But you have to do it explicitly. You can grep your code base for "unsafe" (literally). You can lint for that. You can code review for that. You can document it. And that's what makes it OKish. In a way, types are like big fat oven mitts, and you're the lead programmer of a bakery, where you pop buns in the oven all day. You can discard the oven mitts for a time, though, and do whatever delicate work you need your actual fingers for. Just don't touch the hot stuff, lest you get yourself burned!
Because every step towards that line has not just cash value, but exponential cash value if it’s a startup.
And by constantly pushing themselves into unfamiliar territory coders develop the most impressive possible resumes. They either have relevant experience or have experience in a tech even beyond what the company they are interviewing is using. That’s your best position to get hired.
It’s easy to write JavaScript you can trust. Heck, you can write a trustworthy web server in Visual Basic. Trustworthy code just isn’t worth much to 99% of the players involved.
I worked on the system which is now the customer admin for Square’s ecommerce platform they bought in Weebly.
We were all learning Vuex, while also trying to ship code. Across a half dozen pages we had a half dozen slightly different ways of addressing Vuex data. It wasn’t so much complexity that a professional coder couldn’t keep it straight in their head. But it was a trivial amount of complexity to fix. Two days work, maybe a week. Mostly variable naming.
But we chose to ship that feature because we wanted to prove to the company we could deliver. Now those coders will live with that choice for a log* time. New pages will use another slightly ad hoc scheme.
Eventually some architect will decide something like “Vuex is too conventional, we’ve got too much glue code, should we try switching to Redux or Vuedux?”
And the correct answer for the devs is: yes, it’s good for your resumes.
And (speculation) the engineering management will probably never swallow “we have to do three weeks of variable renaming and function moving to fix this now”. If they couldn’t swallow two days of cleanup in exchange for meeting an arbitrary internal target, they won’t swallow three weeks which probably pushes back a user-visible release.
* I meant to type long, but log time, pun intended
Using your example but applied to other companies, I've seen the below, or some combination of all:
* Unskilled devs that have never used any of the new technologies and spent little time prepping beforehand.
* Management who demanded unrealistic deadlines or demanded devs skip design phase in favor of shipping faster.
* Lack of technical leadership, senior engineers are hard to come by at many large orgs and are often overworked or too busy to be everywhere at once.
* Cut throat career minded individuals that always jump ship to the next new thing.
In your case, it seems insane that you would be writing software with multiple teams or even multiple developers without some overarching plan. It shows a clear lack of technical leadership on the part of management and senior engineers.
Personally I learn new technologies if they are fun to learn and make my life as a dev easier. That's it. I'm not sure if anybody jumps on wagons just for the sake of it. For example I tried out Rust, Julia and Elixir, hated the first but liked the later two. I also tried Vue a bit but with Phoenix framework a lot of things can just be done with server-side rendering so I just dropped Vue in the project and felt totally fine with it.
Except by “dump every six months” it’s more like “after six months don’t add much to that anymore, unless we can dedicate a good three weeks to hacking it into place. And then dump the whole component in three years”.
Since your customer base and business requirements are likely to turn completely over in 3 years this makes some (perverse) sense for a startup.
But writing that code is HARD. Training a growing team to write that code is hard. Banging out new features, on the other hand, is (at first) easy and seems more fun.
Personally I learn new technologies if they are fun to learn and make my life as a dev easier. I'm not sure if anybody jumps on wagons just for the sake of it. For example I tried out Rust, Julia and Elixir, hated the first but liked the later two. I also tried Vue a bit but with Phoenix framework a lot of things can just be done with server-side rendering so I just dropped Vue in the project and felt totally fine with it.
The author claims "functional programming, types, JavaScript: pick two". He supports that with links to a number of libraries that have incomplete type definitions, and a few patterns that are hard to express in a static type system.
I don't think that assertion holds. Some JavaScript APIs are indeed written in a style that's very hard to give static types for, and some libraries have sub-par type definitions.
But odd JavaScript APIs != the entirety of functional programming, and sub-par type definitions are just missing features that need to be fixed. You can write perfectly fine, statically typed, functional programming idioms within the TypeScript type system (and presumably within Flow). You can even do that within the confines of Java's type system, and that's a lot weaker than TS' or Flow's.
These type systems are optional, and you can subvert them. If you use the recommended strictness flags though, it's hard to do so accidentally.
You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language.
That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect.
If a dose of common sense is still not enough... then program in TypeScript.
In theory weak dynamic typing works great. In practice humans suck. Hence checklists.
[1] https://www.newyorker.com/magazine/2007/12/10/the-checklist
It is absolutely critical in a discussion like this to understand that defects in software are not necessarily defects in data. This conversation is about data, and checks for processing data are already present in various forms. For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables?
Well, I don't know about the person you're replying to, but I certainly complain about these all the time.
Interestingly and counter to your point, I think, XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA), and SQL actually has a reasonably strong type system for most implementations albeit dynamically checked at query planning time. (SQLite seems to be the outlier here.)
JSON is actually the outlier here, but even with JSON there's been attempts at a sort-of type system with JSON Schema. Why? Well, because beyond a certain size the "just chuck some data somewhere in there" ceases to be workable when you want stable and maintainable interfaces between software components.
Ultimately type systems are invented and used for almost entirely pragmatic reasons.
The type system came from both XML Schema and XPath and was codified with XSLT 2, then expanded when XQuery was defined. It's not been a type system "of sorts" since XSLT 2, which filled in a lot of gaps and undefined behaviours.
XML, well, there's the schema language for that, but that's fairly "outdated" and I haven't touched it in ages, but if you've ever looked at an HTML parser, it's borderline sentient because of weak typing.
JSON? That's a huge source of issues. I've run into them professionally and personally, and they cause outages all the time. Thats why protobufs are great (schema) and GraphQL with generated interface is great.
>> You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language.
Correct, but in a statically typed language defensive programming is not required on your part to the same extent (because the compiler enforces a whole set of defenses on your behalf). You'll also forget sometimes, mistype sometimes, check the wrong things sometimes, forget to update some defenses. The compiler won't.
>> That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect.
The type system enforces you checked the data (by for instance, providing a conversion function `fn convert(input: String) -> Result<StructuredData, Error>` which then forces you to handle your error there, and then you don't need to check anything else later.
>> If a dose of common sense is still not enough... then program in TypeScript.
That's half a solution because unless your entire ecosystem is typescript those types are at best aspirational. An improvement to be sure, but still aspirational.
Which, IMO, can all be summed up by "we have automated checks so humans don't have to write them and make mistakes doing so as they go". Outsource the checks to an automated system.
There is not much of a difference, theoretically speaking, when data is passed between functions inside an app, or between apps via some network api. What you describe is actually a real issue in typing. Types are lost across systems. JSON is not strictly typed, neither is XML or SQL. So if I have a typed application that spits out data to these formats I lose type checking. If you think about it, even jumping across abstractions from one typed language (say rust on the backend) to another typed language (say typescript on the frontend) you will lose type safety due to incompatibilities between languages.
I don't know if there's research into or thought going into solving this issue. But a universal type language/syntax that can be translated into respective type systems across api abstractions could lend a lot to safety of entire ecosystems. There is no type safety across http services and designing a system to overcome this issue is very possible and to my knowledge not something people have thought about yet. New idea?
Imagine a language/framework that compiles into web apps and microservices on some cloud provider. The language has syntax that allows me to decorate functions to specify which microservice it runs on. Data transfer between microservices can now be placed in some binary format or maybe the language has syntax that allows me to specify human readability. Either way the language abstracts away the devops and webapp layer and combines it into a single source code... this would allow type checking of the shape of data to not happen just within an app, but across services.
Is there such a framework/language that does this? I'm positive it can be done. I would totally love to work on an idea like this. The microservice fad talked up modularity between services but in practice people realized that microservices are too complicated. Perhaps an entire language framework that combines the web app layer and dev ops layer and adds type checking across services would solve this issue.
type Person { name: String }
function parsePerson(json: String): Person { ... }
function usePerson(person: Person) { ... }
In practice, with parsing, this usually means that runtime type metadata is used. Effectively, types become your schema, and you can validate against them as needed.
Whether the language runtime also uses type checks while running depends entirely on the language and the runtime.
IIRC, the JVM (to take an example) more or less throws away a lot of the static type information -- which is often too generic to be useful in guiding optimization -- in favor of runtime profiling, and several of its key optimizations involve inserting runtime type checks.
The difference is that in a statically-typed system, you're validating data at your boundaries such that they comply with your nominal types that are then compiler-guaranteed to be internally consistent.
Then again, if both ends use schema, then comparing schemas is easy and gives you fair chance to fingers the place that differ between systems.
If there is one place or lib as a source of schema on both ends, then mismatch is just a result of different version being imported by them - something easy to check.
Now with static type checking. That program wouldn't even compile. The compiler can prove that the program is wrong rather than you going through hundreds of unit tests to try to catch an error while proving nothing. Proving a program is wrong is better than writing billions of unit tests that cover the entire domain of a function.
Your strategy of using common sense to go through 10 million (exaggeration of course) lines of code is an effective strategy, but not an intelligent one.
I think that the focus now should be allowing other more modern languages in the browser by means of Web Assembly. Rust is a great candidate for dealing with immutability and unsoundness concerns. I can't wait for wasm to be fleshed out more.
It does beg the question though, would web development be as ubiquitous and as popular if the core language in was statically typed? Does the freedom JS offer you allow greater adoption overall, meaning that it's beneficial to loosen the type restriction as the ecosystem will flourish because of it?
TA mentions 3 languages (Elm, Reason & Purescript) that combine sound type-checking and immutability right now.
None of these bring the cognitive overhead of Rust’s borrow checker and, in the case of Reason at least, compile at lightning speed to readable (though optimised) JS.
I've seen some of the worst java apps before, but no one says "never use java". It's generally accepted that skilled engineers can make clean, easily maintainable apps, and unskilled engineers make garbage apps.
This just sounds like sloppy engineering and a total lack of professional rigor.
It's definitely an anecdotal observation though.
The thing to realize is that at the end of the day, all you are doing is reducing bugs, and while your type system (IMO) really can help reduce bugs, it will never catch all bugs. There is still value in a function declaring that its argument is of type User (and not defensively checking it) even if it is theoretically possible, in the presence of a bug, though unlikely, that the argument is not really a User. The function has said what its precondition is, and (assuming you are validating any data that comes over the network or from an untyped source), this precondition will in practice be checked 99% of the time. If a bug happens to line up with a hole in the type system, you’ll just have to debug and fix it.
Even those don't have to be escape hatches as such, in the same way e.g. List.head doesn't have to be partial, that was an explicit decision.
When casting from rust's Any trait to a concrete type, you get an option. There's no subversion of the type system, there is no runtime error unless you as the developer decide to generate one.
[string]$String = "Dog";
$String = 5;
Cannot convert value 5 to type "System.Objects.String" var s: string = "Dog";
s = 5;
"There is still value in a function declaring that its argument is of type User (and not defensively checking it) even if it is theoretically possible, in the presence of a bug, though unlikely, that the argument is not really a User."
I'm trying to think of a statically-typed language that would allow such a thing, and I can't, so I assume that you're referring to TypeScript here where you have a world where things are "partially-typed" ?
Is there a vision for how PureScript/ReasonML/Elm etc. can accomplish the same thing?
The brute force approach is to place a boundary between everything external and handle it via an FFI. This adds friction: apps built under this approach are less capable and evolve less gracefully. I hope there's a better solution.
Mint to Javascript is what any languages to web assembly.
I’m not sure who is comfortable to evaluate Real World code example: https://github.com/mint-lang/mint-realworld
Along with some Crystal shards (LLVM, assembly, C librares) to validate your data or other structure formats is a breeze to work in.
You have a `JsObject` type (which is basically `any`) which lets you write your Nim->JS code in a dynamic way and cast to static types whenever you need to interface with static parts of your program. You can still also define the type signatures of a JS API as well and have a fully type safe experience, but you can do it lazily: just for the functions/types you need.
Basically you can use a JS api in any way you want: from almost fully dynamically to gradually more and more type safe way
[1] https://github.com/jashkenas/coffeescript/wiki/List-of-langu...
I really like Rust personally and think it's a good option for web applications, but there's many options that are becoming available.
As someone who has (almost entirely) given up writing plain JS, my skimming of this article is "2500 words about issues with JS mutability, then: just use ClojureScript".
That's why we have both tests and code reviews.
Aside from that I'm surprised the author didn't mention checking constructors. It's arguably the most dependable way of checking if something is of a certain type.
But I agree that it's a shame that TS/Flow lose their ability to check types whenever partial application or currying is involved - it's not like it's impossible in principle.
Nobody ever cared to use tests to replicate what a type system does. That's a straw man. Tests go straight for the actual goal by testing actual values. Correct values are what are necessary. Types on the other hand are neither necessary nor sufficient.
Frankly, I don't even know what to do with such a statement. What is your definition of "internal" ?
I just did a very large re-factor of a codebase a few days ago where I changed a lot of event handler (method) types in a codebase of around 200K lines of code (Object Pascal). This means that the method signature of every single event handler could have been wrong in some way if I made a mistake, and it would have introduced a pretty egregious, non-internal bug in every case. How would you test such a massive change without replicating every single thing that a static type system and compiler already do for you ?
None of these projects had unit tests, but that's fine - you don't really need them for this sort of modification, just some patience, a tolerance for boredom (that, or the ability to suck it up anyway), and a complete lack of concern about messages such as 'ERROR: 0/1,577 file(s) built. NOTE: Error limit exceeded - only showing the first 99,999'.
I haven't touched OO Pascal in 15 years, so I can't speak to how you would write tests for it, or if there's even a testing framework. I will tell you that I left the if-it-compiles-it-works attitude around that time too though.
Re: your definition of "internal": I suspected that this was what you meant. I also disagree with this particular version of "correctness". If the developer intended that a particular function accept an integer, then accepting anything else is essentially a bug, and the fact that it works at all is down to pure coincidence.
As for writing tests in OP: you write tests in the same manner that you do with Java, C++, C#, etc.
Your statement that "nobody ever cared to use tests to replicate what the type system does" is trivially false, have you ever tested what happens when you pass "null" into a function that probably shouldn't take null? If so, you just did. Some type systems allow you to even specify valid subranges of types, and combined with algebraic data types, it's effectively impossible to misuse.
That's not necessarily true though is it? If the "1" in the "1 hour ago" message in the header of your reply wasn't the expected type of the writer of the output function but ultimately still shows up as "1 hour ago" to the user, there's no user-facing defect. I can promise you that this happens all the time in javascript. You're in a thread about Javascript.
> have you ever tested what happens when you pass "null" into a function
Not that I recall. Javascript will throw an appropriate error when it detects that your duck isn't quacking correctly. No one writes validation logic everywhere on internal units, so there's no functionality to verify with a test.
With that said, null is decidedly not "just a type". It's also a value. I generally avoid using null everywhere as much as possible too, because it's a pretty terrible concept.
> Some type systems allow you to even specify valid subranges of types, and combined with algebraic data types, it's effectively impossible to misuse.
Sure but you're in a thread about javascript and typescript. And let's be honest: the vast majority of type systems do not support that.
Using Javascript as a platform where you mix code generated trough different conventions and tools is a halfway solution. Javascript treated as the unmodifiable binary from the compiler creates same level of trust as using other languages.
Technically:
* In C++ somebody can use different header for the class and break the class abstraction.
* In Haskell some library might go trough the FFI into the runtime and assign values to data in non-functional way and break the functional abstraction.
* In Python and Java there are ways to access data in ways that circumvent the compiler or the language semantics.
The difference is how strong the convention against doing so is and does breaking the abstraction have any benefits.
If (user && user.name) { ...
This type of developer provided shorthand validation is pretty much the cause of all hacks/data breaches currently. It’s more of an attitude or how a developer prioritizes their own wants/needs over maybe a client/end user. Like I mentioned in another comment, checking for more advanced, abusive behavior will make issues with type checking seem basic in comparison.
For example in Scala, this example would be that user is of type Option[User] and the name property would also be Option[String] for example.
Then to safely get the name would be:
user.flatMap(_.name)
and the resulting type would be Option[String]
Which would then carry on the info that you don't know for certain that you know the name.
The problem isn't that `user && user.username` style validation is verbose, but that it's very error-prone. Making it less verbose just means you'll more likely use it before you reach for a better validation strategy, like a declarative validation library.
Type checkers are tools that help us reason about, structure, and correct code. That's all.
Flow is decent these days but will probably never be as powerful as Haskell. My only gripe is that it forces my team to write verbose code at times in order to gain the benefits: you lose point-free style, gain a bunch of identity and instance checks... but it's a small trade off to make.
One small trick worth remembering and using in Flow to gain something similar to exhaustive type checking is to cast to the empty type in the default case when pattern matching on a type[0].
It's not an elegant type system but if you have experienced a good type system (like Haskell's) then it does make using Flow/TS much easier.
[0] https://medium.com/@ibosz/advance-flow-type-1-exhaustive-che...
Being dynamic doesn't preclude variables from being type checked by the runtime. Reading this just sounds like "javascript has no built in support for type checking, so lets pretend it's not possible"
All the time people spent making weird unreadable 'fat functions', but you still can't define the types of arguments you want to receive?
So if I wrote a library in ts and someone called the built js directly it wouldn’t have any type checking would it?
This may not be a popular opinion for those coming from C or Java, but runtime type checks in JavaScript are usually an anti-pattern. If my function defines a contract that the caller violates by passing in null or undefined, then my function should raise an exception when it tries to access `undefined.xyz` or whatever. Otherwise, embrace the duck! If the function can work with what it was given, it should instead of trying to enforce some Java-esque type system at runtime using typeof or instanceof.
But be pragmatic. If there is some critical area of your app that absolutely must never ever throw, typecheck away (although this is JS after all, so it still may throw if it damn well pleases).
How do you know that it "can work" based purely on the name of the function?
> But be pragmatic
I am. I consider javascript a last-resort option, only really useful for adding minor enhancements client side. Beyond that, I don't use it.
JavaScript is no different from most other dynamic languages this way
This is exactly why I can't take javascript seriously for anything besides basic UI enhancements to a web page.
Use whatever tools work for you, but also don’t disparage or dismiss other tools because they have different strengths and weaknesses than what you know or appreciate. There is no need for proselytizing; these are just tools after all!
I like JavaScript and TypeScript because they have let me quickly build many great things and share them with people. I also subjectivity find writing in them to be enjoyable. I may always turn to a different language if it is more productive and enjoyable than js/ts for the task at hand.
If the runtime can guarantee your function receives an instance of a particular class, that's a lot of boiler plate you don't need to write yourself to assert the same thing.
My point was that if your TS project is used in a non-TS context (i.e. the compiled JS is called from regular JS code, not compiled TS code, you no longer have that compile step.
I also think you've misunderstood what I mean. I'm talking about native language level support for typed function arguments at the javascript level, not relying on TS to provide that.
What you're saying is that explicit type annotations can be enforced at compile time. However, they can also be enforced at runtime, and they're no less safe for that - you just see the errors later, but any call that would be blocked by a static type checker can similarly be blocked by a dynamic checker. I think that's what OP refers to by "runtime checking", not writing explicit asserts.
A similar example that web-focused developers are likely familiar with, is type hinting in PHP. For the developer's purpose, there is no 'compile' step (yes the code is compiled and then run, but it happens at request time so its not realistic to compare it to compile time type checking in a compile language like Java).
The 'compile/run' lifetime (in browser based JS) bears similarity to that of PHP: compile time effectively is run time, because it's compiled during the request and then executed immediately.
It kinda does, because in a dynamically typed (as colloquially defined - if you want to debate that definition, that's another discussion) language, variables don't have types - values do.
Nothing precludes values from being type checked.
There are contexts in which two values are involves, where we can at least check that the types are matching (or reasonably compatible), like (a + b). JS is a massive failure in that regard, too, and there are dynamic languages that do it far better - e.g. in Python, ("1" + 2) is a runtime type error, not 3.
But in the context of this thread, I feel that's not what we're talking about. We're really talking about contracts - and contracts require typed bindings (variables, function arguments etc), not just typed values. Which means that they do require static type annotations. As we've discussed in another subthread, those annotations might be checked dynamically - but they still have to be declared statically by the coder, which makes the type system effectively static from their perspective.
To put it differently: if you require types to be declared explicitly, there's no reason to not check them statically. Dynamic checking of static types really only makes sense for gradual typing systems, where code with static type annotations might be called from some non-annotated code, and you want it to be possible without forcing static typing on the caller. At that point, yes, you can have a dynamic type check in the callee, which ensures that if code without type annotations doesn't actually get the types right, things fail fast at the static/dynamic typing boundary, instead of allowing it to propagate into the statically typed code (as is the case in e.g. TS today).
I think some of this thread is suffering from confusing terms so, my entire point is this:
There is no native way in javascript to say that the first argument of function foo must be an instance of class Bar (which in JS could be a string or an array or some custom object or even a DOM element - everything is an object and thus has a 'class').
The common response is "use typescript" which means you write the code with argument type declarations, and then compile to javascript. This (in theory) means any code you've written in the project in TS which calls our foo() function, will warn/error if the first argument isn't an instance of class Bar.
But, and this is the problem: it's not enforced by javascript, it's "enforced" (for whatever definition of enforced typescript uses) by the TS compiler.
So if I then use the compiled JS of function foo() and call it from native JS (i.e. not compiled by TS) - it won't tell me I have the wrong type.
THAT is my issue.
Getting back to your issue - it's a pragmatic decision for TS, because enforcing its type system efficiently is virtually impossible on top of the JS runtime type system, when you consider how it has to work. It sounds like simple, obvious check for trivial cases, like if I annotate a function argument to be an number or a string. But what if it's an interface? At that point, the runtime check must ensure that 1) it's an object, 2) it has all the members listed in the interface, and 2) they have the corresponding types. If they are themselves interfaces, this becomes recursive. If they're arrays or maps, then this has to be done for every element. An array of arrays is O(N^2) already!
To check it efficiently, you need to have proper runtime type tags - an array has to know that it's an array of number, not just an array. Then the runtime checker can just query that, like it does in Java when you, say, downcast Object to int[]. But TS has to work on top of JS runtime, so any such runtime type tagging would manifest as visible artifacts in JS; and TS is intentionally designed to be "invisible" from JS interop perspective.
If we want quality strong typing that works properly with cross-language interop for the web, we need to throw away the JS runtimes entirely (it would be a good idea in any case - it's an atrocious object and memory model for anything other than JS itself), and replace them with something saner.
> Adding types to our example doesn’t solve the underlying problem. It improves trust within the code base by helping to ensure that data is used consistently, but it says nothing about data received from the outside world.
But anyone who's used type-based programming knows this is nonsense if you simply write a typed parser (of particular interest to gradually-typed Javascript is packrat-style parsers). Then, your code needs to specify what it wants and what its policy is if it doesn't get it.
But if the author was imagining that compile time integrity and correctness checks could EVER perform runtime checks... well... they're not even wrong they're misunderstanding the function so badly.
The author then proposes out of left-field:
> But it’s like plugging holes in a leaky ship. The problem isn’t just that you can’t trust the types in your system, but that you think you can. You rely on the types to tell you when a change breaks something, but because they were quietly disabled by an any type, or by use of a library, or by a soundness issue, it doesn’t. Types in JavaScript are different from types in most other languages people use: They can’t be trusted in the same way.
Which is both true and useless. Yes, it's true that type circumvention methods exist, and you never really can know if you're calling into a library that circumvents your type constraints unless you read the code. But yes, that's useless because nearly every typed language also has this mis/feature, including OCaml, Haskell, C++17 and even Rust! So the author is not actually giving a real recommendation or even novel insight.
And then we get the final substantive giving us arguments that the author has themselves just discredited:
> Or you change the game and just use PureScript. Or ReasonML, or Elm, or even ClojureScript. These exist today. Production software runs on them. They work with the JavaScript ecosystem, where necessary. And they provide a higher base level of trust in the code that you write and an environment where immutability, functional programming, and types (where applicable) work well and work together.
Firstly, I've written a fair amount of Purescrpt and we call into unsafe code ALL THE TIME for the sake of expediency in Purescript. To even pretend otherwise is just... either it's willful lying or someone who has barely interacted with the Purescript ecosystem in any production fashion.
If you use a dependency and don't vet it somewhat (or at least trust a crowd to vet it as a minimal effort) then you're in for a bad time. Pretending otherwise, or that this is a unique flaw of Javascript, is just wrong.
And if you disagree with that, then I have a challenge for you. I propose to you that you can substitute ANY language and runtime's type and GC as "types" and "assembly" as underlying javascript and not change any underlying axiom presented in this article.
It's not the types that fail, but the type system and its misuse.
> This is like pretending really hard
No, that's not pretending — static typing is literally about theorem proving, type theory being equivalent with math logic. This is the famous Curry-Howard isomorphism, types corresponding to propositions.
Of course, you can have situations in which your type system cannot describe the propositions that you need, or situations in which holes are left for pragmatism. Such holes include for example the ability to use Object and then downcast in Java and have the implicit contract that the people using them know what they are doing, being at the same time a symptom of the type system not being expressive enough.
In TypeScript in particular you can just use the "any" type, which is there for pragmatism and because JavaScript developers would have had a hard time adopting it otherwise. It doesn't help that TypeScript's generics are unsound either.
Of course, you can point at "the outside world" and say that you can't trust it to deliver the types that you expect. But with a good type system, you only need to do that runtime validation only once, at the edge where you receive the data and afterwards the statically typed compiler can take over that responsibility. And indeed, in my experience libraries for JSON parsing in Scala (e.g. Circe) or Haskell (e.g. Aeson) are very well behaved and due to automatic deriving, they make that validation painless in 90% of cases, because the runtime validation is derived from the static types automatically, no work required, something I haven't seen in TypeScript.
The whole notion of "optional types" is my opinion flawed.
You either work with static typing, or with dynamic typing and choosing a side will change the way you work, it will change your mentality, your approach to problem solving.
Working with optional types, which is what people tend to do in languages like TypeScript, brings you the very worst of both worlds.
Optional static typing doesn't work. A contracts system, like Clojure's Spec, works much better for dynamic typing and yes, there are fundamental differences, basically the difference between "for all" and "exists", or between compile time and runtime.
---
> Convention: Pretend immutability
N.B. in an actual FP language, immutability is not a convention, but something being enforced by how data structures are built (e.g. persistent data structures), via the type system or the runtime system — even in Java, you'll have a really hard time to change the definition of a class or of a "final", or to mutate a persistent collection (see vavr.io).
---
I like TypeScript overall, I think it's an improvement over JavaScript and it certainly has pretty cool features — but don't use it to judge either static typing or dynamic typing or functional programming for that matter.
If you want to see actual dynamic typing and how it can shine, use ClojureScript. If you want to see actual static typing and how it can kick ass, use PureScript.
And both are good for giving you a real taste of functional programming. JavaScript and JavaScript++ languages, are really poor at FP, mostly due to the ecosystem's culture, and the very few FP libraries that exist for JavaScript are very unpopular, with only a few gems here and there, like Rx.JS. So if you're doing JavaScript or TypeScript, chances are your codebase doesn't do much FP.
> Or you change the game and just use PureScript. Or ReasonML, or Elm, or even ClojureScript.
> If you want to see actual dynamic typing and how it can shine, use ClojureScript. If you want to see actual static typing and how it can kick ass, use PureScript.
In the context of software development, it means that "prototypes" get shipped as production code all the time.
The killer feature is global type inference. I pretty much prototype in Elm as fast (or faster) than any dynamic language, and when the code compiles it’s free of runtime error except at the boundaries (i.e JS interop). This allows me to focus on fixing error arising from my misundetstanding of the problem (and my general incompentence).
It's worth noting that this is still not quite the same as the leeway that most dynamic languages allow you, because, while you can downcast, all such downcasts are checked. You can't just say that Foo is a Bar, and proceed to treat it as a Bar, in Java and similar languages. You can assert (via a typecast) that it ought to be a Bar - but if it's not, you just get an exception or some other indication of failure.
As a result, it's much, much more difficult to hammer square pegs into round holes in those languages. As it should be.
I think as time goes on type checking will be the least of our issues. Checking for different types of abuse (technical, social, semantic), context sensitive boundaries, setting tripwire-esque traps to control abuse etc will make any current concerns over type checking seem elementary.
We need to give up on the idea of coding 3 line, beautiful functions that look great on Twitter and realize we live in a time where defense and privacy are a priority, while (some how) simultaneously pushing innovation.
I hate it, trust me, but if you talk to typcial corporate/business clients, this is where their thoughts/concerns are currently. I realize it’s a bigger conversation than what I’m mentioning here.
The author doesn't mention the critical need for any nontrivial JavaScript project to build and maintain an automated test suite.
Type systems needs to work statically as well as at runtime via some kind of reflection API. This way you can unmarshal the data that comes over the wire directly into your language's type system. With JavaScript+TypeScript/Flow, the static type system and the dynamic reflection system is split into two separate paradigms, which in my opinion makes things rather complicated to the point of sacrificing developer productivity.
I've seen some of the worst java apps before, but no one says "never use java, it's bad".
It's generally accepted that skilled engineers can make clean, easily maintainable apps, and unskilled engineers make garbage apps.
This just sounds like sloppy engineering and a total lack of professional rigor and in that case, why is javascript to blame?
It's about being consistent with yourself.
If you find yourself writing defensive checks in fulfilled promise handlers because the data you get is sometimes invalid, stop. Get up from your keyboard, go to whoever owns the endpoint, and ask them why their endpoint is returning a 200 OK response with data that doesn't live up to its spec. This is a bug in their system, not yours.
By the way, there is distinction to make between library code that should exception/assert/abort on error and application code that should verify input and display an error to the user.
All of this could have been avoided if those companies had just created a culture, from the beginning, where you force the upstream data provider to fix its issues rather than coding around them by allowing code to fail-fast rather than gracefully.
Now companies are beginning to introduce integration and unit testing across the board and type systems (again, to ensure internal consistency for systems so that other systems can depend on them) and the cruft and defensiveness is slowly going away.
To repeat the old cliché, the best defense is a good offense :-)
Were it such an easy thing...
Here's how I've experienced such things:
I write naive code. Application hard fails. Test environment down - everyone cross with me.
Me: Yo upstream - fix your data please.
...no response.
So I raise a ticket to get data source fixed - send it to the backlog.
Me: Yo delivery - can we get this ticket prioritised, assigned and put into a sprint?
Delivery: Yeah... nah - cause y'know... delivery.
Me: Yo leadership - can we get a decision taken on whether or not upstream should fix their data, or whether we should just program defensively? If the former - can someone tell delivery to tell upstream?
...no response.
At this point - I give up. Write my little defensive check and admonish myself to do so consistently from now on. Commit, push... go home - sleep like babe.
Often the systems you're connecting to aren't maintained by people in the same office or company as you.
Every back end and front end system I've worked on as long as I can remember has had dependencies on third party API's. Getting bugs fixed in those APIs could sometimes take months ("change requests"), if at all.
Finding a good balance for how your system gracefully degrades is a bit of an art form.
The point about 3rd party libraries is taken but I also think if you are using a 3rd-party endpoint that publishes a specification that it then doesn't live up to (and takes months to fix) then it's not a reliable partner in whatever you're doing and is not living up to its own promises about the data its customers can expect. Like one of the other commenters here said, time to deprecate or write a wrapper around it that enforces the data contract.
If you’re getting bad data you need to address the issue higher up. Either through fixing your source or if you’re forced into some badly behaving library then you should at most write a wrapper around it and make that as isolated as possible. Then you should start making plans to deprecate that library.
I'm referring to a general situation. You're referring to a specific situation of http apis sharing what I assume is untyped data and the typing is lost during transport. The solution is to develop protocols on top of http where the type signature isn't lost.
Is there a built in language strategy, in Rust for example, to define a complex numerical type that has common properties like min/max range, exclusive min/max, conditionally required logic etc?
Of course you can put some custom checks/logic to validate, but not sure why people instantly shame json-schema. From a validation perspective (not tooling) typescript is simplistic compared to a reusable, extended json schema.
It may be a matter of time for other js/ts tools to improve. I’ve been learning Rust and I haven’t seen a better solution currently for runtime/user input validation. (Not a c/rust expert by any means)
In other words, languages should not be used as a substitute for discipline, good patterns, or scalable best practices, but rather as an enabler of such conducts.
TypeScript favored pragmatism over purity, but it is a very well thought-out, robust, and expressive typing layer on top of JavaScript.
Sure, for the sake of language purity, developers can "bytecode your way" to JavaScript Dart, Java with GWT, ReasonML, Elm, but there is a reason why all of those options, how elegant they might be, have limited traction and relatively short lifespan (at least compared to JS), and that is because they add too much friction points (often unexpected ones) for the sake of language purity, which at the end is a never-ending quest anyway.
TypeScript is a structural typing system which has its differences with commonly known nominal typing systems (which often confused with soundness), and while there are some unsoundness is some specific cases, overall, TypeScript is a very robust and expressive typing system on top of the most pervasive language, JavaScript. Not perfect, but very robust nevertheless and a huge step up from traditional JS. TypeScript combined Modern JS (module, let/const, async/await/promise, arrow function with related scoping, deconstruction, ...) does create a very robust environment to scale code and developers.
We used to all our backend in Java for 20 years (personally comes from Oracle) and we now see nodejs / typescript ecosystem and runtime environment has competitive for big app system. For one of our client, we actually just ported the last of our multi-service system from java to TypeScript, and it got 40% less code and it is better typed than Java. Java's high typing friction and Class-For-Everything purist approach makes good code design more of a contortionism exercise than an intellectual one, and while nowadays most agree that this OO-Obsessed approach was the wrong one, it was considered very pure and "robust" at a time.
Anyway, the short version is what makes a big code base scale is not the purity of any given part (e.g., language) but how the whole can minimize present and future friction points. The less friction, the more velocity. Discipline, best practices, tools, environments, and ecosystem are all part of a whole and one needs to make sure to not over optimize one part on the detriment of the others.
(long term lurker here)
Err...you're talking about promises from the community, not Promises in code.