Code is Engineering; Types are Science
tweag.io
tweag.io
3.3810 software engineering
1. systematic application of scientific and technological knowledge, methods, and experience to the design, implementation, testing, and documentation of software ...
It's all there just waiting for someone to read it.
But we do get a preview and there is a link to www.computer.org/sevocab so we can get a sense of their taxonomical approach. (Try searching for "Data type").
If you want to go beyond the safe norms, you would have to pay the cost of validating the safety of your design. The analog in programming might be theorem proving that your elegant design was still safe.
Once in a while, civil engineering has a revolution exactly because somebody proves that a class of new designs is safe. The argument is compelling on every field.
We don't really have that for programming, but it's not like we shouldn't try. (Eg we have TLA+ for processes, we can use Coq and others for proving some parts of programs.)
We do in the form of abstract interpretation and various other global static analyses. Unfortunately, these analyses are non-compositional, where small code changes can lead to dramatic and unpredictable long-range effects in the properties of your code. Finite-element analysis has similar non-compositional properties.
A type system is a compositional form of static analysis, and composition obviously has considerable reuse and cost reduction benefits, which is probably why most programming doesn't use static analysis tools beyond type systems.
> [...] most programming doesn't use static analysis tools beyond type systems.
Yes, unfortunately. :( It'd be great to have access to much better/stricter tools to be able to check just some interesting/important parts of programs (eg. pure business logic modules/functions) without resorting to reimplementing them in static-analysis systems.
btw, I think you're missing an important qualification, which is that a given formalism can always be enriched with new axioms to increase the scope of what it's able to prove. This is relevant for type theoretic approaches, which generally have a mechanism for adding new axioms.
Types are not just an engineering technology, they are a QA tool.
(also, at no stage have you provided an example for such "more elegant" statements)
To take a much simpler example, the Simply Typed Lambda Calculus precludes non-terminating programs, but is not generally useful.
For Church-typed languages as opposed to Curry-typed languages, it's not even apparent what it would would mean to "preclude otherwise valid statements" because types are a part of the language. It's akin to saying structured programming precludes otherwise valid statements in the language because the lack of goto precludes otherwise valid statements in the language. Sure, but the lack of goto is precisely a facet of the language! For a Church-typed language the types are just another part of syntax (this is more than just theory, Racket shows that a type system can in fact be implemented simply as macros).
Invoking Godel here I don't think makes sense. Godel's incompleteness theorems apply to what you can prove about a program with types. The majority of typed programming languages have type systems that are too weak for Godel's incompleteness theorems to apply. And there is no need for a type system to be sound (if thought of as a logical system with programs as proofs via the Curry-Howard isomorphism), again most programming languages' type systems are not sound in this manner.
Even with a Curry-style system of semantics, the presence of casting is enough to allow any type system to allow every valid statement in the language (the proof of this is to cast every term to the same type).
That being said I am sympathetic to the feeling that "I'm just doing this to satisfy the type checker," a feeling that can show up increasingly frequently for languages with increasingly sophisticated type systems. I would argue this often happens precisely when you take the power of casting away. This is the dark side of "correct by construction" techniques where if the construction technique is devilishly complex, then you're forced to deal with that complexity. But this usually only applies to languages with some amount of dependent typing, which the majority of languages do not have.
The original comment's "otherwise valid statements" are precisely UB in C++.
There's others where you can try to finesse some sort of Curry-style semantics out of a language that doesn't have those semantics.
The crucial question though is to what end?
The original contention is that types represent a limitation on a language, a limitation that you might want to sometimes lift. To the extent that you can separate the types from a language itself I agree that's true in theory, but that problem is solvable with casts.
And to the extent it isn't solvable with casts, it isn't solvable precisely because the Curry-style interpretation fails and the line between being excluded by a "surface-level" typing rule vs a deeper matter of language semantics is blurred.
All that being said, there is an argument to be made that you sometimes want to preserve a Curry-style fragment of your language precisely in order to facilitate this kind of "escape hatch" with a cast
However, going back to my original point, at least for me, this isn't something I worry about with mainstream languages. It feels more like a theoretical concern than a practical one. If you have a language with a type system that cares a lot about soundness and expressivity then it's something to keep in mind. I worry in dependently-typed languages that without preserving this distinction between types that have runtime-significance and types that do not, you lack an effective cast escape hatch.
For mainstream languages my concerns are exactly the opposite. The type system often is of little use to me because so much code makes use of escape hatches.
It's slightly tweaked to remove some of the parts that don't really make sense without the other comment's context.
You're implicitly assuming a Curry-style semantics to your language. Here's an outline of what I assume you mean. Correct me if I'm wrong.
There are effectively two separate languages. Your type-level language and your term-level language. Each of these languages have their own notions of validity. At the type-level, a series of statements is valid if the type checker accepts them. At the term-level a series of statements is valid if the program runs without crashing. There is also some way of associating the two languages together, i.e. type assignment.
Given these two languages, we would like for their two notions of validity to coincide. That is it would be great if given a type assignment, the type-level language is valid (type checker accepts it) if and only if the term-level language is valid (program does not crash).
The first thing to note is that many (most?) statically typed languages don't have Curry-style semantics. As UncleMeat points out not crashing is not sufficient to be semantically valid in C++. It might just happen not to crash on your machine with a certain set of optimizations and compiler flags. Types play an integral role in what it means for a language to be semantically valid and it doesn't really make sense to consider the language independently of types.
For Curry-style languages, there is an indirect linkage to Godel, but the more obvious link is through the halting problem. If type checking is decidable (something you probably want to be true) then your term-level language must be decidable as well if the two languages truly coincide. But then you lose Turing completeness as your first comment points out.
The thing is though that no mainstream statically typed language tries to ensure this. All of them allow assigning a type to something that might loop forever or crash.
I would argue you're effectively attacking a straw man of a type system.
However, I agree with you that "there are times when a type system is too cumbersome, and we must escape it to do useful things." However, this is not something I view as limited to type systems. I view this as true of every facet of programming (and in a greater sense true of every facet of life).
Every system must have a way of piercing its assumptions when you as its user have extra knowledge the system does not. This shows up in performance and protection against side-channel attacks (assembly intrinsics, FFI, etc.). This shows up in orchestration tools (yes yes yes it's all good and well to treat infrastructure as immutable and machines like cattle not pets but I would sometimes like to just SSH onto a machine and make some one-off changes!). This shows up in structured programming (once in a very blue moon Python, it would be great if you just gave me a goto/label). The list goes on and on.
Computational Type Theory (CTT) blends the distinction b/w terms and types, and in fact C-H correspondence bears no consequence on CTT research.
From a previous comment [1]:
> I worry in dependently-typed languages that without preserving this distinction between types that have runtime-significance (Church-style) and types that do not (Curry-style), you lack an effective cast escape hatch.
I think we've basically two categories of (turing-complete) programming languages:
1. Mainstream languages that use type systems for multiple purposes: operational semantics, compiler optimization, lightweight formal verification, etc, and provide casting escape hatches.
2. Languages where the type system's primary (and only?) purpose is formal verification, and there are no escape hatches.
It's just that 1 and 2 are really meant for different domains. 2 is mostly used for high-assurance domains like aerospace, automotive, healthcare, blockchain smart contracts where formal verification is high priority and escape hatches aren't sought, hence not built-in.
I am not sure that the distinction between Curry and Church semantics is so critical - after all we have but a single 'engine of computation': beta reduction, applied to statements in the untyped lambda calculus. Thus a statement with Church annotations will reduce exactly as one without?
I think I'm taking the view that statements in the untyped lambda calculus, with beta reduction, are our objects of study, and a type system (Church or Curry derived) is a set of theorems (or indeed a mechanism for generating theorems) that we use to prove certain facts about such objects, again much like the Peano axiomatization of natural numbers. Thus I profess my heresy!
The distinction between Curry and Church semantics does not seem particularly relevant for something like the simply typed lambda calculus.
After all
f : Int -> Int
f(x) = x + 1
f(5) : Int // 6
f(x) = x + 1
f(5) // 6
f : a -> b where b is Zeroable
f(x) = zero
// E.g. List[a] is Zeroable where zero: List[a] = emptyList
// Int is Zeroable where zero: Int = 0
// String is Zeroable where zero: String = ""
f(5): List[Boolean] // This is emptyList
f(5): Int // This is 0
f(5): String // This is ""
Now there are ways of trying to recover Curry-style semantics. One way is to replicate the type system at runtime. Then of course you're free to "erase" the types since you still have them at runtime anyways!
This is essentially what statically typed object-oriented languages do with late binding. Note that in the extreme though, the type system cannot limit the scope of valid programs because valid programs at runtime are precisely those valid at type-checking time because the types are replicated! Often though only fragments of the type system will be preserved at runtime so that there is still a discrepancy. For example, usually return type polymorphism (the previous example of f) is disallowed in these systems (see e.g. Java) to avoid having to embed even more of the type system than just the types of individual objects.
Another option is to compile the program down into a language with a simpler type system (or no type system at all!) and then use Curry-style semantics with the simpler type system. However, that feels more like an implementation detail than anything else. Just because I can compile Java to C does not mean that they have the same semantics. Indeed the point of compilation is precisely to translate between two systems with incompatible semantics, otherwise the compilation is trivial.
Stepping back from all this, Curry-style vs Church-style semantics have implications for type checking and type inference. In general things are harder in a Curry-style regimen. Type-checking which is decidable in a Church system often becomes undecidable in an equivalent Curry system. This makes Curry-style semantics burdensome to work with in rich type systems.
Curry-style semantics are of course an ideal fit for gradual typing systems that aim to go from an untyped system and gradually add typing on top of that untyped system.
Beta reduction is not affected by types ascribed to terms. The crucial difference between Church and Curry is that Curry allows each term to be ascribed a set of types - perhaps 0, 1 or many - whereas Church insists on each term having exactly one (often annotated). That is all. Both are systems for proving type correctness of terms, and are no more than that.
I imagine Church never dreamed that his approach of unitary type assignment would lead people to believe that types actually exist, and that programs should do different things when meeting such 'types'.
Yes let's get back to Curry, and build a better world.
Eh... I dunno. Church was around for a long time. He saw a lot of the languages we use now.
That ship for ad hoc polymorphism sailed a long time ago. Almost every widely used language I can think of has ad hoc polymorphism, whether that be through runtime types (late binding a la Python, PHP, Ruby, Common Lisp, Javascript, Java, Clojure, Scala, C#, C++, Smalltalk) or through compile time translation (early binding a la Rust, Haskell, also Scala, ML, also Java, also C++, Idris, Agda, Coq). The only languages I can think of at the moment that doesn't have ad hoc polymorphism are C and Elm.
To put it another way every language that supports some notion of overloading has to do a different thing when meeting a type. And I think you'd have a hard time taking overloading away from programmers.
The tweak is that any safe type system will preclude programs that are otherwise semantically valid. You can have a type system that catches some errors and lets others through, or you can have a type system that catches all errors but also excludes some valid formulas (or even, and most commonly, a type system that catches most errors, also prohibits some valid formulas, but nevertheless still lets other errors through), but you can't have a decidable type system that permits all semantically valid programs and excludes all erroneous programs.
I think it’s a bit like any applied science. There’s a theoretical limit to the efficiency of an internal combustion engine. We grow closer to it, often with more complicated machinery, occasionally with better construction (ie, the manufacturing becomes more complex so the product is less complex). And we balance the consequences (pollution, reliability, cost) against the benefits.
Once in a while we substitute a different model (E.g., Atkinson) with small but important differences in the theoretical limits, by looking at the problem differently.
I think we don't suffer from this because we end up asking questions that are relatively simple. Technically, there are many more uncomputable numbers than computable ones, but we are much more interested in the infinitely smaller set of the computable numbers than the uncomputables. (Furthermore, we are much more interested in rationals than we have any right to be, considering that there are many more real numbers than rationals).
There should be a similar premise for type systems. Perhaps any type system will disallow valid statements, but we can try to make it so that those valid statements are above a certain length, say 100,000. Then, there are still infinitely many valid statements that are illegal, and they may be much more elegant than their counterparts, but they're so long that it is irrelevant for practical purposes.
Now, actually building a type system like that is a whole 'nother beast. I imagine you would give up some notion of completeness for a level of "predisposition", that is, the type system predisposes programmers to making legal statements in the same way that normal number systems(reals, rationals) predisposes mathematicians to making provable statements. A slight nudge towards writing programs in a certain style can go a long way in terms of avoiding "illegal but valid" statements.
It's very different than the 'conventional' engineering that goes into building a competitive race car
it's not to contrast static languages with dynamic languages.
> Starting with the slogan "proofs-as-programs," we now talk about "theories-as-systems."
But if you rewind history, you could also see the parallels to Peter Naur's "Programming as Theory building"
Or more recently: Luciano Floridi's "The Logic of Information: A Theory of Philosophy as Conceptual Design"
Different descriptions of the same thing. You could probably relate most of these ideas all the way back to the Greek classics. Democritus. Aristotle. Socrates. Plato.
Our tools are finally catching up...
If by this you mean that you are wondering how often bugs in Haskell programs make it to production, the answer seems to be "at about the same rate as bugs in C or Java, as far as we can tell". Which, given that Haskell's (default) type system is not much more powerful at expressing constraints than C or Java's, shouldn't be a big surprise.
Haskell's type system is much more flexible, allowing you to express specific types for many cases where in Java or C you would have to resort to Object or void*; but otherwise, Haskell types can not encode much more powerful constraints than C or Java types - you can only express the available operations and the "shape". The bulk of the power comes from enforcing purity, but that is essentially a property of the standard library, not of the type system. For example, you can't express that a type is a Monad in Haskell - you can specify that it supports the Monad operations, but you have to rely on manual checking to see if it respects all of the monad laws.
Note that Idris or Coq are another matter entirely (as are dependent type systems implemented in Haskell extensions).
That seems surprising. What are the bugs that Haskell has but C and Java don't that replace all the null pointer exceptions?
Here are some more info http://blog.ezyang.com/2011/05/space-leak-zoo/
It's been almost a decade since though. Maybe the compiler catches more nowadays.
- perhaps Haskell programs end up being more feature-ful within a given bug budget
- perhaps Haskell induces more logic bugs, because it is more alien to the regular way of thinking
- perhaps the vast majority of NPEs are actually logic bugs, so that preventing them through the type system does not actually significantly reduce the number of bugs
- perhaps the study had some flaws in how it was counting issues
- perhaps some or all of the above are true, in various proportions
I think this is worded deceptively. Bug counts are about the same when measured as bugs per line of code. A line of code in Haskell can do quite a bit more than a line of code in C or Java. I think that's a meaningful difference.
https://m-cacm.acm.org/magazines/2017/10/221326-a-large-scal...
They find some effect of language on defects, but it is small, and there are other factors with much larger impact.
Technology is applied engineering, and engineering is applied science.
The study of types may fall into some field of science. Applying it is engineering.
However, the point of the article was different - the article was claiming that when you are writing the actual code for your program, you are normally using engineering-style reasoning (abduction), whereas when you are defining the hierarchy of types for your system, you are mainly applying scientific-style reasoning (induction).
Personally, I think that the article is a gross over-simplification of every single term - of deduction, abduction, and induction; of maths, science, and engineering; and of what programmers actually do. It is in fact so over-simplified that I don't see any valuable insight into the article at all. It almlst reads like a description of programming from a TV show that is trying to sound brainy.
Why not the other way around ?
Static analysis needn't be compositional, and so a small change could cause dramatic and unpredictable long-range effects elsewhere in your code. Figuring out what changed or what went wrong can be a nightmare.
Writing the hs part was a nightmare at first. Once the project grew and I needed refactoring, js became the main nightmare. Hs simply did its thing which at that point, I already had got used to it. Js couldn't tell me if anything wrong outside of function boundaries.
A smarter js linter may prove me wrong but I believe types serve a long term goal by sacrificing productivity in the beginning.
Ref: https://cs.stackexchange.com/questions/122066/does-the-under...
It is by will alone I set my mind in motion. It is by the brew of arabica that thoughts acquire speed, the teeth acquire stains, stains become a warning. It is by will alone I set my mind in motion.
I don't think the article is saying otherwise, it's just highlighting a similarity between what type systems do, and the sort of reasoning we associate with science.
for example, given:
f(x) = x*0;
for all n in Real Numbers, f(n) = 0
1*0 == 0
2*0 == 0
3*0 == 0
4*0 == 0
Note the isomorphism between science and unit testing; Same bs with none of the statistical rigor.
To do the same in logic you have to make assumptions from axioms and logically derive a proof from axioms.
The previous thing which was called a hypothesis is now called a theorem:
for all n in Real Numbers, f(n) = 0
That being said how does Type theory and code apply to science and logic and engineering?
Type theory is part of logic it is not part of science. Don't conflate the two.
What is engineering? Engineering is creating solutions for problems using either or both Science/Logic. In the realm of software engineering there is very little of both, It's all an art. Logic is rarely employed for verification and when it's done it's usually just type checking; and testing isn't done to a rigorous level that other scientific and engineering disciplines require.
For say the creation of the 787 airliner. The entire plane is built theoretically in software to such a degree that when they actually materialize the theoretical model as a physical prototype it can actually fly. Then after it is built they test it rigorously to a very high degree.
For most of software, none of the above is is ever done. We mostly work like artists, just making shit up (design patterns) and adhoc automated testing.
So in all practicality your coding job has no relation to either science, engineering or logic. You are an artist.
I know technically you apply bits an pieces of science and logic in your programming job but ultimately you use adhoc logic and science for even crossing the street, I'm not going to count programming as a "science" due to this.
Usually equating null-hypothesis significance testing with the scientific method is a sign of cargo cult.
I'm not strictly equating inference or verification to science. I am simply equating this: https://www.wikiwand.com/en/Scientific_method
Testing and hypothesis are the key words. Prediction is the other keyword. Analysis I never covered, but you assumed I used null hypothesis testing.
Tell me what is your opinion of all of this since you think my opinion is a 'cargo cult.'