Dynamic Languages are Unmaintainable
williamedwardscoder.tumblr.com
williamedwardscoder.tumblr.com
If you're going to write a physics simulation, you're really going to benefit from some language features over others. I would argue that you especially benefit from static types because you control a lot of your stack and type checking can make your code very robust. On the other hand, if you're going to write a webapp, think about what you're doing: essentially slinging text around and taking random inpust from users, casting all of it to actual types, and shoving it into a data store (casting it again sometimes). What is the type system providing you in that scenario? Everything is cast at runtime somewhere.
Though the data flowing in and out of a web application is usually in string form, that doesn't mean that it needs to be treated as such in your application. You'll convert the string to a richer, more specific datatype and the static type system will help you correctly manipulate the data.
A nice blog post on the subject is [1] where the author explains how you can represent different kinds of strings (raw strings, SQL strings, etc.) using the type system.
[1] http://blog.moertel.com/posts/2006-10-18-a-type-based-soluti...
I'm not arguing one way or another that static typing is/is not good in this scenario. I'm just pointing out that the foundation of the system as a whole is not as solid as other programming scenarios where you have much more control.
1. Ability to minimize the amount of code you write. Code you don't write is code without bugs. this means either writing problem specific libraries or having them in the system (Matlab creates more robust code than writing it all by yourself)
2. ability to minimize boilerplate. This is where dynamic languages often win.
3. Ability to change code easily and test or run frequently (not necessarily unit testing). Rewriting usually makes code more robust. Time and money always runs out. Always.
The only constant law for selecting programming language for numeric programming is: IF YOU DON'T HAVE TO SPEND SIGNIFICANT TIME OPTIMIZING YOUR CODE AFTERWARDS, YOU ARE USING TOO LOW LEVEL LANGUAGE. If you don't have to optimize for speed, it means that you have been micro-optimizing your code everywhere without even noticing it. This is why C, C++ or Fortran should never be the main programming language you use. Use Python, Common Lisp, Matlab, Haskell, whatever and when you hit performance bottleneck, you write minimal amount of C or C++ code to get over it.
The lack of static analysis is extremely irritating when you've been running a long old calculation for 20 minutes and it chokes on a misspelled variable name. You can often catch these with PyDev/eclipse, but not always.
I try to rewrite any code which isn't fast enough in C (or C++, using a few of its "features" as possible) and wrap it up as a Python module; this gives me the best of both worlds: the speed and type checking of C/C++, with the convenience of calling the code from a quick script or an IPython shell.
This approach is much easier for scientific code than for web apps, though, which do actually need dicts and string parsing.
[[One web app option might be to cython as much of your pure python code as possible (including cdef'ing many of your local variables) as this will dramatically help typo checking - I don't know if this is particularly scalable though.]]
If you are writing C++, you can't hot edit and continue without re-compilation. With a scripting language and proper flow control, you can edit upon exception and resume.
But with recompilation you can. watcom allowed stepping back during debugging as long as the operation was possible to revert. From there patching up the code to include a new version of the function shouldn't be that hard. Not sure if anyone's actually done it, but it's certainly not impossible.
Ksplice works similar way if I'm not mistaken to do reboot-less kernel upgrades. (patching the functions part that is, not recovering the code that's already running them)
What happens if your binary is faulty?
This is probably because I don't have the setup required to persist the changes back down to the source file, even when that source file is different from the one actually loaded (eg in my development folder rather than the installed site-packages version.)
If the app starts getting too large, then by all means rewrite it in a statically typed language. If you're getting to that point, a rewrite is probably warranted anyways; regardless of the language you choose, first versions are usually awkwardly structured and difficult to maintain anyways since features evolve faster than architecture.
All you need is a sophisticated type-system and smart type-inference that works everywhere (not just locally) and you'll be able to be just as productive as with today's popular dynamic languages but get all the benefits of static verification.
Easier said than done and sometimes not an option, but it's a strategy that can work with web app types stuff. The buzzword here's probably SOA.
People have written 500kloc C++ systems and they've written 500kloc of TCL.
So I find the post "debatable"...
Of course, sweeping generalizations about static versus dynamic languages tend to fall flat.
> If you misspell a variable [...], you discover this at run time. Ouch.
CL-USER> (defun square (n) (* nn n)) ;; Ouch! nn isn't defined!
; in: DEFUN SQUARE ; (* NN N) ; ; caught WARNING: ; undefined variable: NN ; ; compilation unit finished ; Undefined variable: ; NN ; caught 1 WARNING condition SQUARE
$ perl -Mstrict -E 'say $foo'
Global symbol "$foo" requires explicit package name at -e line 1.
$ cat err.py
def a():
print b
$ pylint -E err.py
************* Module err
E: 2,8:a: Undefined variable 'b'Technically speaking, the more reflection a language provides, the less correct this becomes. Python code can dynamically inject local variables into stack frames, attributes into objects and so on in more or less silly and probably non-portable ways. So in the end, you end up with pragmatic correctness for mostly sane code. If that's enough for you and your project, go with it.
Well, I think he has an interesting data-point, and makes some interesting points despite maybe making a somewhat incorrect generalization.
But even that generalization points to something I find interesting, which is that Python and JavaScript (maybe Ruby as well) possibly form a separate group with the characteristics he describes.
I also found the observation that Mock object testing doesn't seem to respond well to changes in the underlying code interesting.
It's that mix that makes it "debatable" ;-)
rspec-fire for example validates that the object you're mocking responds to the method name you've used, and takes the same number of arguments. It also does it conditionally based on whether the class in question has been loaded, so your unit tests can run in isolation, but when you run your full test suite it will catch any mocking errors.
Pretty handy!
Well I thought refactoring went all the way back to Forth.
I'm sure that with some language xyz it worked out very well for some project abc and that obviously disproves my whole point and you can move on...
I found writing large projects in Python - a language I was very familiar with, expert even - to become very difficult to extend aka maintain after a long time. My post should be seen in the bigger picture of the post I was following up about "Scaling My Server", and others I have written.
The original refactoring browser was in Smalltalk. Forth programmers don't go about that type of thing the same way as other language since the goal is really to write good "words" that express your program. It is more building a better language than traditional refactoring.
And the final recommendation for Go seems a bit odd, as I wouldn't really cite that as a language with a thorough, bullet-proof type system (not saying that it isn't worth recommending, but given the context of the post...)
This is what I hate about the unit-test everything approach. There seems to be a class of developers recently that believe robustness is more important than anything else.
Often speed bumps in changing API contracts is a higher price to pay [0] than the kind of bugs which unit testing uncovers.
There are lots of things which are good in small quantities and very bad in large quantities [1].
Better to be able to iterate quickly, spot a few bugs with sparse unit tests and completely rewrite your small components if they no longer fit the bill. There is a tendency (perhaps with those too familiar with large corporates) to try to de-risk with more and more robustness tests. With all of this robustness monolithic software flourishes, and you eventually lose the ability to iterate quickly; the ability to destroy and start anew.
[0] Form does not follow function, form is interdependent with function. Make the functionality robust and you make the form rigid! How will you find product-market fit when you are unable to move?!
It all depends on your problem domain--the cost of failure for certain things (pacemakers, telco software, car engine ECU code, and so on) may be much, much greater than the cost of being super retentive in your development.
Infrastructure is a different kettle of fish, while it might sometimes be a life-or-death matter, generally it's just safer to seek pure robustness here: monopolies with government connections do not need to fear losing competitiveness since they can (a) manipulate the rules of the market, and (b) bind market players with contracts.
I don't like your insinuation that I'm just talking about cool hockey-stick startups; in the domains in which non life-or-death, free-market business and software colide, losing competitiveness/product-market fit is the central failure mode. And you absolutely cannot become robust against the market. You must either (1) become faster than the market, or you must (2) control the market.
In this case, it is about shifting the costs to different stages of the projects lifecycle. Statically typed languages require more mental effort while developing, and, by definition, catch more bugs early in the process (compiler has more information to play with, and the programmer is required to be more thorough). Pushing it even further, formal proof systems require exponentially more effort at development time, and, in some cases, rewards it with the proof of software correctness (i.e. no bug-fixing costs at maintenance stage). On the other end of spectrum, dynamically typed languages allow for much faster development (which can be mission critical), but a rapid release comes with a price of long-term maintenance difficulties.
It's up to the developer to choose which approach is the most appropriate for the situation (amongst the myriad of other trade-offs, like performance, scalability, user requirements, existing infrastructure and so on).
Whether or not that makes your code unmaintainable is up to you, it helps if you keep your classes small and give each of them a well-defined and documented purpose. But that's just good coding practice in general.
Also, static code analysis is nice, but it can't determine everything about your code; Turing proved that in 1936.
Meanwhile, by skipping the the compiler and linker, you can deploy enhancements and bug fixes to a running program without shutting it down and restarting it. For a rather long but enjoyable read on why this is important for long-lasting software, see: http://steve-yegge.blogspot.com/2007/01/pinocchio-problem.ht...
If you have this problem you have probably been trained to be sloppy by compilers.
Mind you, this is the sort of thing you usually discover quickly, but it's also the sort of thing that most other languages will catch at compile time.
I've wrote a system in ruby that does clean type checking when I want it to and it cleans up the code and tests a lot.
Just because you're not taking full advantage of your language tools doesn't mean that "dynamic languages are unmaintainable"
http://www.rubyfleebie.com/ruby-is-dynamically-and-strongly-...
I say generally because the Haskell type system (with some extensions) is Turing complete, so you probably code implement a code prover in the type system.
They'll brag about how they have 45,000 or more unit tests, for instance. Yet when we actually look at these tests, many of them just end up implementing a half-baked type system.
Even if they don't realize it, or don't want to acknowledge it, users of dynamic languages do want strong, static typing. It is one of the most useful tools there is when it comes to helping detect or avoid various problems. Rather than doing it in a sensible matter, by letting the computer (via a compiler, for example) efficiently do the hard or tedious work, they just choose to do it themselves manually.
This program is arguably incorrect, even though the types match up and there is no mutable state.
fun square {m:int} (a: int m): int (m*m) = a + 2
The difference between writing proofs and writing tests is subtle though and serve similar purposes. Tests can make you feel confident that things are right but proofs should make you certain of it.
[1] http://bluishcoder.co.nz/2010/09/01/dependent-types-in-ats.h... [2] http://bluishcoder.co.nz/2013/07/01/constructing-proofs-with... [3] http://bluishcoder.co.nz/2012/10/04/implementing-a-stack-wit...
> the type system of the program proves the program correct.
Is not a valid statement, unless you're actually writing proofs in Coq. I bet you aren't.
"The type system of the program proves many simple properties of the program correct" is much more correct.
Also:
> Because my code has no mutable state
No, that's not why. The type system you use may be incapable of reasoning about mutable state, which coincidentally serves to indicate that it most likely can't reason about such things as "this list is sorted", further supporting my initial counterpoint.
Check out ATS or Why for examples of languages that permit reasoning about mutable state.
>> don't have to write any tests at all
>> maintenance is easy.
You're zero for two, my friend. :(
Why should this be? Plenty of companies have teams of functional programmers.
No reason.
Static typing is right for some problems, but the idea that dynamic languages are inherently unmaintainable in some way that most static languages aren't is one that I find incorrect.
Example: Runtime wouldn't be able to give you a protocol between how two classes would communicate. Say your service object that handles authentication. You would have to write unit tests so the two classes involved would know how to respond to each other.