Replacing Python
roscidus.com
roscidus.com
[1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/
ATS makes everything more difficult than it's worth except for some very specific use cases. Unlike some of the other languages like Haskell and OCaml, I don't think this is just because it's different; rather, it's because ATS is simply so much more demanding. It gives you extremely solid static guarantees and high performance, but it's a tool that really sacrifices expressiveness and programmability to get there. Haskell and OCaml may be difficult to learn, but ATS is actually difficult to use--a very different concept.
Go, on the other hand, is neither difficult to learn nor difficult to use. But, more generally, Go has very little to commend itself and--compared to the other options--quite a few shortcomings. If you're willing to put in a little bit of effort to learn something new, there are a whole bunch of options which are simply better. And you should be willing to put in the effort: your programming language is your single most important tool; it affects not only how you write your code and how you maintain it but even how you think. So it seems extremely shortsighted to choose a language because it's easy to learn and similar to what you already know!
Rust is awesome, but it's simply not ready yet. This is widely acknowledged by everyone in the project, and is part of their policy of open development. Once it is ready, it will be a very good choice for certain domains.
I have been writing large, server side software for many years. These are long-running applications that require superb performance as well as first-class monitoring tools. So, for me, one requirement for a language is that it runs on the JVM. The JVM gives you superb performance as well as unparalleled monitoring tools (made even better by the inclusion of the flight-recorder and Java Mission Control in the latest version of the JVM); you also get a vast ecosystem with a huge selection of very high quality libraries (and some other important features like dynamic linking) – all that before you even choose a particular language. You can also buy commercial support for every single component you're using if you need it.
On the other hand, if you're writing a command-line tool that needs a very fast startup time, or if your RAM is very constrained, then the JVM might be a bad choice for you, which would rule out the JVM languages no matter how good they are.
My point is that there are more important issues to consider than the mere merits of the language itself.
Also, you're putting a lot of emphasis on expressivity, while downplaying the importance of a short learning curve. I think the two are of similar importance. It's been my experience that programs written in some of the more expressive languages are actually harder to maintain than code written in the easy-to-learn ones. The reason seems to be that programmers use the expressivity of the former (which seems to be of a particular nature or natures: functional construct, higher-order types, meta-programming) to model their own thought-process, which may be hard to replicate for someone else maintaining the code. The easier-to-learn languages usually work at a lower level of abstraction, which can sometimes be easier for others to follow because its a well-understood common denominator.
Now, I'm not saying that worse is better, or that more "primitive" languages are better, or even that one should pick them over more expressive ones. I'm just saying that, especially when working in a large team or writing code that would need to be maintained for a long time, there are other issues to consider.
I agree with that. Working on code in a large and ever changing team is very difficult, and it seems to benefit from less abstraction, even if it leads to more verbose code. That's the whole secret of Java. I've been implementing a small part of my current project in Go and it has similar benefits.
That said, when working alone or with one or two others, I feel kind of dumb repeating some things over and over at the same level of abstraction. There are lots of things like this little piece of C++ code that I tried to translate into reasonably efficient Go:
template<typename V, typename R>
R defaulted(const V& val, const R& default_val) {
return val == V() ? default_val : val;
}
While Java and its variants' appeal at most large organizations is considered by some a sign of their conservatism, Google is anything but when it comes to picking the best tools. Google's fondness of Java is evidence of the merits of the Java philosophy when it comes to maintaining large codebases by large teams.
Creating a language that follows this philosophy while still boosting productivity (and maybe providing other benefits as well) is an interesting challenge. I think Go falls short. Kotlin looks interesting.
Most of the devs who were hired at Google from 1999 - 2002 still prefer C++, and products built in that era (Search and much of the infrastructure) are still in C++. In general rewrites from C++ -> Java have not gone well; I know at least one such frontend that was rewritten back in C++ a year later.
The part about C++ was to describe a similar scene. When they had mostly C++ devs, their language of choice was C++. and now they have mostly java devs, so java it is!
template<typename V, typename R>
R defaulted(const V& val, const R& default_val) {
const static V v0;
return val == v0 ? default_val : val;
}So it's basically defensive programming, but I agree with you that it's probably too defensive nowadays.
Also, I don't think Google's style guide is the be-all-and-end-all of good C++ style. Virtually non of Boost's code would comply with it. "We do not use C++ exceptions" indeed!
Also, there are plenty of nice modules in boost that just don't belong in the standard library. ASIO is quite nice, but it has io_service implementations that are platform dependent. Statechart is pretty good, but it isn't the only state machine implementation in Boost and neither is clearly better than the other. Spirit has it's uses, but does it belong in the standard library?
The important part is how easy that model is to communicate to the other programmers on your team, because that's what source code is. And that's going to depend upon their backgrounds. Hire a bunch of hackers that are all familiar with idiomatic Python, and Python is a great tool for communicating. Hire folks who like things spelled out in more detail, and Java is a great tool for communicating. Hire people who all think at the level of the machine and C is a great tool for communicating.
I do think that one of the major benefits of choosing a language is that you pick the people who will be willing to work with you. If you pick Haskell you will end up with a lot of folks with mathematical backgrounds and an interest in programming languages for programming languages' sake, which may or may not be a good thing depending upon problem domain. If you pick Python you will end up with a lot of folks that have command of large toolbox of scripting functions, although that toolbox may not perform great when adapted outside its original domain. If you pick Java you will end up with a lot of folks that don't want to think too hard about their language.
Of possible interest: http://www.nhplace.com/kent/PS/Lambda.html
EDIT: might it be possible to develop ATS programs that are easily called from other languages? ATS seems to have strong ties to C, so maybe one could call ATS code like one calls C code, or call C code that is a thin wrapper around the ATS code? Then you hopefully could have efficient, statically verified (to a high degree) code that can be use by other languages.
I can't help thinking that using a better tool like ATS would actually increase productivity without compensating safety. If anything, the types would provide an extra sanity check! Of course, just using ATS won't be enough--most of the existing checks and reviews will still be necessary, but I do think it would be some improvement.
Languages, as means to create programs that express some sort of logic, must get out of your way.. So i dont think its smart to pick a language because its hard, unless its for create new sorts of mind games and puzzles, and not by the sake of the final objective that is coding itself.. to create programs
But the benefits are that many new people will be able to pick it up quickly and easily. Other people's code will be easy to understand and maintain. So it's good for everyone in the long run. Being able to express complicated ideas in simple ways is very valuable.
If you think it's an advantage that it'd take someone else years of learning a language before they can understand the code you wrote, Go is not for you. [1]
[1] http://commandcenter.blogspot.com.au/2012/06/less-is-exponen...
If you're not working with the lowest common denominator, Go is not for you.
However, it may still be quite good to move developers not doing kernel/drivers/embedded stuff, away from C into a more secure language.
* duck typing * goroutines * channels
And some smaller but still important differences:
* Very light-weight Public/private distinction * Slices * Objects that embed directly in other objects, not as references * Return-value error handling; only a very restricted form of exceptions
java.util.concurrent offers tasks and queues.
As for the rest, they are not unique to Go, better languages offer similar features.
Yeah, only that you wont do this sort of things in a memory managed language like Go.
They make you think is low level(PR ?), only that it isnt THAT low level ;)
Go should be able to switch off the GC, and allow people do to manual memory management if it really want to be that low-level..
Also, a way to take off the runtime, and work as skinny as any C program could..
Only than, you could make real low-level programming like OS, device drivers and the like..
The use cases for Go right now in its current incarnation, are things like: cloud/distributed stuff(against C++), or command line tools (against Python)
To make it a contender against C, there's a long road.. and im sure its one route they dont want to go..
Oberon family, Modula-3 are two examples of languages with OS that were used in real life situations, although in academic context, for several years.
Native Oberon System 3 and AOS were quite powerful desktop OS for the 90's.
Although I must concede their unsafe package is more powerful than Go's unsafe package.
Anyway, nowadays I personally prefer Rust or D nowadays.
Having used Native Oberon in the 90's, I became convinced that system programming languages with GC are possible.
Latest versions of AOS even had video players with good framerate written in Active Oberon, with some Assembly snippets.
However, in the Oberon family, besides NEW, the other implicit memory allocations are only triggered by string manipulation and starting tasks, if I recall correctly.
Don't get me wrong, I do 90% of my coding in a garbage-collected language. But any mandatory GC does limit your domain.
However such systems exist.
Kernels => Native Oberon, Blue Bottle, EthOS. All desktop oriented systems used at ETHZ in Switzerland during the 90's. Used for teaching OS design, and a few teachers even used them as their main system.
High end games => The Witcher for XBox 360. They made use of a GC runtime in the game engine.
> But any mandatory GC does limit your domain.
I tend to think the limit is more religious than practical.
Please note that I defend GC enabled systems programming languages, not necessarily Go for such domains.
I'm just curious, how much time have you spent with a really strongly statically typed language like Haskell or OCaML?
I do find Go to have the "best of both worlds" when it comes to the type system (ignoring all other features of the languages).
As you know, one of the advantages of writing in dynamic languages like Ruby and Python is that you don't have to think much before writing down the first few lines of (working, runnable) code. This makes it really easy to sit down at a keyboard and bang out a prototype, or even a one-off utility script.
With Haskell, I have to spend a lot of time thinking up-front about how to design my types, because the arrangement of the datatypes largely determines the functionality of your program. The upside is that, once you've sketched out what you need your types to look like in Haskell, the entire structure of your application has essentially been stubbed out - you simply need to fill in the gaps. This is really nice when making incredibly polymorphic functions - as has been pointed out, you can implement an entire Haskell library without understanding anything about the logic, operating solely off of the type annotations[0].
The downside is that it takes longer to get those first few lines of (working, runnable) code down. Maybe that's not a big loss, or maybe it is. But Go does provide a lot more type safety than any of C or Java (and certainly a lot more than any dynamic language than Ruby or Python).
I'm a functional programmer at heart, and given my druthers, I'd probably prefer a little more type safety in Go, rather than less. But having written a lot of Go over the last year, I don't find myself commonly running into the sorts of errors that a stronger type system would have prevented.
In short, (IMHO), the biggest additional benefit to strengthening a type system like Go's to resemble Haskell's would be in how it forces you to change the code-formulating process, not in how it prevents more errors. Unfortunately, this is also the biggest drawback to strengthening the type system - it means one can no longer use the same programming workflow that one uses for Python, Ruby, Java, etc.
I personally tend to see it as the worst of both worlds. I have to deal with types _and_ I don't really get strong guarantees. I don't run into problems in Ruby that Go's type checker could help with. But that's also my personality: I like extremes.
In Haskell, the type system ensures that side effects only happen in one place: things inside a monad of some kind. For example, if I'm given a function:
foo :: Int -> Int
Haskell and Rust both don't have the concept of null. This is fantastic, as even the inventor of null thinks it's a bad idea. It's borderline irresponsible to write a new programming language with nulls today. So how do they handle a computation that may fail? Higher order types:
use std::option::Option;
fn call_me_maybe(x: int) -> Option<int> {
if(x > 5) {
Some(x)
} else {
None
}
}
fn main() {
let i = 6;
match call_me_maybe(i) {
Some(x) => println!("Yes! {:i}", x),
None => println!("nope"),
}
}
rust.rs:13:8: 15:9 error: non-exhaustive patterns: None not covered
rust.rs:13 match call_me_maybe(i) {
rust.rs:14 Some(x) => println!("Yes! {:i}", x),
rust.rs:15 }
Finally, in Rust, data is immutable by default:
let i = 6;
i = i + 5;
rust.rs:3:8: 3:9 error: re-assignment of immutable variable `i`
rust.rs:3 i = i + 5;
^
rust.rs:2:12: 2:13 note: prior assignment occurs here
rust.rs:2 let i = 6;
fn main() {
let i: ~int = ~6; // read: i is an owned pointer to an int.
let j: @int = @6; // read: j is an managed pointer to an int.
let k = i;
let l = j;
println(i.to_str()); // error: use of moved value: `i`
println(j.to_str()); // this is fine
}
fn main() {
let i: ~int = ~6; // read: i is an owned pointer to an int.
let j: @int = @6; // read: j is an managed pointer to an int.
do spawn {
println(i.to_str()); // fine, prints
}
println(i.to_str()); // you can't use it after you've given it away, either!
// error: use of moved value: `i`
// println(i.to_str()); //
// ^
//note: `i` moved into closure environment here because it has type `~fn:Send()`, which is non-copyable (perhaps you meant to use clone()?)
//do spawn {
// println(i.to_str()); // fine, prints
//}
do spawn {
println(j.to_str()); // error: cannot capture variable of
// type `@int`, which does not fulfill
// `Send`, in a bounded closure
}
}
Anyway, hopefully that explains some of my beef with Go's types. I think Go gets a lot of things right, I think the type system is one place where it gets things really wrong.
By what standard of measure? I personally appreciate the simplicity that Go brings to the table, usually at the cost of features we've come to expect in other programming language environments. Do I miss those features? Sure. But do I like simplicity and straight-forwardness? Definitely. Do I believe there is a way to combine them? I'm not sure. Rust is probably the closest contender. But certainly, languages like Haskell or Java don't do it for me (as frequently as Go does, anyway).
What I'm trying to say is that I won't tell you which side of the trade-off you should take in every instance, and maybe you shouldn't either.
It's true that Python can leave you with hidden crash bugs in unusual code paths, but the language brings so many benefits that it's easy to forgive this. And there seemed to be a vague concern about performance at the beginning of the article, but I didn't see anything substantive in that regard.
So yay, let's all just keep using Python. Back to work.
And since when are we willing to settle for good enough?
[1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/
(In fact, the author writes that he's basically willing to move to OCaml in the conclusion of this post.)
Here are some of the motivating factors identified by the author at the outset of the search for a better language:
- Canonical’s Colin Watson is worried about Python’s performance on mobile phones.
- Marco Jez and Dave Abrahams proposed a C++ version.
- Bastian Eicher would like a .NET version (though IronPython might work here).
It doesn't seem like OCaml effectively addresses the above very well. On the other hand, here are some items the author liked about Python which may now be lost:
- Widely known and easy to learn.
- A large standard library.
- You only need to ship source code (interpreted).
- Can run inside a Java or .NET VM (using Jython/IronPython).
- All current 0install contributors know it.
- The current code is all Python and is well-tested.
Of course the last item is the most alarming: see http://www.joelonsoftware.com/articles/fog0000000069.html (2000), Brooks' Mythical Man Month (1975) for the Second System Effect, etc.
This certainly holds for OCaml as well. The main OCaml company I'm familiar with is Jane Street, and they certainly have no issues finding OCaml programmers. More importantly, many of their programmers come in with no OCaml experience or even no functional programming experience, and yet they have no trouble getting up to speed. In fact, these days they even teach their traders--often complete non-programmers--OCaml. And it works. I don't think I'd ever want to hire somebody incapable of learning a language like OCaml in a reasonable time frame.
So yeah, this is not an issue in the least. If anything, it seems to work out more favorably for the companies using OCaml, Haskel and the like!
Ever since we decided to get shit done, ship, finish our weekend project, etc. Ever since we first read "premature optimization is the root of all evil" and understood it as wide, abstract truism.
C has become the lingua franca of computing. You can call everything (except Go) from C and you can call C from everything. So you want to actually reuse standard libraries, you will want to know C. And if you're a good C hacker you'll know C++. I'm not saying you'll necessarily use it, but you'll know it.
The vast majority of python programmers do not know how to call C/C++ code and are therefore limited to pure python libraries exclusively. They usually use writing to files and os.system() as an alternative. Needless to say, the performance of this approach can be questioned.
* Experience with Python, and general experience in duck typed environments.
* Discipline. Not just your own, but anyone else working on your project.
There are certain mistakes that will make your life miserable down the road. These could be bad organizational conventions, inconsistent exception raising/handling policies, lacking a strategy for documentation, etc. If you make mistakes like this, you probably won't get bitten by it until later.
Writing good Python can be harder than writing good <X Language with more rigidity>. With experience and time, this gets a lot easier.
I appreciate all the niceties that dynamic-typing language like Ruby provides. But I am not sure the trade-offs are enough (all you already described).
Likewise, asserting that "For storing general records, Python provides a choice of classes and tuples" completely ignores one of Python's fastest and most powerful data types: dictionaries.
Lastly, in get_value, the nest of ifs is unnecessary and makes the code seem more complex than it really is.
"As before, note that I’m a beginner in these languages."
"I’m not an expert in these languages (except Python)."
In this case the author wants to parse "program <foo | bar | baz> [optional args]" and does that in a few lines of code. You seriously think that's a "big red flag" ?
Do you also believe the author does not (despite having created the zero install tool 10 years ago) know Python has dictionaries?
This is a blog post where the author evaluates a number of languages/environment to replace his particular Python needs -- do you think that him not using a new standard library module invalidates his finding?
I'm not the one who first mentioned that hand-written option parsing is a red flag. But it is one--especially in Python, which has nice things built in.
It wasn't stated in the post, but the next step after writing this code was to use it as a front-end to the real Python version. If invoked as "0install run NAME ARGS..." exactly then we handle it (the fast path), otherwise we fall back to the Python version. In particular, that means that the Haskell/OCaml version must NOT try to handle --help, etc. I wrote the comparison Python code to be similar to the other languages.
This OCaml front-end appeared in 0install 2.3. For 0install 2.4 there is a full option parser written in OCaml. I didn't use a library for this because a) it needs to be 100% compatible with the Python parsing and b) it needs to handle tab-completion too.
What a class buys you is behavior and a semi-defined interface.
[1] - http://stackoverflow.com/questions/472000/python-slots
Though I'm not sure `__slots__` actually saves any time, as it's really a space optimization. I've tried it before for hot code and even seen very slight (probably not significant) slowdowns.
let {cache; config} = b in
print_endline (String.concat "," cache);
print_endline (String.concat "," config)
;;
print '%s,' % b.cache
print '%s,' % b.config
print ',\n'.join((b.config, b.cache, b.data))
print ',\n'.join(b)
Other than misuse of term "pattern matching", that statement is true and is trivially overcome with two line function, one line lambda, or once and for all by subclassing NamedTuple. Here's the function variant:
def GimmieThing(keyword args in any order)
return ThingNamedTuple(args in correct order)And as for a language / call semantic look at Erlang for what actual pattern matching looks like.
These sort of statement always bothers me when I encounter it. If you're not testing the code you are refactoring how do you know it still works or even worked in the first place?
Static typing gives you useful things (with a trade off), not having to write tests isn't one of them.
Static typing gives you useful things (with a trade off), not having to write tests isn't one of them.
Your point about the value of unit testing in general is well-taken, however.
Static types do not mean you don't have to write unit tests. But they do mean you can write fewer. (And, looking at libraries like QuickCheck, a static type system can make writing tests easier.)
Quite a bit of the safety you get in Haskell especially comes down to controlling effects. Mutable state implicitly couples all the code in a given scope--unless you've read and understood all of it, distinct parts might have effects on each other that you're unaware of. In Haskell, on the other hand, this is impossible, so refactoring like extracting variables and reordering your code can be entirely safe even if you haven't looked at exactly what the code does. The refactoring actions are guaranteed not to change the semantics of the code by the language itself.
The best way to think about it is that refactoring becomes a purely syntactic action. You just remember a few rules akin to algebra, and you can rewrite Haskell code in a bunch of different ways all preserving the original meaning. Regardless of what that meaning really is. This also extends to library code--most good libraries come with algebraic laws, which ensure you can rewrite their code in logical ways. Once you learn these laws, you can start doing things like changing multiple passes over a datastructure into one with the same confidence.
If you make a change that could break things, the type system will often help you find every place that does break. This extends beyond just type mismatches: Haskell also ensures that you consider every possible case in your functions; if you forget an option, it will give you a warning. This means it's safe to add a new alternative to an existing type: you will get a warning everywhere you haven't considered this new option.
I've found this to have a profound effect on how I program. With Haskell, I actually follow the rule of making any code I visit look better than before, simply because refactoring has very little mental overhead. I can move things around liberally, break them into multiple modules, condense them into fewer functions and even change the types, knowing that any mistakes I make will be caught by the type system.
Well that claim is False. I and thousands of other Python developers disprove it daily. Just because they can't refactor dynamic languages doesn't me we can't. Something not-dynamic is probably the language for them. And Python is the language for us.
In other words there is far more variability between developers than there is between languages. Find the language(s) that work for you and quit believing they are the languages for everyone.
There are other ways, different than testing, to convince yourself and others that a program does what it is meant to do. They complement testing. You use them already while constructing the deterministic parts of your program: much of what the machine does for you is predictable, and good languages and libraries are designed to make it so.
In fact, tests won't tell you that the program works, only that it doesn't fail the test cases. You then use the predictable aspects of your domain to convince yourself that the program works if it doesn't fail those test cases.
In the same way that programmatically renaming a variable does not usually warrant the writing of a test case, many forms of automatic refactoring are theoretically guaranteed to not break your program. If they involve type renames, you might be able to deduce that any resulting errors will be caught by the type checker.
(Please don't take this as an argument against testing, but against always requiring test coverage)
That's not the case?
You just add typing to your python and you're good to go?
It makes me wonder if we're closing in on the day when Cython gets rolled into Python proper, and you can just "import static-types" to activate optional typing, then add some static typing to your code, and you would get Java performance from your Python code.
Nah, I doubt it would ever use _that_ much memory.
If you're calling something thousands of times, fast process startup times are good. This isn't the model windows uses though which is the primary target of c#.
I certainly notice any time I boot something up that requires the JVM, I refuse to use the CLR, so I can't tell you much about my own experience with that.
As for page load times, this is silly. There is no startup time on a page at all. Possibly the first hit but there are warm start options for that in CLR at least which make this a complete non issue.
To give you an idea, 98% of our page hits are under 80ms processing time and we have big, heavy pages (we're old school asp.net mostly).
I'm talking about command-line applications, like the one in the article and in your first post. Then you get the startup every single time you run the command.
I use zeroinstall a lot (I'm a contributor), and I turned off tab completion because the lag (of the python implementation) made me wonder if my terminal had locked up, which was far more distracting than trying to remember the available arguments. I have re-enabled it in the ocaml port, because now it's effectively instant.
This is only the case if the code is JITed.
With ngen and mono-aot it can be compiled to native code directly.
> JVM-based languages would not fare any better.
It is all a matter of which JVM is used. Some of them offer AOT compilation and on disk cache of JITted code from previous runs, thus matching the startup time of native binaries.
That said it does not do well for desktop apps due to slow startup, high memory use and lack of native toolkit. It does much better as a server side runtime.
The JVM isn't easy to beat for software that's more complicated than "Hello World". Sure you can beat it, but you're going to have to work very, very hard.
The security flaws exist in the area few people care about (and shouldn't even really be installed any more) -- the sandboxing and web start code.
Did you include gcc and friends in your "too big" calculations? Hard to program without them.
Not such a big fan of Java, but Scala seems to be a strong contender for Java.Next* , and is certainly a viable dynamic-to-static transition language given its terse syntax, deliciously rich collections library, and implicits support (for the MOP fans).
* Twitter being able to withstand spikes of 140K+ tweets per second without lagtime is impressive to say the least.
Really, this article is about "I want to rewrite my program in the newest, coolest language", not about which is the best tool for the job. And that's fine, but the author should present it that way.
I'm sure you meant "newest and/or coolest", or "recently become coolest", but it's worth noting that Haskell is about as old as Python, and OCaml is not much younger.
It's the standard library and such that are moving fast still.
I always joke around saying that if your Haskell code doesn't have typeclasses and monads, you're doing something wrong. By which I mean that unlike other languages, there's a huge, huge, huge gap between writing great haskell, and regular haskell, with a very steep learning curve.
I think it is a community issue actually. Not the language. It is just not converging. Just think of how many distinct OCaml code styles you have seen. Some use hardcore math notation, some not; some use recursions everywhere; some use imperative syntax; some use pseudo-OOP approach. Some even try to fork the language syntax. And a result. Well. Even with the language itself providing easy and concise syntax, ocaml code is not very readable. And not very popular.
http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r...