Why Arc is bad for exploratory programming
arcfn.com
arcfn.com
Programmers are not welders. Programming means writing new code. Language is the medium of that code, it is hugely important, and it gets more important as the complexity of one's problem increases. The only way that libraries would become more important than language is if programmers stopped working on complex problems or new problems.
Ultimately, this overemphasis on libraries is a variation of the discredited "code reuse" meme, in which programs are imagined to be strung out of components, so all you need is enough components.
Edit: Another common thing about these discussions is otherwise intelligent people ignoring the fact that it is possible for a program written in one language to call a library written in another.
Not that it's Arc's fault; Arc just isn't at the point in its lifecycle where the libraries are at Python level. It's more like Python 0.5. It's a good goal to have though.
You have to factor in the cost of finding the library, evaluating it (possibly rejecting it), reading its documentation (if it exists and is accurate and up-to-date), and living up with most of its warts because you can't be bothered to learn the innards of the library to make those few changes that would simplify your work. Or you could contact the maintainers and hope they will maybe honor your feature request/bugfix at some point. Libraries do have overhead.
edit: In the specific case of HTML parsing libraries those caveats probably don't really apply because it's a well-known problem with a well-known solution. But in the general case those are valid considerations.
Also from building a first draft (could take less than a day) to a professional solution you have to pass two big hurdles: feature set and performance. Each of them done properly can take easily more then a week.
I do like the idea of using the libraries of different platforms. Wrappers are much easy to write, and a mechanism for using java classes from arc/lisp would solve a world of problems. Unfortunately I can't imagine it being done without a big performance hit.
There are many library functions that would take a day to write. If you need one, and you are missing it, then you will spend a day when you could spend 10 seconds. It's that simple.
And this is a huge differentiator among languages.
Scala and Clojure have immediate access to the vast expanse of Java libraries. Python has a very good mechanism for wrapping and calling C/C++ libraries.
As far as I know, Arc does not yet particularly shine in this area, although I believe some arc hackers are working on it.
I like JRuby, cause it lets me use the gynormous amount of java libs in terse, "powerful", ruby without jumping through hoops.
Why are so many people afraid/resistant to this paradigm?
Open source used to have a similar issue regarding Microsoft Word documents: "Open source operating systems were rubbish because there wasn't any software to read and write Word documents." Now that such functionality is available, the argument moves to another feature such as boot time or support for native applications.
Let's imagine that we have an open source operating system that runs all games, boots in 10 seconds, works with 6000 random devices and comes installed with a terse language which fetches web pages, parses XML, talks to 20 databases and Java applications. How much of this functionality will be of any consequence in 10 years?
Also, I disagree with his criteria. Speed isn't really important for it. And dynamic typing, which he omits, is very important.
Edit: How is speed not important?
Otherwise, we could all go pointing random ideas as false, anyway we haven't to explain why they are false, couldn't we?
Edit: also, maybe you don't like blue after all :)
Example: why is my shirt red? Because I only have red shirts on my house. Why isn't my shirt blue? Please refer to the previous explaination of why it's red, then it follows it can't be at the same time red and blue, etc etc.
And it's great it's like this, because otherwise we could be saying a lot of random predicates are false and no one could argue with us. So I think asking someone "why x is false" isn't illogical, after all if there is a reason to say "x is false", surely there is a way to prove it's false indeed.
Edit: part of the problem with this discussion, of course, is that there is not a fixed meaning for "why". "Why" is a very ambiguous question, it doesn't specify whether we want to know the intention, the proof, the cause, etc. And even, what is a cause? If you are talking about historic events indeed you can't explain why X didn't happen, but from the same pov you couldn't ever explain why Y did happen at all, if you see history as something deterministic & linear. So whether you're right or wrong actually depends on what you see on the word why...
That should be enough to disprove: "there's never an explanation for why something isn't true", and that's the point. I would have preferred a simple explaination of why PG thinks X isn't true than the assertion that such a thing can't be explained (actually I suspect he was being humourous or something like that, obviously he has to have a reason to think like he thinks).
Also, you can always reply: "because I don't want to!", or "because I have other things to do", or "because I don't see a reason to" (it's like asking "why should I..?"), to "why don't you ... X?" :)
First of all, Gatsby contains no animal characters.
The Great Gatsby contains two dogs and a cat. While none are given names, one of the dogs (an Airedale bought by Myrtle Wilson) appears several times and has some identifiable characteristics.
Second, there is no evidence than any of the characters are attracted to animals.
In fact, it contains this scene:
"She looked at me and laughed pointlessly. Then she flounced over to the dog, kissed it with ecstasy and swept into the kitchen, implying that a dozen chefs awaited her orders there."
In conclusion, although bestiality is not one of the novel's primary themes, there is evidence that at least one character makes out with a dog at some point.
Please note: the only reason I have time to go and grep -iw gatsby.txt for "dog" is that I'm waiting for the program I'm writing to finish executing. This is why speed is important in exploratory programming.
E.g. it's fairly easy to prove that Boston is not in Oklahoma, but there's no explanation for why it's not.
Unless you are asserting that speed not being important is a fact of exploratory programming, rather than PG's opinion, I see no problem with answering my initial question.
If it makes you feel better though, I can rephrase:
Why is speed not important to you, PG?
I need a bottle opener to get the lid off a bottle. I need it because I need to bend the lid. I don't need a scarf to get the lid off a bottle. Why not? There's no reason you don't need a scarf to get the lid off a bottle; it's just not that useful.
If I may give this one last shot:
Why is slowness acceptable in a language tuned towards exploratory programming, and therefore an unimportant criteria when judging Arc?
"Boston is not in Oklahoma because it's in Massachusetts, and Oklahoma and Massachusetts are two separate states and no city can belong to two states at once (afaik)." Doesn't that sound like an explaination?
Now, an explaination of "why isn't speed important", could be, for example, that it's not important because it doesn't affect your final productivity at all, or at least not in a considerable way compared to some N other factors.
Now, what's the cause of such a difference in the final users' productivity, maybe that can't be answered. But I don't think you were asked about it.
Sorry for wasting your time with this discussion, but I don't really see your point. Maybe I'm wrong after all but until now I don't see how that could be.
Similarly, "why is your shirt not blue" is a silly question, unless there's any reason to suppose that it might be. For instance, if you show up to my "blue shirts only" party wearing a non-blue shirt, it's a perfectly valid question.
Now, as for questions about "Why is X not important for Y?", these can have sensible answers. Why is aerodynamics not important for a lunar lander? Because it never flies through air. Why is peanut butter not important for a lunar lander? Now that is a silly question, because nobody would suppose it might be.
So a question like "Why is speed not important in exploratory programming?" should really have a sensible answer. You could, for instance, say something like "Because in exploratory programming you typically only need to run short tests which don't take a lot of time anyway". To which I'd reply "Oh, I don't know about that, I typically run tests which take quite a long time". To which you could say "Oh, well I'm sure there are some cases where it is, but for most people most of the time it isn't". And I'd say "Yes, I agree with that statement".
The answer you give to that, as you imply in a later paragraph, is really an answer to the question "if a typical person thought aerodynamics was important for a lunar lander, why would they be mistaken?" It's fairly easy to answer that, at least with respect to our peers; we're used to vehicles designed with wind resistance in mind. But this is not the answer to the question of why it's not the case that aerodynamics is necessary for a lunar lander.
Similarly, though there is no answer to the question of why it's not the case that speed is necessary in exploratory programming, I could, if I knew why the OP mistakenly thought it was, have explained why he was wrong. But while I could guess why one of my peers might mistakenly think aerodynamics mattered for a lunar lander, I have no idea why the OP might think speed mattered for exploratory programming.
Sigh. What a huge waste of time this thread has been. I feel like I've just spent an hour trying to convince someone that the Monte Carlo fallacy was false.
;~)
In fact, one could clarify the question a little for the benefit of somebody who couldn't see why anybody would be so confused as to think that aerodynamics had anything to do with the moon, by saying: "Given that aerodynamics are important for many [though not all] vehicles, what properties of a lunar lander distinguishes it from those other vehicles in such a way as to make aerodynamics irrelevant?"
So ahem, bearing that in mind, given that speed is important in many [though not all] programming tasks, what properties distinguish exploratory programming from those other types of programming in such a way as to make speed irrelevant?
It's hard, because it _could_ be. The only way to properly answer is to take a random sample of programmers and show that there is no statistical difference.
This doesn't say anything about weather speed is important or not, of course. But it does give an idea that statistics (or practical experience) might be involved. Just remember, random sample = good, anecdotal evidence = bad.
Anyways, this meta-discussion has gone long enough anyways. I'm having too hard a time following it. Can't we go back arc and exploratory languages, please? :)
i can only see speed being an issue in these heavy data type things, for which you should probably be using an APL language (as they are quite exploratory-friendly -- and fast -- for these types of problems)
i've been using Arc for a while and the speed has never been an issue. most of your time developing a program is spent writing it, not watching it run. about the only thing that annoys me is that it currently requires mzscheme
This totally depends on the kind of program you are writing.
I just completed a class on Information Retrieval, the exact same kind of tasks and algorithms in the Collecting Intelligence books. One assignment, I spent most of the time allocated trying to get the program reasonably fast enough in Python to finish in less than a day. If a mistake was not apparent until the program had run for a couple of hours, it totally killed all of the exploratory programming advantages that Python has.
I finally gave up and rewrote the entire thing in Java a day before the assignment was due. I got my algorithm to converge in under 2 hours (had left the Python version running much longer with than this with no converging in sight), and was able to get some decent results to turn in.
Maybe I'm just greedy. I want it all. Speed, exploratory programming, lots of libraries. Common Lisp offers the first two, the last, not so much. Java offers 1 and 3, Python 2 and 3.
My next languages to learn are Scala and/or Clojure. Potentially the speed and libraries of Java, with many of the features that make Common Lisp and Python great at exploratory programming.
I do not see Arc on a path that will give it all 3 of these attributes in the foreseeable future.
I had an enjoyable time prototyping optimization heuristics in OCaml, where Arc would have been completely inadequate because it lacks floating point speed.
The reason for both is that you don't know the data types you'll be using when you start programming. It's very common to switch from a string to a hash to a structured record to an ADT, and if you have to change type declarations in every place that datum is passed to, you'll be spending most of your time changing type declarations.
How much slower are floating point operation in Arc than OCaml?
The reason I mention this niche (where C libraries you could call do you little or no good) is because things like optimization heuristics and machine learning are the most common form of exploratory computing and programming I know of. The biggest symptom of this is that there are dozens of "machine learning" toolkits, environments, libraries, etc all intended to provide a situation that frees you from worrying about extraneous details (like how to get data out of crappy HTML in order to do some kind of data analysis project), but a lot of people just code everything from scratch in vanilla Java/C/Python/whatever.
The way I write programs, generally, is I do the simplest possible test that still embodies the essential idea behind what I want to do and grow it from there. Thus a language supporting that has to be flexible, and I don't see why dynamic typing is necessary or sufficient for that.
The type system has little or nothing to do with also supporting essential features that support the way I think: closures, combinators, metaprogramming, and other things that often make life a lot easier.
Just like flippers, scuba gear, and a diving mask might be suboptimal for exploring a jungle or the moon, but great for exploring underwater, choosing an appropriate environment for the space you want to explore is something worth thinking about.
As an example, this is an interface to the Cairo graphics library for MzScheme: http://planet.plt-scheme.org/package-source/samth/cairo.plt/...
I haven't tried it out yet, but I sure had expected a lot more code. Now that is just a connector to an existing C library, but in other cases I was also surprised by the brevity of the code.
Actually I was surprised by the brevity of the code in "Programming Collective Intelligence", too. For the time being, I consider that kind of stuff part of my fun projects, so I decided to do them in Scheme. The upside is that if a library is missing, you get a chance to actually provide it yourself and make a name for yourself in the Open Source world.
Also, I like implementing those algorithms. With another language, there might not be an excuse for implementing the algorithms from the book by yourself, because libraries already exists. With Scheme, I have a justification to implement them myself.
Interfaces to web services might also not be so important, and easy to implement by yourself.
The Scala/Clojure approach makes this a blessing, rather than a curse.
Interesting language, with all the benefits of the JVM and Java libraries. (Which is why these languages are next on my list of languages to learn.)
True. It's just better to go into it with open eyes.
> Maybe you can often use libraries from other languages, too, as in the example of the Cairo
That helps, but it's still a time consuming task to understand a standard and a C (or whatever) API, design your own API in the 100 year language of your choice, and implement it (and document it if you're really being good). I spent a fair amount binding C libraries for Chicken Scheme (which has a good C FFI) and Scheme48, so I know this from experience.
There are two choices I see going forward immediately. Arc and the arc community starts implementing their own libraries or we find a way to use PLT scheme's libraries (ok maybe sbcl if you are using arkani).
Okay, I haven't used Arc in a while -- existing Scheme codebase -- but Arc is built on PLT, which is compliant with R5RS, which does include basic math functions:
http://www-sop.inria.fr/mimosa/fp/Bigloo/doc/r5rs-9.html
(search for sin, cos, etc).
Did Arc forget how to talk to the numerics library? Rebind "cos" "sin" and "exp"? Is this guy just being stupid?
'splain me please.
(xdef 'cos 'cos)
(xdef 'sin 'sin)
(xdef 'exp 'exp)
(xdef 'cos cos)
Perhaps there should be AEP = Arc Enhancement Proposals a la Python's PEPs.
Interest on arclanguage.org seems to be fading.
It's hardly programming at all.
i wonder why people are not writing more apps in arc. there seems to be a lot of activity porting it to different environemnts, writing compilers, optimizers, creating a repository .... but no app. (other than hacker news)
These are real apps. I must admit Arc is really great (even as it is now) for these small, basic but usually time-consuming webapps. It doesn't prevent me for writing a compiler for Arc :) (which is itself written in Arc and is actually the biggest piece of code I've written in this language).
As for your next question ("Why is enhancing Arc fun?"), it's because Arc provides many tools for manipulating Arc code (macros, code-is-data), and few tools for writing applications (libraries). The implementation is also fairly simple, vs. trying to hack on Java or Python or C++ compilers. Path of most impressiveness per unit effort seems to go through porting it to another language, rather than writing another webapp that'd be more impressive if done in Rails or Django.
There's a place for great languages with less-comprehensive libraries, no doubt. But if you want to make something that works today...