My Company Is Using OCaml for web development. Why?
nicollet.net
nicollet.net
The advantage is a uniform interface, like a file-descriptor in Plan-9 (an analogy used in latest InfoQ interview with GO/Newsqueak creator), the HTTP protocol.
And for that, languages like OCaml or Haskell should flourish. Especially functional languages, since typical REST API request should be nothing but data+auth token (whether that's cookie or oauth token).
Applications developed in that model—besides being much simpler to write and test—are easily cacheable and more secure by default. The added bonus is that object oriented languages, that have been most successful in GUI programming, can be once again utilized purely for that.
If not embracing this model one is left with little appealing options: 1) continue to process text in perl-like manner with RoR/Django/etc., coping with nondeterministic behavior and security 2) to invest in frameworks that hide HTTP at all, coding to widgets instead of pages, and to let framework decide how to push that over the wire, which usually means that some sort of continuations are employed, which are inherently HTTP-unfriendly and uncacheable outside of the app's context 3) to replace HTTP with something else, probably some sort of RPC
But the opposing force is the need to ship quickly, and languages like Ruby are highly optimized to this goal, and thus they attract a lot of the best developers who spend time increasing the ability to ship quickly with Rails. This gives an ongoing advantage, where new startups have to look at the potential library and framework disadvantage when they're evaluating more powerful functional languages. The cost of the extra bugs is hard to factor in, partially because you never see all of them, and much of that code may be discarded before you have to pay the price of a bug anyway.
Due to this dynamic, as well as the fact that imperative programming requires less training and mental rigor, I think functional languages have an uphill battle. I hope I'm wrong, which may happen if enough Rails apps flame out spectacularly enough to bring down their companies, and people realize what a ticking time bomb you can create with loosely-typed languages if you're not careful.
The market demands different qualities at different times (and in any market, there will often be niches that need different qualities from the rest of the market). The "market" is probably best represented by paying customers, but other parties (managers, developers and users in general) also have an effect. What does the market demand at the moment?
It seems that at the moment, time-to-market (i.e. fast development) is most in demand. This includes being "agile": able to add features quickly, improve them or change them, and to debug them. This is driven by development speed, which can decrease both time-to-market and development costs (and therefore customer price). Other qualities, like performance (runtime speed) and memory usage seem less important. Reliability seems less important.
However, there are different parts of the market that value these differently. For example, the financial market value correctness, so it's not surprising that ocaml has found a home in janestreet. Space shuttle software also does. I would say that in desktops, quadcore, 4GB RAM performance now outstrips what many people need, resulting in the hiccup of "netbooks", and finally the smartphones - much less powerful, but customers are willing to buy them because they offer convenience. Within smartphones, performance of software became important again, because within the category (given convenience) the hardware hasn't yet outstripped demand for it.
Another example is in MS Windows, when it first got an adequate GUI interface in windows 95, people bought it in droves even though it was buggy and unreliable. Then, combining it with NT in Windows XP, it became reliable, and people bought that. Then, security was the issue, and that got fixed. It may seem a little surprising that Windows 7 is popular, but one thing is that all those books and sites about "windows annoyances" mean that there's room for windows to improve... which means they can sell another version. If it no one complained because it was perfect, that would be bad news for microsoft. (of course, their run may be over now, if smartphones replace the desktop, and they can't win that market away from apple).
Now, just because development speed has been important for a while doesn't mean it always will be. Reliability and performance may become more important again, especially in niche markets. And it is there that ocaml may have an opportunity.
But meanwhile, although ocaml is amazing in that it has the expressiveness of functional programming, the correctness of strong typing, and the performance of C, it is a double straightjacket language: first it has to be functional (though you can mutate, it's not as nice); then it has to be strongly typed. You can't just code it they way you think about it. You have to adapt yourself to the machine. (note: this is based on my personal experience with ocaml).
tl;dr OCaml has many benefits, which I don't question. What I am saying is that the mainstream market doesn't currently value those particular benefits of ocaml over other attributes.
PHP is weakly typed. Common Lisp, Clojure, Haskell and OCaml are strongly typed.
PHP is dynamically typed, Haskell and OCaml are statically typed.
That said I would like to learn more about these statically typed functional languages.
I was trying to make point that simple nature of REST protocol makes it possible to write webapps ‘guaranteed’ to be correct, at least server-side.
Isn't that the definition of static typing?
On the hindsight the better term to use was probably just “type safety.”
I see it as a trend, actually, statically typed languages grew stronger in various fields, from operating system kernels [1] to web development and shell scripting [2].
Stuff like how to process a REST request is simply details. I don't understand why you are focusing so much on this aspect, but I have seen this anti-pattern so often that it repulses me; most examples of RESTful APIs are bullshit, written by people who did not take the time to actually understand REST principles. A good example I documented on my blog is how awful and stupid the ThreeSharp Amazon S3 library is for C#. And writing it in the same style in F# would be just as ugly and stupid! I also discuss why it is a joke that S3 itself is called a REST API, and point out how the language examples in Amazon's documentation is COMPLETELY inconsistent from language to language -- there is no correctness-preservation across language implementations!!!
Incidentally, when people say "web development", they can mean many things, with True REST being just one of them. Most people are actually referring to corporate intranet applications, such as data entry forms and stock analysis tools that chart trends, provide real-time data feeds, etc.
I don't actually get what there is relevant about any possible business-oriented perspective you mention at the beginning. Actually, you have allocated most of your comment for advertising your blog…
As to your last paragraph, there's nothing like True REST etc. I'm not buzzword happy and I don't generalize things like that, as you can see in my other comments anyway. I wasn't referring to all those irrelevant things you've mentioned. My observation was that most web development—of various sort—was done by keeping state server-side. That approach is fine as long as you don't have multiple pieces of independent logic on one page. In that case the whole thing becomes complicated—you can either 1) use text-processing techniques, which is my hypothesis for why languages like PHP, Python and Perl are so popular for webdev or 2) abstract stateless HTTP protocol away, which results with very unfriendly and difficult to scale artifacts, like continuations.
The separations of concerns: server logic and presentation, that is possible by properly utilizing HTTP and by having powerful enough client-side environment (Javascript VMs) enable for a much saner approach, and that's why I think strongly typed languages are relevant for web once again.
Hope this clarifies my position :)
Sorry if it came across that way. I really don't check who reads my blog, there is no comment mechanism, and I do not like linking it anywhere except to friends. Even then, I prefer they not read it directly. My blog is mainly a personal thought dump and collection of links.
Quite sad, if you think about it…
I can't wait to see what people cook up over the next five years. Particularly with the trend towards tablet computers.
I see you're actually eager to make a full leap from 'WEB' to native apps. I don't disagree, but I think REST+DOM/JS will still have its place there. A lightweight alternative to full-blown applications, interactive documents that load in the blink of an eye and transparently integrate with your ecosystem. I guess it's what web was supposed to be in the first place, but with some added zing.
Most really awesome apps being written these days, that use HTML, are using the CANVAS tag and doing stuff like building terrain maps of buildings and showing people holding 4.5" smart phones where in the building druglords are. Crazy stuff like that.
Still I am jealous of the numerical advantage the web gives people. The iphone/ipad ended that though.
Are you referring to HTML and CSS?
RE: You don't have to keep state with a stateless protocol.
You are referring to HTTP.
RE: Now you get a nice (mostly) standardized interface and you get to build a lighting fast native GUI for your app.
What are you referring to?
The model for writing drivers, and plugging in drivers, in ODBC, is R. E. T. A. R. D. E. D.
Edit: I forgot to mention that using the drivers suck, too. The ODBC transaction interface is basically RPC, and is becoming out-moded with recent language research on remote batch execution.
Usually, you have a custom transport protocol for client/server, like the Microsoft SQL Server Tabular Data Stream protocol. It is ultimately not very extensible. Imagine something like LINQ today. Would be nice to just send that expression tree to SQL Server, and not have a client-side or middle-tier ORM tool have to transform the tree into SQL.
And wouldn't advertising OCaml (or Haskell) be a way to get talented folks quickly? It seems a lot of such programmers wish they could have a job where they get to use a decent language.
More: http://www.janestreetcapital.com/minsky_weeks-jfp_18.pdf
1 - can express 'weird' things that other languages would not be able necessarily express without strange gyrations.
2 - if you can develop in it, you're probably on the upper half of the developer bell curve. That brings systemic improvements in general.
Why hasn't this happened? Preumably if you use an advanced language you'll be a lot more productive because of the language and because you have the best developers.
Why haven't we seen shops that use advanced languages just utterly clean up? "Oh, you're doing 30fps with 1m vertices -- we just did 90fps with 50m vertices -- and btw, we did this w/o using the GPU". Or "Chrome? We built a new web browser that supports every draft of every aspect of HTML5, w/ a Javascript parser that runs at twice the speed of Chrome, and we wrote a new H264/WebM codec that doesn't require a GPU and can do HD video on a 286".
It just seems like for all the advantages that using an advanced language presumably gives, the software I see is simply "competitive". Maybe it will be the best in some spaces, but within the range that one would expect the best to be if everyone used C, for example.
And even among those who know these advanced languages really well, we don't see them fighting for them in their organization the way one might think given all they should presumably provide. Think Norvig, Steele, Abelson, Meijer, etc...
If development time, algorithms, and ability to wrap the language around the problem is the gate, an advanced language is going to be better.
According to the Alioth Shootout, OCaml, Haskell, and SBCL are in shouting range of C/C++ speeds. Link:
http://shootout.alioth.debian.org/u64q/which-programming-lan...
Therefore, if I was selecting an 'advanced' language for a performant real-world system, I would select one of those.
I agree, but I think this is the problem. If advanced languages are better, its surprising that it doesn't manifests itself in virtually any way that one can measure.
For example, if you're a CTO deciding which language to use, there is virtually no data to suggest using an advanced language is advantageous except religious arguments. Which is odd given they're supposed to confer both a superior language and superior developers.
Where's the beef? If I'm building a skyscraper, I'm using steam shovels, not standard shovels. The data makes it clear which is more effective. There's no remotely similar argument to be made for advanced languages.
Or how about the idea of having special syntax for some common data types, like lists and dictionaries and (sometimes) sets? This kind of thing is spreading and becoming more popular, because it really does seem to make people's jobs easier.
Some more up-and-coming language attributes include type annotations, more immutability by default, and Erlang-style message passing concurrency. And maybe support for asynchronous I/O, either with JavaScript-style anonymous functions or with coroutines. Some of these things really can make certain programming tasks dramatically easier.
The religious arguments tend to be anchored to languages, but there's some real information if you look past that to a deeper level.
/done with thread
> Why haven't we seen shops that use advanced languages just utterly clean up? "Oh, you're doing 30fps with 1m vertices -- we just did 90fps with 50m vertices -- and btw, we did this w/o using the GPU".
More productive doesn't mean more performant. It means building systems more quickly, a phenomenon for which there is ample evidence.
Large companies using less productive languages can still achieve great performance because they can throw lots and lots of resources at the problem, even if the payoff per head is much lower.
Where performance matters, I'd think they'd just build codegen systems, using the advanced language. They should still have a fundamental advantage here since they could build codegen systems faster and presumably with better codegen techniques.
It means building systems more quickly, a phenomenon for which there is ample evidence.
So what is the evidence?
Large companies using less productive languages can still achieve great performance because they can throw lots and lots of resources at the problem, even if the payoff per head is much lower.
But even with small/medium sized companies you don't see this happening. It just seems like everytime someone tries to show me how some company has done something incredible with advanced languages it is really borderline. It's never, "wow, slam dunk"
For example, I've never seen a Wolfenstein 3D from an advanced language. Soemthing that just blows away everything else. WordLens wasn't done with some advanced language. Google search engine. Excel. VMWare. There aren't a lot of big jumps in SW, but few, if any, seem to have come to market quickly (if at all) due to advanced languages.
You'd think they'd be able to take these advanced languages and top flight devs to change the face of software. But it usually turns into arguments about how their code is less buggy or produced faster. Both hard to really judge objectively given one rarely writes the same program concurrently in two different languages. And the delta rarely seems large enough that it is obvious.
http://www.nt.ntnu.no/users/haugwarb/Programming/Haskell/has...
I'm also curious as to why functional programming hasn't been more successful. I think that, whilst such languages are especially elegant for some forms of control flow, they require more mental contortions than mixed-paradigm languages. Since this makes them less comfortable, they're less popular still.
While it depends on what you're comparing to (is the competition writing assembly by hand or writing Python in a functional style?), I don't think you'll see someone working in an advanced language getting to market shockingly faster (and their code won't run faster), or something similarly strikingly noticeable from the outside. More likely, when a company working in an advanced language does get ahead, you'll see them having a much easier time keeping up with competition and adapting to new trends due to much lower maintenance overhead. Resources expended on debugging and refactoring isn't a measure we really look at when talking about most successful projects, but it is still worth improving.
So, most of the time, you end up with a piece of software that's short and sweet and hopelessly brittle.
I've been using Haskell a bunch lately, and I've found that by the time the compiler stops snarling and hissing at me, most of the stupid little bugs are gone. That's a nice thing sometimes, even if appeasing the type system can be annoying.
I can do a fairly adequate analysis of OCaml-versus-PHP or OCaml-versus-C# because I'm familiar with both sides of the equation, but I wouldn't be able to do an honest comparison of OCaml and Haskell.
http://www.reddit.com/r/ocaml/comments/fmpl8/i_have_yet_to_l...
http://www.reddit.com/r/haskell/comments/fmp66/i_have_yet_to...
and this guy's universe of FP languages:
http://www.slideshare.net/brweber2/functional-concepts-for-o...
slide 12: haskell, clean, F#, ML/ocaml, lisp, scheme, scala, clojure, erlang, XSLT
I would disagree that lack of popularity of language is not a technical problem- it is because there is less community code available.
Anyway you can access it at: http://webcache.googleusercontent.com/search?q=cache:http://... as well. And I believe OP is working on bringing the site back online.
For example, at any point in time, content can be removed from cache—before completing its usual 24h lifecycle—to accommodate resource usage for different sites.
[1] "Experiences with CoralCDN: A Five-Year Operational View" http://www.coralcdn.org/docs/coral-nsdi10.pdf
EDIT: it's back (or at least, looks like it). The traffic is being drained to a static copy of the page, so don't be surprised if comments you post on that page don't appear.
I'll be back with an actual (cache-based) solution soon, hopefully.
http://rimuhosting.com/howto/memory.jsp
Scroll down to RESOLVING: HIGH APACHE MEMORY USAGE
EDIT: I realized after I posted that simply posting two links might seem snarky. I don't mean to be. I know these links because I've been in the same spot you are. I think we all have. That's how we learn :)
I'll try to install the cache plugin as soon as I retrieve control over my server.
http://www.cherokee-project.com/
as close as you'll get to Zeus Web Server for free.