sdtimes.com
Specifically:
Rumpus: So tell me about the engineers.
Employee: They’re weird, and smart as balls. For example, this guy right now is single-handedly rewriting, essentially, the entire site. Our site is coded, I’d say, 90% in PHP. All the front end — everything you see — is generated via a language called PHP. He is creating HPHP, Hyper-PHP, which means he’s literally rewriting the entire language. There’s this distinction in coding between a scripted language and a compiled language. PHP is an example of a scripted language. The computer or browser reads the program like a script, from top to bottom, and executes it in that order: anything you declare at the bottom cannot be referenced at the top. But with a compiled language, the program you write is compiled into an executable file. It doesn’t have to read the program from beginning to end in order to execute commands. It’s much faster that way. So this engineer is converting the site from one that runs on a scripted language to one that runs on a compiled language. However, if you went to go talk to him about basketball, you would probably have the most awkward conversation you’d have with a human being in your entire life. You just can’t talk to these people on a normal level. If you wanted to talk about basketball, talk about graph theory. Then he’d get it. And there’s a lot of people like that. But by golly, they can do their jobs.
I lost count of the number of untrue statements there after about ten.
I do wish they hadn't had so many timekeeping problems, though (and I'm not sure what was up with the aggressive travelling calls in the first half either).
I guess the problem being described, somewhat obtusely, is that massive amounts of dependencies (in Facebook's case million of lines of code) are loaded in order to execute a relatively small page.
I'll admit that to a CS person who understand compilers, their description makes it look like they don know what they're talking about. But if you explained Facebook's interpretation/compilation problems to a non-techy, it would probably come out sounding similar.
(Note comment reuse: http://www.reddit.com/r/programming/comments/aodzg/one_faceb...)
SBCL and V8 and TraceMonkey are all very fast "scripting languages" because they have a good codegen in the runtime. PHP does not do any codegen.
If I were explaining this to a non-technical person, I would say: A computer works by executing instructions, one at a time. When PHP is executed, it is compiled to a list of instructions, but not ones the processor can directly understand. It uses a "virtual machine" to read each instruction and then execute the corresponding instructions on the real computer. This is slow, because what could be one instruction actually becomes many, sometimes hundreds, of actual instructions. The goal of this project is to skip the virtual machine and run our code directly on the real machine, meaning that we won't have to do as much unnecessary work. This will make our application run faster!
Now, this still might make CS folks mad, because there is bookkeeping overhead other than deciding which instructions to run (you can't store "scalars" in registers, after all), and of course modern CPUs don't execute one instruction at a time. But now we're wandering off into esoterica -- at least the basic idea is right, and I don't use meaningless expressions like "scripting language".
They're trying to explain the problems that Facebook are having, which is that there is just such a large volume of code. They aren't trying to give a short course in interpretation and compilation.
Scripting language is not a meaningless expression. Many papers use it. I submitted papers with "scripting language" in the title to PLDI and POPL, and there was not one complaint or problem with the term from reviewers (who found plenty of tiny "errors" otherwise).
By this definition C isn't a scripting language, as it was small and stood that way.
Python main() functions by Guido van Rossum
Its a tricky term to define, much like "compiler". Actually "compiler" is a nice analogy: everyone knows what one is, but the actual test for "is this a compiler" is shaky, and returns true for lots of things that aren't _really_ compilers.
Anyway, Ousterout coined it in his paper on TCL, but annoyingly chose not to formally define it. The set at that time was (I think) {TCL, Perl, sh}.
But because he didn't define it properly, the word has been used for years without a proper definition. The best I can do is to say read my PhD thesis, starting on page 7, where I'd spent a meaty six pages explaining in as much detail as I can what it means to be a scripting language.
[snip]...consequence of this is that in many implementations, operating on a variable with automatic or static lifetime through a reference, although syntactically similar to accessing it directly, can involve hidden dereference operations that are costly.[/snip]
I admit it sounds a bit off, like the quotee doesn't know exactly what she's talking about, but she's not egregiously wrong.
1) The term "scripting language"; meaningless, and usually just used as an insult (as in this case) rather than with any reasonable definition.
2) Browsers don't execute PHP
3) Not all scripting languages are executed based on line order.
4) PHP doesn't require values to be declared before they're used.
5) Not all compiled languages are compiled to a separate file.
6) Most modern compiled languages compile to byte-code, which is not more "executable" than the original source.
7) Compilers do have to read the program from beginning to end. This might just be my ignorance, but I've never heard of a random-access parser.
8) Being compiled doesn't necessarily make execution any faster.
9) The compiled file still has to be read entirely, before it can be executed. Depending on code size, sections of it may be read repeatedly.
However, not all software engineering groups have the capabilities to produce such highly optimized binaries, and there is always the risk that a user's particular usage patterns will differ enough from the expected patterns that they will lose the benefit of this extensive optimization. However, in an interpreted or byte-code language a lot of the same information needed for optimization is available to the run-time. A run-time designed for optimization may be able to take advantage of that, creating super efficient code paths based on actual usage. This model is more difficult to implement but potentially more robust than statically optimized compilation (and also has the potential to take greater advantage of differences in hardware, a statically compiled native binary doesn't have the ability to morph its optimization based on whether its running on a single core Atom or a 6-way Core i7 cpu, or some 100-core monster of the future, but a run-time potentially can).
In the average case most of this is just theory, but the potential is very real.
Some worthwhile background reading:
Trace Trees: http://www.ics.uci.edu/~franz/Site/pubs-pdf/ICS-TR-06-16.pdf
A blog post / talk from Steve Yegge on dynamic language performance and other topics: http://steve-yegge.blogspot.com/2008/05/dynamic-languages-st...
BTW these technologies are not interpreters, but compilers (but runtime compilers).
1) You can read from the interview that the quotee was speaking to a non-technical person. Even so, they're using the term in the commonly used sense. But you haven't shown that they're wrong, only that you disagree with the term.
2) You are right. This is the only falsehood I can see here. I'll stipulate that its ok in a 'you know what I meant' kinda way.
3) Your answer to 2) depends on the quotee speaking about PHP, but now they aren't? And in 1) you said that scripting languages don't exist. Basically, you're reaching. Anyway, please name a scripting language which is not executed on line order. PHP certainly is.
4) Values aren't mentioned anywhere. Anyway, PHP requires functions and classes to be declared before use. (Strictly speaking, values don't exist before they are used, so what you said doesn't make sense. I presume you meant variables, but variables aren't declared. So I presume you meant defined, but strictly speaking there is no such thing as a variable in PHP anyway, just a mapping of strings to values (see: all the literature on the topic, including mine)).
5) They're obviously talking in the context of a native compiler, so I see nothing wrong with this.
6) They do? First I heard of it. I'm guessing you've got a funny definition of 'modern compiled languages' to back this up.
7) Executables don't though.
8) Not necessarily, sure. The first iteration of my PHP compiler was 10 times slower. But to say that the quotee is _wrong_ just because some implementation can be slower than some other implementation, is faulty logic.
9) What's your point? Why are they wrong?
So I'll give you one. Two if we count number 8, but that only counts for the most extreme form of pedantry, and it doesn't really contradict anything. So, yeah, one. A long way from ten.
<?php
echo foo('baz');
function foo($bar) { return $bar; }
?>
This looks to me like you can use PHP functions before declaring them above.
echo foo('baz'); // error
{
function foo($bar) { return $bar; } // not pulled to top of file
}1) All the more reason not to introduce incorrect terminology; non-technical people won't be able to understand that it's wrong.
3) There are legitimate definitions of "scripting languages"; the one I use is that a scripting language is not useful without 3rd-party code. The Bourne shell and JavaScript are classic examples of scripting languages.
QuakeC is a scripting language which isn't executed in line order.
4) Values are procedures (PHP doesn't have functions; the keyword is mis-named), classes, or variables. They do not have to be declared before use in PHP -- if they did, writing mutually-recursive procedures would be impossible in PHP.
5) I see nothing "obvious" about your statement. They state that compiling results in a separate file, which is wrong because some compilers don't.
6) JavaScript (in all popular implementations), Python, JVM languages (Java, Scala), .NET languages (C#, F#, VB.NET), Perl 6. I think even Ruby has a bytecode compiler, now.
7) There is no indication that this new PHP implementation compiles to native executables.
8) Their claim is that compilation makes execution faster. There exist cases where compilation does not make execution faster. Therefore, their claim is incorrect.
9) Their quote states that compiled binaries don't have to be read from beginning to end to execute. This is incorrect. When a binary is executed, or a library linked, it is loaded entirely into memory.
3) Please define it. Note that I spend six pages in my PhD on defining it, and there are no formal definitions. Ousterout introduced the term, and didn't define it.
That's a crap definition of scripting language. I don't even know what it means, and it's unusable. C++ is pretty much useless without the C++ standard library. Is it a scripting language now?
3) Nice. The fact that you could dredge up a minor language which peaked in 1996 does not make the quotee wrong.
4) This is all wrong.
> They do not have to be declared before use in PHP -- if they did, writing mutually-recursive procedures would be impossible in PHP.
Whether or not mutually recursive procedures have to be declared is a matter of parsing style. For example, its there in C since it was created using a one-pass compiler. Its not there in Java. In PHP, declaring a function (say x) puts the function into the function-symbol table under the entry "x". When calling x(), the function-table is looked up. It is not necessary to have defined x to parse code that uses it. For example:
if (false) { x(); } // legit, x() is never called
> Values are procedures, classes, or variables.
I presume you mean that variables, classes and procedures are all kinds of values. That is simply incorrect. Classes are not first-class values, and first-class classes and functions are approximated in PHP by allowing classes to be instantiated by name (using a string) at run-time. This is changing slightly in 5.3, but the semantics are complicated, and still use strings.
Variables are just syntax in PHP. As I said, look at any literature on PHP (or javascript if you prefer). Variables are syntax for keys in a local symbol-table, and aren't real entities. They are certainly not values.
> PHP doesn't have functions; the keyword is mis-named.
Evidence? I can't think how this is correct.
5) So what if some compilers don't. Nearly all compilers do. The level of pedantry here is astounding. Just because you can list an edge case in which they are wrong, does not make them wrong in the general case.
6) Ah, your definition of "compiled" is "bytecode compiled". What a funny circular argument.
7) Except that the quotee says it! "But with a compiled language, the program you write is compiled into an executable file."
8) "Is a rocketship faster than a bicycle?" Yes. "Aha, but if my rocket isnt moving, the bicycle is faster. Therefore I assert that rocketships are _not_ faster than bicycles". Bollox.
9) Loading from memory is not "reading" in the sense the quotee used, which was clearly parsing. I think you're being deliberately obstinate: try using the context of the article to determine what the words mean. I could argue that binaries do not have to be fully read into memory (they are loaded lazily, page-by-4k-page, by most modern OSes), but I don't want to start a discussion on it. Heaven forbid an obscure 90s OS used 8k pages.
Anyway, you're deliberately twisting the quotee's words and being pedantic, so I'm done here.
8) Being natively compiled does not make that program automatically faster. It depends on the use case (I/O vs compute bound) and it also depends on the quality and complexity of the code generator and optimizer. Furthermore, bytecode compiled languages (as opposed to native compiled) may be able to better optimize at runtime using JIT compilation due to being able to make assumptions you couldn't make at "compile time" in AOT compilation. Saying that natively compiled programs are faster than other kinds of programs is simply not always true.
- thought that Facebook's servers would be hosted in their office
- was unfamiliar with the idea of usability studies or tracking eyeballs during usability studies
- was baffled by the idea that facebook would track your clicks throughout the entire site to help determine who your closest contacts were
- was also surprised by the fact that in the cloud, nothing is ever truly deleted
- was surprised to see that facebook employees would have the ability to login as any user in the system
Time for a break.
With byte-code you can compile sections of code to machine-code, based on runtime profiling / type-inference (as the JVM does). With ahead-of-time compiling to machine code, you're probably going to end up with a serialization of the PHP code in the final executable, plus an interpreter :)
This person knows just enough to be dangerous.
Well, I was able to put all the pieces together on this one, finally, and I now understand exactly what is up: Facebook has rewritten the PHP runtime from scratch.
Look if it's true, this is cool, it will no doubt be a great contribution to the OS world, but let's wait until Tuesday or until we have more concrete info beyond this author's guess that FB has completely rewritten the PHP runtime. I realize the author has little to gain from making this up, except for maybe 15 min of fame on HN, but still, don't believe everything you read.
Unfortunately, I have this sinking feeling that this is not going to be that.
Twitter from Ruby into Scala and JVM: http://www.artima.com/scalazine/articles/twitter_on_scala.ht...
I think there are two aspects of a language (correct me if I am wrong) -- it's elegance/simplicity/breadth of code produced by programmers that write with it --- and then the efficiency/effectiveness/HW-optimized executable code produced by its compiler once it's run: so how does PHP and Facebook fit in with all this?
The only complex thing about Twitter is its size, and I bet their developers are working round the clock just to keep it from falling apart.
I've worked on many Rails apps, and have upgraded the apps from version to version. It's a pain when key elements of the API shift, but it's not that bad...even when the project has monkey-patched Rails a lot. And twitter certainly has the resources to afford to dedicate a few programmers to this task, so I'm just not sure I buy it.
Supposedly, the problem was caused by Cache Money, but nobody at Twitter wanted to risk moving to a different version again. They're still on 2.0 today. :-)
Another fun fact: Twitter has over 1,500 remote git branches. They also have bright green deer in the reception area of their office. :-)
Deep down inside I wish they were using ERB (or HAML) however, and he was writing HERB (or HHAML). I'm still enamored with Ruby syntax, and wish it would get some more big business love.
Facebook has a large, well-tested codebase, with a huge amount of infrastructure built on and to support PHP. It would be ridiculous to expect Facebook to migrate all of this to a different language.
Re: the article itself, you'll have to wait until Tuesday.
This caught my eye - an interesting use of the term "open source" that I haven't previously been aware of. This is only a single datapoint (and the article states the project is going to be opened up anyways), but I do have a feeling the term has been diluted and joined the buzzword ranks.
Kinda silly to have a facebook page for an open source project, though.
http://developers.facebook.com/opensource.php
I agree it's a bit silly. I think they hope that it'll catch on sometime in the future and they'll have yet another type of group socializing on facebook.
It seems that the only effective purpose for pages is for celebrities/companies to push updates to their fans (and almost always ignore the reverse direction), and groups are even worse. Most pages and groups I see are things that you join because you agree with/identify with the name and then totally ignore. I wonder if facebook a) cares, and b) could do something to fix it.
I recently learned of a counter example. There is a locally owned sports bar and restaurant that has a very active group. Most of the members are fans of sports teams that are across country and the games play regularly at this bar. The owner is active in the group too and definitely encourages and responds to conversation. The other group I know that is relatively active has more or less the same characteristics. It's people who identify with the group but are otherwise disjoint - and this is the best way for them to casually communicate. I think it's safe to say this was the original intent of groups and pages. Celebrities and companies are just taking advantage it. Of course, I have a handful of pages on my facebook profile, so I guess I'm as guilty as anyone else. (I'm With COCO!)
EDIT: abusing -> taking advantage
And is PHP really considered "pokey"? Sounds like this guy is making stuff up because I get execution times of >0.01 seconds on my micro-framework.
Also, his comparison of Zend's "folks" (which ones? Founders? Execs? Sales people?) to "gestapo officer looking for a spy: "What? Who said that? Who said it was slow? Tell us their name!"" isn't too apt - they'd probably like to know who complained about PHP performance, so they can offer them their products/services, rather than silence them somehow.
The implication is that writing code that uses PHP's built-in libraries is pretty fast, but the more you write in PHP itself, the slower it gets. For example, my impression is that Yahoo isnt really written in PHP - its written in C patched together using PHP.
The Zend engine was "rewritten" for PHP 4, PHP 5, and to a certain extent for PHP 5.1. I guess it wasn't a complete rewrite because the legacy code from about 10 years ago is still in there. Anyway, its still dirty, badly written, slow, and very very badly commented. Hacks abound (and not the good kind).
The problem is that because it's interpreted at some point PHP slows down in proportion to the size of your code base. A small app runs really fast, a big app with 100,000 lines of code will kill your server unless you modularize it really really well - which harder than it seems, because the more modularized you make it the more separate "includes" you end up with in different files and then you come to realize that including a lot of files itself is a problem. And the nature of PHP's very loose coupling tends to lead to code that is nearly impossible to do large scale refactoring on once you have gone too far down the path.
I work on an application that has a very thin PHP layer that performs some simple web services that are the back end for a pure Java web app. Amazingly, when we load test it, the PHP part is the bottleneck, burning CPU like crazy just parsing all our files ... over ... and over ... and over. The java code meanwhile, while theoretically doing far more "work", is completely bored. We will probably look at using an accelerator of some kind or maybe just rewriting all the PHP in another language.
On the other hand, if you have an opportunity to switch out PHP for something better (read: nearly anything), you should. Otherwise it might grow to a point that you can't remove it.
Yahoo! is a company, not a single application. Not everything at Yahoo! is written in PHP, but the vast majority of Yahoo! properties do use PHP heavily on the frontend, and not just as a way of patching together C extensions.
Really? In what actual benchmarks, real world situations?
You did read this part right?
Many of these are largely interpreter.
http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
The Python spectral-norm program uses 4 cores the PHP spectral-norm program uses 1 - that's why Python seems so much faster on spectral-norm.
The Python binary-trees program uses 4 cores the PHP binary-trees program uses 1 - that's why Python seems so much faster on binary-trees.
I wonder why its changed. PHP certainly hasn't gotten faster in the meantime.
Of course if your workload is not dominated by script running CPU time then it doesn't really matter. Even then, until the cost for extra servers and their management exceeds the engineering cost to recode, add servers.
I get execution times of >0.01 seconds – hope you aren't planning to handle more than 50 transactions per second.
It all depends what kind of world you live in. I know people that use their hard disks for booting and loading caches and that's it. If one query in a hundred requires a seek they will not keep up. Most of the rest of us could happily fork a CGI PHP for each request and not notice.