How Badoo Saved $1M Switching to PHP7
techblog.badoo.com
techblog.badoo.com
The problem was usually the frameworks.
I remember benchmarks that went down in orders of magnitude as soon as you loaded a framework. That experience was 7-10years ago. I assume this changed by now.
Python 3 made huge changes, especially to string handling and input/output which is what most applications do all the time.
And perl 6 is practically a completely different language. I wouldn't count that.
What is interesting about php7 is how much effort and how little complaining goes into porting the C extensions: that API has changed a lot and the documentation is still spread all over wikis and blog posts. And yet there is a php7 port of many third party extensions already
But I guess the huge performance improvements are a strong motivator even there
too bad for py3's fundamental problem: that it broke the interface so badly, people still don't use it. PHP7 has no such problem, and gets to add boatloads of new features (which will be immediately used/adopted) while deprecating old versions of the language.
Php7 did nothing like that, that's why the "porting" process was so light. But it also means it didn't fix any of the big things.
This is nonsense, PHP 7 contains a long list of backwards-compatibility breaks:
http://php.net/manual/en/migration70.php
If PHP 7.0 didn't break anything, it would have been called 5.7.
The upgrade is easy simply because these breaks do not have as much of an impact as Python's did.
http://php.net/manual/en/language.types.type-juggling.php
How would you fix this without breaking most PHP code out there?
How would you fix ==, > and < behaviour without breaking most PHP code out there?
How would you fix Unicode without breaking most PHP code out there?
If PHP fixed things like this, it would be a decent programming language, but it would also break a lot of things. Kinda of like what python3 did, only that it didn't have to break quite so much, since python2 was already pretty decent.
1: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=php+7.x
2: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=php+5.6.x
On a higher level: It's much harder to get right.
Git pull and the code is updated. Restart the web server and you are good to go.
I think it didn't get the beating it deserve. If it was by merit alone, this language should be dead by now. People keep saying that it is improving. But actually it is not. The rotten core, what makes php php, very much remain, and it is still a very dangerous language to use for anything serious [1].
[1] https://www.reddit.com/r/PHP/comments/41bm7j/new_rfc_allow_s...
I'm not the biggest fan of PHP (mostly because I learned it on PHP4 and it encouraged bad coding practices back then). However I can't deny how many people use it and how easy it actually is to get started.
Then this happened: https://meta.stackoverflow.com/questions/293930/problematic-...
Stack Exchange websites rank well and they allowed myself and others to provide better answers than they had previously. This leads to more developers being exposed to safer, cleaner ways to solve problems. Which means less terrible code being written from following terrible SO answers and tutorials.
And now things are a lot better. There's still obviously work that needs to be done, but:
- The ecosystem (i.e. tutorials) is improving
- The language is improving drastically
- The community is pushing towards better practices
- Framework developers are taking security seriously
P.S. If you like the direction things are headed, help us make 7.1 even better:
* https://wiki.php.net/rfc/mcrypt-viking-funeral
The Node.js core is particularly bad: It only offers OpenSSL's userspace RNG rather than the operating system's CSPRNG. (That's what window.crypto.getRandomValues() hooks into.) Many of node's deficiencies are tackled by the community, of course. For example: https://www.npmjs.com/package/scrypt-for-humans versus the native PBKDF2 (with no constant-time comparison).
I hear Ruby's SecureRandom still refuses to switch from /dev/random to /dev/urandom as well. Feel free to throw your hat in the ring on that issue.
Historically, C# doesn't have as strong of an open source culture, but if you wanted to make an impact, you could try to nurture it into growing explosively.
But generally: Look at the languages you're already familiar with and see if bad security advice gets more attention from non-experts than from experts. If that's the case, go out of your way to learn how to do things better (if you aren't already well informed) and publish your corrections.
This is a myth. It is adding new features. That does not mean it is improving.
I'm curious what your idea of improving is frankly.
Have you heard the expression, "A chain is only as strong as the weakest link?". So adding strong links after a weak link does not "improve" the strength of the chain.
And that is assuming that these new features added are actually sane. From what I have seen, it is not the case. see here [1]
So you cannot improve php with out making it "not php". And I don't think even if the same people started who are working in the language started from scratch, today, it will be much better than Php. Because they are simply not competent (refer [1]), because if you are competent, and has the vision to see the flaws of the language and the community, you won't go anywhere near it. So I think there is zero chance of Php becoming "sane", let alone nice...
[1] https://np.reddit.com/r/PHP/comments/3j88v4/something_about_...
print 3/2 outputs 1
In Python 3
print(3/2) outputs 1.5
Even a language that is regarded as well designed has unexpected behaviour, it's not necessarily wrong it's just not what the user expects.
They fixed it in 3 but does it's existence in 2 make Python a bad language?
PHP historically did a lot of stuff that a user wouldn't expect but they have started to clean that up, it won't happen overnight but the trend is there, strong and growing.
No such thing as a perfect programming language since everyone has a different definition of perfect.
> it won't happen overnight but the trend is there, strong and growing.
It is just appearance. Just movement, not progress. I just provided you with an example on how new features added follows the same incompetent design, consistant with Php's track record.
>No such thing as a perfect programming language since everyone has a different definition of perfect.
Yea, there is no perfect one, But there is something that is too bad that should not be used. Php is that. It is not going to change.
Can you provide any argued examples? I think not.
I feel this isn't really due to anything inheritely wrong with PHP, but more of it becoming a victim of it's own success. I feel that Node is heading in the same direction too, as more and more people (especially those new to programming) start using it. At least with Node, using third party libraries and frameworks is more common than the 'copy and paste these lines of code' when I learnt PHP 15 years ago :D
Thanks for helping to fix the situation!
PHP hosting is more naturally cheap which is why there are so many cheap PHP hosts out there. It basically boils down to one simple principle: boot up.
With other languages, you have to boot up. You've got to startup the app, load it into RAM and sit there waiting for a connection. With PHP, you can fill up a hard drive with code and it will all run when called as long as the request volume doesn't overload the RAM.
That's the entire reason that PHP hosting is so much cheaper. RAM doesn't get used until a request comes in.
Most performance benchmarks are insignificant for anything involving a single database query too.
It would be. With PHP the language is loaded once regardless of how much code is going to be parsed and executed. The amount loaded matches size of the code base with other languages that have to boot up, which puts a cap on how much low traffic code you can put on a single server.
Last, it really is super cheap to host php, python, ruby, perl, etc sites these days. My hobby site is hosted on app engine, and it pay $0 per month. It's been running for over a year, and gets a fair amount of traffic for a hobby site. Sure, it's subsidized, but I could run the same setup, with a bit more work on my part, on a VPS for $5/month. Yes, php hosting is historically cheaper, but these days that difference is hardly relevant.
Edit:
My hope is that one day that changes, because PHP shouldn't be and isn't the only language in town.
Ruby guys can always do the argument "Our code required 50% more servers, but it increases developer productivity by 50%, and since developers cost more than servers - it is a win".
Has anyone ever really made that argument with PHP? It seems one of the least developer friendly languages possible on a large project. So developer unfriendly + server unfriendly = ?????
The 1 million Badoo saved here is nothing compared to how much more money they made from having these people build their platform.
That is a sad, albeit realistic thought.
It's only unfriendly to those who don't actually know the language.
Where's this better than Wordpress (in functionality, not codebase) project in another language, for example?
Assuming you don't include the cost of converting their 3 million lines of code to a new language and the cost of learning a new language. Of cause they did need to "port" from PHP5 to PHP7, but I feel that it's cheaper than going to a completely different language.
You're not wrong, but the $1M saving is much cheaper bought than the $1.9M. PHP is a weird language though, it's easy to get started with, pretty efficient at this point, and yet somehow people use it to write ugly and sluggish code. I haven't written PHP in a long time, so this may have changed, but it almost like the language is missing ways of structuring the code in a sane way.
Sounds like you haven't touched it in a while - it's quite different from even 8 years ago.
No I am not. I am talking about 1.0 decision. Obviously 10 years down the road with 3 million lines of code you have to compare conversion cost.
The whole mixing of HTML with PHP with SQL is bad form and is widely discouraged. In fact, this code is often seen with usage of mysql_ functions plus manual escaping and interpolation. This is discouraged to the point where that library has been removed.
I'm no evangelist, I prefer other languages, but with these "good parts" so to speak, the language is usable and supportive of good structure. We have had these improvements relatively late in the game but they do appear to come thick and fast nowadays.
There are many warts in the language. Sometimes I'm angered when very good ideas for progress are shot down via committee. There has, however, been improvement since PHP 4.
Anything involving manipulating your data / business logic should not be embedded in a template.
PHP in your html should really be for display/rendering (i.e. looping over a list of Employee records), it should not be fetching those records itself (either by Employee::fetchAll() or especially mysql_query("SELECT * from employee") )
It is very hard to maintain or even reason about a file or make a change or secure exploitable edge cases when the application has html, css, javascript, php, and calls to the database in the same place.
What I meant was mixing them together in the same piece of code. The other comments explain why: its doing too much, its difficult to maintain and reason about so its harder to find bugs, especially serious ones, and difficult to fix them.
That would depends largely on your developers. I know it's weird but PHP developers tend to be really productive in PHP.
If productivity == not writing the same boilerplate over and over, not so much.
Regarding development speed, I don't know much about Ruby but I believe you're conflating speed on Rails (a framework) with Ruby vs PHP (languages). I've also seen RFP respondent development companies assert that PHP development would be faster than compiled languages, despite them having developers with both skillsets available.
No, please show me where I said that? Every program has a developer productivity factor, a library support factor, and yes a runtime speed factor. From everything I know about PHP, while not being a PHP programmer, it has both low productivity AND low performance. Many other languages offer some combo of productivity and performance that are much better.
> The amount of effort described to migrate is orders less than rewriting millions of lines of production code.
Sure, and that is why you should think very carefully about what language to write in BEFORE You write 3 million or even 100 lines of code.
> Regarding development speed, I don't know much about Ruby but I believe you're conflating speed on Rails (a framework) with Ruby vs PHP (languages).
Ruby is rails, there is no ruby without rails, there is no rails without ruby. I see almost nothing being written in straight ruby (why would you).
Funniest thing I've read all day. My last web app didn't use rails at all and I'm actively trying not to use it these days because I don't need most of the features it includes and turns on by default.
This is so.. so wrong. Please take a look here: https://github.com/trending/ruby?since=monthly
You're also forgetting the time period. Many web projects used PHP and built businesses from them because that was the defacto. Web stuff pretty much meant Apache/PHP/MySQL for a time. Now there are better choices, its easy to say better choices should've been made in retrospect. Then again, that's always the case no matter what the less than ideal decision was without the knowledge of today isn't it?
This is so wrong. Regardless of the fact there are very popular Ruby web frameworks out there that are not Rails (Sinatra, Grape), Ruby is immensely useful as a general purpose programming language. I use it daily for all kinds of reasons. I even use it to do my taxes. For anything where speed isn't important, and it often isn't, Ruby is useful.
I know I'm not alone. Martin Fowler, for example, uses Ruby as his personal scripting language. http://martinfowler.com/bliki/OneLanguage.html
You should probably stop, it's quite embarrassing. It's obvious your "everything" is just about "nothing".
I've always wondered on the ROI of saving server cost at the expense of velocity. After all, server cost is for the foreseeable future, but development is just once (yes maintenance, but that is very small if you stop adding features).
Considering the extra baggage and ceremony that comes with most compiled languages, that sounds like a bad proposition, especially given that they ALREADY had 3 millions of lines of PHP working just fine.
Well that's true on a certain scale. There is no way I could have completed some projects with a small team, in a short amount of time if I had to use some like Go or Rust. Ruby,Python or PHP are useful for what they are good at : "front-ends" (on the server). So is Java by the way, but while Java is fast it eats way too much memory while doing nothing. Again, at a certain scale , developer cost VS performance cost is a reality. It's not very useful to boast about chosing Go or compiled language X or Z while your competitor can bring an app into production in 1/8 of the time it takes to develop the solution in PHP.
AFAIK Parse.com spent a year rewriting their Ruby app in Go. In the meantime, there brought no new functionality and then they just closed shop. Sure they got to write a fancy blogpost about how successful they were with Go. so successful they rewrote their app in nodejs when they decided to opensource it.
> So developer unfriendly + server unfriendly
How is PHP developer unfriendly ? It has 15+ years of libraries to do almost anything. You should probably have a look at PHP7 before passing any kind of judgement.
How is it server unfriendly ? PHP is with Java the most deployed language in the world.
Consider programmatically working with email. Nightmarish, despite the number of users.
I'm not sure what you're getting at, Working with email is a nightmare in any language, Go or Rust or language of the moment X included.
It's junior-developer-friendly, but if you want to do something serious, there are better options.
Using a modern framework like Yii2 makes it just as good as any other modern franework in any other dynamic language (rails, django, entity, etc). Yii2 performance also doubles from PHP5.6 to PHP7 for those interested: https://caleblloyd.com/software/yii2-php-7-performance/
Please demonstrate that Yii 2 is as feature-rich as Rails 5. Thanks.
> Modern
Pick one.
But what kills PHP for me (and Rails and Django, for that matter) is the lack of native support for asynchronous operation and poll/epoll.
If you're stuck with one thread per user, then you can't scale beyond a couple thousand concurrent users on one system without latency getting terrible.
You can just do so much more on one server with Node it's not even funny. 70k queries per second with low latency are not uncommon; add in some work and that goes down, of course, but a well written app can have 20k+ queries per second of throughput.
Even PHP 7 with WordPress in cached mode only brags of ~5k queries per second. You could likely do 4x that in Node without trying -- and possibly even without caching.
A fairly large internal php service at work just wasn't working well enough for us (for a lot of reasons), so I decided to try writing a drop in replacement in go (as in, the server would respond to `whatever/endpoint.php`).
It took about a month and a half, but the translation process was actually incredibly straightforward. Like, I automated a bunch of it straightforward. Most of my time was spent writing unit tests where the other codebase didn't have any. Maybe it was just how that php was written (totally frameworkless), but both languages seem to be fond of straight-forward imperative code, and IMO a junior developer could be just as efficient writing go as they could php.
And man I don't even like go.
I've also been looking around for places using PHP7 in production, was glad to hear it went well. While Etsy did it, they have Rasmus. Does anyone else have examples of PHP7 running in production, especially if the development team has written about it?
Our 10 year old inventory/vendor backend that relied on the (now removed) mysql_* calls needed to be shimmed[0].
Our largest benefit was a significant drop (50%+) in memory usage when allocating lots of objects (which we do when pre-fetching ORM relationships), and a significant reduction (40%, 100-200ms) in render latency when rendering many partial product templates.
Edit: One thing that did bite us post-rollout, which didn't show up in development was this php7-fpm bug[1].
[0]: https://github.com/dshafik/php7-mysql-shim [1]: https://github.com/php/php-src/pull/1720
Tons of people love it, but admitting you do is pretty much socially unacceptable.
:)
How did "overall load on the cluster fell below 50 percent thanks to Hyper-Threading technology"? Isn't HT simply exposing a virtual core for every physical core on the CPU?
When an HT CPU is monitored to be at 50%, it's actually at 100% or close to it, so it appears that what fell 50% was the reported values for CPU usage. I don't understand how there's a 50% decrease in load due to "when the load increases to over 50 percent, HT-engines, which aren’t as useful as physical engines, start working"
Oh my. Where do we begin? I guess, using this logic, the language that is just the exact amount of slow to match whatever database workload you have is the optimal language.
Because, as we all know, all parts must be equally balanced, otherwise it's not optimized. So if you need to use 100 mysql servers, you better darned use 100 computing servers of equal computing power, because if you only use 50 you're not balanced.
"if you need to use 100 mysql servers, you better darned use 100 computing servers of equal computing power, because if you only use 50 you're not balanced."
The second does not follow from the first.
"A well designed system is balanced: when the input load increases, all parts of the system take the hit."
"Given this reality, the processing power of the application cluster is arguably the most important factor."
It's just nonsense anyway your look at it, really.
And regardless of language difficulties on your side (which we should try to overlook), the conclusion that app server performance is more important than database server performance is left completely unfounded.
That doesn't follow. You can have a well-designed system where the PHP code only contributes 10% latency and resource usage, in which case the PHP code is clearly not the most important factor.
"This might just indicate you better have 25 app servers vs 100 databases and spend the rest of money proportionally on both clusters."
Exactly. What matters is proportion of latency or resource usage contributed by the PHP code, and that can vary a lot between well-designed systems. The property of being "balanced" doesn't automatically make your PHP application cluster "the most important factor".