Wikipedia (English) starting to migrate to MariaDB | Hacker News Reader

Wikipedia (English) starting to migrate to MariaDB

lists.wikimedia.org

139 pointsneya36 comments

    The main goal of migrating to MariaDB is not performance driven.  More so,
    I think it's in WMF's and the open source communities interest to coalesce
    around the MariaDB Foundation as the best route to ensuring a truly open
    and well supported future for mysql derived database technology.
    Performance gains along the way are icing on the cake.

Several months ago I noticed Wikipedia's Ganglia (cluster performance monitoring software) is publicly accessible. Check out db59 (mentioned in the note) amongst the other servers:

http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&...

If you want to see a bunch of other cool public Ganglia instances you can do a search something like:

https://www.google.com/search?q=%22show+only+nodes+matching%...

Any reason why the load on db59 is so high? Are they trying to stress test it?

I couldn't say the reason, but the I/O for db59 [http://ganglia.wikimedia.org/latest/graph_all_periods.php?h=...] is much higher than the other servers as well.

Also, despite the switch on db59 happening a few days ago, there doesn't seem to be a significant change in overall CPU usage: http://ganglia.wikimedia.org/latest/graph.php?r=month&z=...

Hopefully this will get more people to adopt mariadb and be free of the legal and upgrade uncertainties of mysql. I installed it a few weeks ago and the result is -exactly- like mysql.

http://blog.mariadb.org/mariadb-foundation-to-safeguard-lead...

How does MariaDB compare to Percona?

It's a drop in replacement for MySQL, you can use the _storage engines_ that Percona provide such as ExtraDB so maybe it shouldn't differ at all?

TIL MariaDB is to MySQL what Jenkins is to Hudson.

How long until there is something similar for Java? Let's play name the products that need to be saved from Oracle.

Arguably the other JVM-hosted languages are exactly this.

Agreed, but I think s/he meant 'JVM' rather than 'Java'.

I didn't. But I should have! Thank you. I so often forget those "little" details.

How about 'elCaro' ?

open jdk?

Wikipedia seems like the perfect system for a document store like CouchDB. I would be curious to know if they examined it and, if so, found it lacking.

MediaWiki uses a fairly typical mostly-normalized relational design. You couldn't easily port it to a document store.

http://upload.wikimedia.org/wikipedia/commons/3/33/MediaWiki...

Also, most MediaWiki installs use plugins, and those plugins often create their own database tables. So RDBMS is pretty much baked into the MediaWiki culture.

However, MediaWiki is fairly well abstracted from any particular RDBMS implementation. Everything from creating tables to making a join is done through an API.

MariaDB is a (almost) drop-in replacement for MySQL, CouchDB is a totally different concept that would require nearly a complete re-write of their backend. I doubt they considered it.

My guess is that they have been using MySQL for a long, long time (possibly since the beginning)m, and see no need to migrate to something different. MariaDB is API-compatible with MySQL, so this migration will not involve much changes at the client side; it's mostly thorough testing that needs to be done.

So, there's some legacy here. 10 years ago, there wasn't anything really like CouchDB.

Document DB's aren't an invention of the past 10 years. SAS was invented in the 60's, Lotus Notes in the 80's. XML Databases were all the rage during the dotcom bubble.

The difference today is that open source projects have made document stores and 'nosql' technologies more accessible.

In case anyone else was wondering about InnoDB like I was:

http://en.wikibooks.org/wiki/MariaDB/Storage_Engines

MariaDB has XtraDB, which is a drop-in replacement for InnoDB; I'm not sure if it's encumbered or not, given it's Percona's fork of InnoDB.

(Does anyone have any experience using XtraDB that they can relate?)

Which storage engine is Wikipedia using (MyISAM, XtraDB or Aria)?

Note that unless your mysql server has more than 4 (physical) cores, you aren't going to see any performance improvements from any variation/fork of mysql. 8 cores is where things start to diverge under load.

This website has some benchmarks to prove it: http://vbtechsupport.com/606/ http://vbtechsupport.com/657/2/

Also http://www.mysqlperformanceblog.com/2012/02/18/mariadb-5-3-4...

But there are other reasons to look at mariadb/percona.

What about MariaDB5.5 and MariaDB10.0 performance compared to 5.3?

Why do they care about whether the technology is derived from MySQL or not? Postgres makes way more sense for pretty much every application out there now. What am I missing?

They aren't starting from scratch. Migrating your database is a complex and risky project, and not the kind of thing you do unless you have a very clear, very pressing reason, like horrible performance and stability issues, or saving tens of millions of dollars in licenses. Migrating to MariaDB is more like upgrading a minor version, with minimal risk and zero downtime. And they have a lot of experience in MySQL, almost all of it transferable to MariaDB.

Which database system Wikipedia is currently using?

MariaDB is a fork of MySQL by Monty (original MySQL author)

MySQL

Well, MySQL plus a text store.

MediaWiki, the open source software that Wikipedia uses, now officially supports MariaDB.

We're also using OpenStack Swift for media storage these days.

I couldn't see a way to aggregate it in the ganglia linked.. could you share how many mysql (maria) queries per second you guys are averaging?

I should clarify: I used to work there but I sometimes still say "we".

I'm not a ganglia wizard - maybe this comes close? Although it's just all selects. I think.

http://ganglia.wikimedia.org/latest/graph_all_periods.php?ti...

You might try asking on wikitech-l, the same mailing list that the original post comes from.

I use MariaDB on my Mac for most things that require MySQL. It's a drop in replacement for most things. Is there a special edge case with media wiki they had to solve or is it now just formally announced as being supported?

So, it's pretty much a drop-in replacement. But as with everything, time m ust be spent testing and so on before rolling it out to production. I believe a transition to MariaDB has been planned for a while now.

I remember I had problems with insert statements when previously with mysql inserts without default values worked but MariaDB was strict in this case and these inserts failed...

Reply on news.ycombinator.com