chadr

I've been working on an open source project along these lines: https://github.com/chadrem/condotto

I'm using it with my small team and overall these types of tools are great for enabling product managers and even operations managers to help prototype, triage, etc all sorts of features/bugs/reports that previously would go directly to the engineers.

Yes

Semi related to this, is it possible to run multiple CPython interpreters in the same process, but with restrictions on shared memory? The idea being that each interpreter would still have its own GIL, each interpreter would have restrictions on shared memory (like sharing immutable structures only through message passing). Note, I'm not a big Python user so if this already exists, has been discussed, etc I am not aware.

You've sold me on it. It is well worth the money considering the time/effort it saves.

How long do they plan on supporting this fork? Hopefully for multiple years? I didn't see it mentioned on the site.

Newer blog post about the surgery:

http://linuxlock.blogspot.it/2012/08/it-started-with-hopeles...

from Wikipedia... "The landing sequence alone requires six vehicle configurations, 76 pyrotechnic devices, the largest supersonic parachute ever built, and more than 500,000 lines of code."

I can't give you an exact answer, but I can tell you this... out of the 224,133 apps I track in the android market, here is the breakdown of how many installs they have as of this morning: http://dl.dropbox.com/u/131/installs.csv

Our artists get to choose the technique that works best for them. A lot of it is done using the computer, but some of it is done by hand and scanned. The coolest bit of art tech is definitely the Wacom Cintiq. It lets the artists draw on top of an LCD panel.

No official Android plans as of yet, but it is something we are thinking about. Also thanks for the compliment on the art. We do all the art in-house and I passed your message on to the team.

It definitely sounds like you wear multiple hats at your job. Shoot me an email... I'm interested in hearing more about what you do.

I think you're missing the point. Many developers don't know how to develop systems that will work well in production. At the same time, many sysadmins know only about the OS, hardware, and network. Great sysadmins are learning to combine the two skill sets. It's clearly a hybrid role and not a case of one replacing the other.

High quality sysadmins are evolving into what is called the devops role. Trouble shooting, scaling, architecting, and automating production systems are just a few areas where devops people shine. The cloud just provides them another set of tools to work with. It also frees them from dealing with the annoying/repetitive tasks (spinning a CD to install the OS, plugging in the network cables, etc) and allows them to focus on improving the application. A number of devops people I know can easily transition into developer roles when required. Summary: a great sysadmin should know how to code and does so in order to improve the app.

Do you routinely edit the source code of your web server? Just curious since I've never needed to do that.

Why don't you trust Passenger? I'm severing about 6 million page views/day with it and it has been extremely reliable and easy to work with.

Can you tell me your hardware specs and CPU usage? Just curious since I'm considering mongo for some of our future games.

Pingdom for site uptime & paging, munin for capacity planning and performance monitoring, and monit for process monitoring.

It appears to follow symlinks if I uncheck 'Use git-grep'. Unfortunately, if it finds a file (by following a symlink) it seems to always open the file in a new window instead of a tab.

Any chance nice_find can have an option for following symlinks?

Have you tried contacting the Passenger developers about this? I won't be surprised if they come out with a fix for this.

It seems to me like the Passenger guys could easily add an option so that on a "touch tmp/restart.txt" a set of new worker processes is started before the old ones are killed off. I imagine this would make this slowness a thing of the past. For the record, my apps experiencing this momentary queueing and slowness on restart (5 seconds max).

How horizontally have you scaled your systems in production? The homepage mentions 2 servers. I'm wondering if you are using it in production with more than 2.

I'm currently using Tokyo Tyrant with Rails and the Memcached client (in production). It works great for my needs though I don't need to story binary data. Overall, TT has been extremely fast and stable. The dual master replication is a nice bonus.

Having an ordered hash is ruby is very useful. I've been using ActiveSupport::OrderedHash with ruby 1.8.7.

One last thing... mysqldump has the following option:

-w, --where=name Dump only selected records; QUOTES mandatory!

You can probably use this to backup just the rows you are interested in and then delete them once you are sure the are properly archived.

Here is what I do with mysql 5.1 and my innodb tables on Linux...

mysqldump --single-transaction --skip-lock-tables --all-databases | gzip > "/some/directory/backup-`hostname`-hourly-`date +%H`.sql.gz"

These two options are important: --single-transaction and --skip-lock-tables. At least in mysql 5.1, mysqldump defaults to locking each table before dumping it (not good if you have a busy app and large db).

I run the above in a cron once per hour. It gives me 24 hours worth of hourly backups. I then use rsnapshot (http://www.rsnapshot.org/) to store daily, weekly, and monthly backups offsite.

I also run my backups with a low priority (nice -n 19) so that my application gets higher priority than the backup script.

Do you have any more info on the corruption? I have found bdb to be very reliable. I used it as the backend storage engine for Sun's LDAP server product.

Perhaps memcachedb would be better for this scenario (vs regular memcached) since it is persistent. It's memcached with berkley db.

Which tool(s) do you recommend that do this?

If you are starting out with linode you should be aware of the backup policy. Basically you have to roll your own. Let me know if you have any linode questions. I've been using them for about a year now. I think backups are the only feature linode is missing (compared to slicehost).