Hacker News Reader

In my last job I wrote several automated backup systems for MySQL servers, and they continue to run as part of a SaaS hosting service. Holy cow, I'm an expert! Thanks for reminding me! ;)

Of course, one never feels like an expert, because part of being an expert is that one knows the names of some real experts, who are better than you. In this case, some real experts in MySQL backup work at Percona. Hire them. Alternatively, buy the O'Reilly book High Performance MySQL, which is partly the work of Percona's founders and has chapters about backups.

(If you are the foremost expert in a field, of course, you probably don't feel like an expert either, because you're at the frontier, where everything is confused and all the other experts are most lost than you. I don't understand this field! The references are irrelevant and confused and make no sense! I don't know how to make progress and I can't find anyone else who does! I must be really stupid!)

Mysql Backups:

Ghetto solution is to find a service that will allow you to rsync files to them and take care of offsite backups etc themselves. Once you have done that it's a simple case of setting up mysqldump to create timestamped .sql files and setting that up on a cron job to rsync to the remote server via SSH (you need to generate a key pair and copy the public key to the other end).

If you are using characters from weird character sets you definitely want to make sure all of your unicode settings are correct which may take some trial and error and I've never entirely trusted mysql in this regard.

If you are paranoid you should have a separate job that automatically downloads and restores your backups after a period of time and runs a few test queries on the data. Of course the hard part is knowing what the results of these queries should be.

Of course the problem here is that this will chew up increasing amounts of your remote storage over time, so it depends if you can afford to delete backups of over a certain age.

You can also enable binary logging in mysqld and do incremental backups by writing the binary log to the remote host at certain intervals (I think this only works properly with InnoDB databases), though I have no practical experience doing this because frankly I've never needed to. Nightly dumps were always good enough so my boss wouldn't want me spending any more time on it.

That's the crux of the problem with working for a non tech company, most of the time once the solution is "ok" it's time to move onto something else so you never get a chance to really kick ass which is what you need to do to justify a high rate.

These are really commodity skills though, you could have figured all of this out yourself by reading a few blogs (that's basically all I did). Alternatively if you had posted "Mysql backup solution needed" on eLance you would have probably had at least 20 offers to do the whole thing for $20 or so.

you could have figured all of this out yourself by reading a few blogs (that's basically all I did)

Could have. Didn't. Was too busy building my businesses to care to do the research. Many people are like me. Would listening to Jiggy teach me about MySQL in a nice, packaged, Jiggy-does-the-work-so-I-don't-have to format provide value? Yes, clearly. You already quintupled what I know about the subject with an HN post, think of how entertaining talking for an hour would be.

These are really commodity skills though... do the whole thing for $20 or so.

You are no longer a commodity after you've successfully convinced me that you are the expert who can e.g. listen to my particularized needs and tell me a solution which will work for them. In fact, even if I were non-technical and stark-raving insane and thought that $20 was a reasonable price to pay for this, the fact of having read the above would convince me that eLancelot wouldn't have his uni-codes sorted on the cronSyncServer or whatever that was, so I should entrust the future of my business to the guy who seems to know what he's talking about who is quoting numbers closer to what I pay for e.g. insurance than what I pay for toilet paper.

Something about this exchange reminds me of the old Monty Python "Motor Insurance" sketch:

http://www.youtube.com/watch?v=kO2R_DDZPCM

Well, Reverend Morrison, in your policy… [pause to remove wadded-up document from the inside pocket of an old tweed jacket] …in your policy it states quite clearly that no claim you make will be paid. You see, you unfortunately plumped for our "never-pay" policy, which if you never claim is very worthwhile!

(Incidentally, it's fun how well this sketch holds up today if you imagine that Michael Palin's character is playing the role of the Internet.)

Anyway, the point is that it's not buying the insurance that's hard: It's ensuring that you can collect when the emergency happens. The same is true for backups. When you try to restore the backup, and you can't find the guy who set it up in the first place, and the second guy you hire for $20 from eLance tells you that the first guy messed up the Unicorn settings or whatever, or that his script broke four months ago and nobody noticed, now you're screwed, and you don't even know which of your two $20 minions made the mistake.

How much will someone pay, as a monthly retainer, to avoid that nightmare? Well, how much is their data worth? How quickly will they want to restore it when disaster strikes, and do they already have a full-time employee with a pager who knows how to perform the restore?

As a side note to this - being a generalist can be valuable too, because you can see cross-overs between disciplines.

My entire online presence is pretty much built on this - being knowledgeable about storytelling and the birth of the 3D gaming world at the same time led to me leaping in whole-heartedly at the inception of "Machinima", the technique of using 3D gaming technology to make films. There are a lot of people who are better at making films than I am - but very few of them were also competent at editing Quake levels.

But more trivially, there are dozens of times when you can cross over your skills to achieve results a specialist couldn't. Patrick has referred to one of his in the past - if you're knowledgeable about both advertising best practise and Web programming, you can create landing pages which are generated programatically to appeal to the inbound traffic from your PPC ads.

That's a nice bit of synergy which I believe has netted him a non-trivial amount of money in the past. If you're a programmer and an ad guy, it probably seems like an obvious thing to do - but if you're just one or the other, I can assure you it's not.

Look at two fields you know, preferably both valuable ones, and see if you can figure out things you can teach from your knowledge of both that most people wouldn't be able to.

(Another more trivial example: I've recently done some consulting work on PPC ads for a Pilates instructor friend of mine, considerably assisted by actually knowing a reasonable amount about Pilates. There are a lot of PPC ad people out there. There are a lot of people who know Pilates out there. The number of people in the intersection of the two sets, however, is small enough that I'm considering spinning the area out as a specialised consultancy.)