Hacker News Reader

jividon MySQL infrastructure testing automation at GitHub

Super interesting post. Would love to read more detail about their backup and restore infrastructure.

If Tom and/or Shlomi are reading this: you mention taking multiple logical backups per day. What benefit does this bring versus just having one per day and doing a point-in-time restore using binlogs? Is this just a tradeoff between time taken for a restore and storage you're willing to dedicate to backups?

Disclaimer: I work on Facebook's MySQL backup and restore system (https://code.facebook.com/posts/1007323976059780/continuous-...)

jividon 'Check Your Backups Work' Day

Author here.

We use it because it works well for us. We've put a lot of work into making MySQL scale for us to the point where it's a very well supported system and one of the main choices for a lot of storage decisions.

We even use MySQL as a queue for Facebook Messenger. More details about this:

https://www.youtube.com/watch?v=eADBCKKf8PA

https://code.facebook.com/posts/820258981365363/building-mob...

jividon Continuous MySQL backup validation: Restoring backups

In addition to what evanelias said, a logical dump also means we can load it into a MySQL instance running a different storage engine as well. In our case, it allows us to take a mysqldump from an InnoDB instance and load it into a MyRocks instance if we wish.

jividon Continuous MySQL backup validation: Restoring backups

Instance size varies a lot because we've got a lot of different MySQL workloads, some with very different configurations.

Remember we're backing up each database separately though, not the entire MySQL instance at once. Each database's backup is in a separate file.

jividon Continuous MySQL backup validation: Restoring backups

We run a single mysqldump with --single-transaction for each database on the server, nothing special.

jividon Continuous MySQL backup validation: Restoring backups

We take logical backups, not physical, and mysqldump is the best option for that. Having logical backups means we can do logical diffs as well (see https://youtu.be/Fe2oLZ4CWD4?t=951), and we've added table checksum support to mysqldump in our branch of MySQL (https://github.com/facebook/mysql-5.6/commit/54acbbf915935a0...)

jividon Continuous MySQL backup validation: Restoring backups

There's a few different ways to verify, a few of them involving stopping replication, like you pointed out. These can also sometimes be quite expensive, so depending on the type of verification required, the verification method can be tuned.

We also implemented table checksums inside of mysqldump, allowing us to dump out the restored data and compare checksums as well, if required. https://github.com/facebook/mysql-5.6/commit/54acbbf915935a0...

jividon Continuous MySQL backup validation: Restoring backups

> Do you diff against the same base, or create an incremental chain? How many diffs do you take in between recapturing a full image? At $DAYJOB we always take full backups into a fast in-house deduplicating store.

We always diff against the same base and have 5 days in between subsequent full dumps. The number of days just comes from a trade off between space occupied by the backups and time it takes to generate them.

> Is there no better way to handling this than polling?

There's definitely different ways to approach this, we find polling works well for us. We also use the same database for crash recovery, so doing the assignments through it serves both purposes.

> Presumably you can only get this parallelism by disabling FK integrity. Is it re-enabled in the following VERIFY stage?

I'm not sure what you mean by parallelism through disabling FK integrity. Splitting the backup into its tables means we can restore a subset of tables instead of the entire backup. This allows us to load individual tables concurrently, but also not have to wait to load a massive database if all we need is a few small tables.

jividon Microsoft Ships Python Code in 1996 (2009)

Facebook has a very extensive Python codebase as well. 21% of all backend code is Python:

https://code.facebook.com/posts/1040181199381023/python-in-p...

jividon Origami – Design prototyping app by Facebook

It works on top of Quartz Composer, which is OSX only I believe.

jividon Listen to Wikipedia

Those addresses are websocket servers that seem to publish wikipedia changes. The app is just subscribing to the changes by opening a websocket connection to the appropriate server.

Looking at their other repos and the addresses you linked to, looks like the project for publishing the changes is at https://github.com/hatnote/wikimon, which is in turn getting changes from an IRC feed at https://meta.wikimedia.org/wiki/Research:Data#IRC_Feeds

jividon Slack was hacked

> If you have not been explicitly informed by us in a separate communication that we detected suspicious activity involving your Slack account, we are very confident that there was no unauthorized access to any of your team data (such as messages or files)

jividon Sparklines for your shell

This could be because Github's having issues with serving archives. See https://status.github.com/

EDIT: Should be working now

jividon Show HN: The Markdown Resume

Out of curiosity, what do you use as an alternative to Word if you do have to open something in a .doc/.docx format? Surely just saying "I don't have Word because I'm a developer" can't work in every situation?

jividon Show HN: Grasswire – an Internet newsroom curated and fact-checked by everyone

Instead of pausing the feed, maybe a simple 'Load X new tweets' banner at the top of the feed would do a better job? Twitter actually does exactly this when loading new tweets on the feed.

jividon Passwords in plain text

The keypair AWS generates can only be downloaded once, at the time of instance creation. Beyond that, they expect you to be in possession of the keypair when launching another instance that uses the same keypair. If you happen to lose the file, you're basically out of luck.

So to directly address your concern, you can't download the keypair at any point in time, it's just a one time thing. To me that seems much more secure than emailing out a root password and enabling password authentication by default.

jividon Passwords in plain text

I am a Digital Ocean customer and the only password they've ever emailed me is the root password for the server I just bought. Arguably this isn't as safe as AWS' process of making you download a kaypair and only letting you login with that. However, VPS owners should get in the habit of logging on to any server they buy and immediately disabling password auth and root login via SSH, which helps negate the root password being sent over email issue to a certain extent.

jividon Runnable – jsfiddle for everything

There does seem to be a Python option below the fold.

jividon Teenagers say goodbye to Facebook and hello to messenger apps

Correct me if I'm wrong, but aren't Kik Cards exactly what you're looking for?

jividon Age of Internet Empires: One Map With Each Country's Favorite Website

Google would be defined as google.com (or the appropriate geo-specific domain) since this map is specifically for websites, not companies owning the most popular products .

As for what would happen if they did group by company? I'm guessing that Google would be #1 pretty much everywhere, save maybe Russia and China.

jividon iOS 7 UI Stencil for Omnigraffle

I'm a little confused by the "Main action" and "Cancel" buttons. Shouldn't the Cancel button have the red text and not the other way around?

jividon Tell HN: Startup school acceptance emails are out

I'm down to split the cost of a zipcar as well! Will be going from Lower Haight

jividon I left Google, and lived to happily tell about it

Does it bother anybody else that 'back' and 'up' are used as antonyms here?

jividon Facebook Premium

Facebook themselves use Facebook pretty heavily for internal communication.

jividon Show HN: Floobits – Remote pair programming done right

This looks great, good job! A couple of things:

1. I renamed the default FLOOBITS_README.md to teach.py, but the path at the top of the page still showed the README filename. teach.py showed up in the left nav, so I don't really know what was going on there.

2. When I was loading one of the workspaces under https://floobits.com/u/ggreer, clicking on the files in the left nav took really long to load, about 8-10 seconds. I almost thought that all the files were blank at first and were there just to show the nested directory structure.

Apart from that, I think this is really cool! I'm definitely going to be using it over the next few days.

jividon DuckDuckGo Direct queries per day (28 day average)

Why not just create a URL alias? Firefox and Chrome both have pretty good add-ons for creating aliases.

jividon Show HN: My new homepage

This looks pretty sweet! I'd add tooltips or captions for the icons at the bottom of the page. Some of them are instantly recognizable (github, instagram, twitter etc.) but it's not really clear what the rest of them are.

jividon Show HN: Dissipate - Free ephemeral cloud storage

So ridiculously simple, I love it!

That being said, would it be possible to add an option to only count a successful download as a download attempt? Right now, if I just hit the download button, the count gets decremented.

jividon Click on a few dots and our program will guess your age

Despite being about 10 years off my actual age, I think this is pretty cool. I wonder if there's any other information they're taking apart from speed between clicks? Geo-location might help to narrow down the range of possible ages.

jividon Introducing Vdio

This looks really neat! However, I'm going to echo a lot of other people on this thread and ask if we really need another pay-to-watch video service? Amazon Prime, iTunes and Netflix are already very big players in the market, and though I'd love for Rdio/Vdio to race ahead of the competition, I don't think a pay-per-item business model is the way to do it. A subscription service with unlimited streaming is what got me hooked on Rdio in the first place; I hope they are able to apply the same model to Vdio soon.

That being said, I immediately loved the interface for Vdio. I wish they had the grey/yellow interface for Rdio also, looks a whole lot better than the Blue on White they have right now.