So this is very interesting. It's not exactly a strength estimator (note that Microsoft isn't calling it one, Bruce Schneier is) because it doesn't really retroactively evaluate the whole password. For example, "Entrolax" passes the test even though it's the name of a drug and therefore likely in dictionaries used for attacks. It passes though because the letter sequence is sufficiently strange. Similarly, the relatively unobscure "omnipresent" gets 4 checks for the same reason.

This also seems to discourage passphrases because even though any good passphrase (such as "correct horse battery staple") will get 5 checks easily, you have to get through a lot of Xs to get to those checks, each one suggesting that what you are doing isn't a good idea.

The tool does however suggest an interesting alternative to passphrases: Only use as many letters of each word as are unpredictable. In other words, only type each word until the next letter is predictable. For "correct horse battery staple" this gives "c h bat stap". I really have no idea if this is a good password, but it does get 12 out of 12 checks and only one less check than the full phrase.

As with any password evaluation, there are patterns it doesn't know how to check. It takes a long time to figure out the pattern "a1 b2 c3 d4 e5" and still gives a check for each space. It doesn't detect spiral patterns on the keyboard at all ("gtrfvbnhy" on a qwerty keyboard for example).

Still, it will be interesting to see what they can do when they get more data and think through some of these issues.

EDIT

You can also use this to choose letters that make sense but aren't in the prediction. I came up with the password "pitabakes" (pita bakes), which is obviously pretty easy to remember and presumably fairly weak simply by deliberating taking something which seemed likely but not in the top 3.

One also has to wonder whether deliberately being unpredictable somehow becomes predictable. This tool, to some extent, suffers from the fundamental mistake of gamblers: seeing patterns where there aren't any. See jerf's post about getting X marks in a LastPass-generated password. If you tweak your random passwords to not be predicted by this engine, are you actually making them somehow more predictable?

EDIT AGAIN WITH MORE NEW FINDINGS

You can also get high numbers of checks (and a relatively high ratio of checks to X marks) using common dictionary words if you switch words in the middle. So, taking the words "grape", "pasta" and "toffee", using "grapastoffee" which gets a 50/50 ratio of checks to X marks (6 of each). This actually seems like a pretty good way to generate hard to guess passwords. "Pen", "noodle" and "duck" giving "penooduck" does even better, 7 out of 9 checks.