way to go matt. Did you say 1.5G? For those reading & who want a summary the problem is on 'entity resolution'.
'... A common problem that we face is that there are many people with the same name. Given that, how do we distinguish a document about Michael Jackson the singer from Michael Jackson the football player ...'