I've actually been building something like this: a desktop app that basically records everything you see and hear, and makes it searchable. Maybe we could team up: my email is govind <dot> gnanakumar <at> outlook <dot> com.