it didnt work because people don't search for videos via google usually. They get the video directly from youtube, or was linked by friends virally.

Plus the early days of google (before youtube acquisition), i dont think their video search was as good as the text search anyway (but i might be misremembering of course...).