If you're using a general engine, or even the built-into-site search box of youtube, then your results will be typically poor. The trouble is most of the available types of searches are trying to determine what you mean, rather than keeping things simple and returning results based on a pattern match. Video searching is one of the places where the "what you mean" interpretation yields very poor results.
Additionally, you can use specific "video search" features of general search engines and receive better results than if you used a "built-into-site" search box.
Here's the "quick-link" I use for google video searching in firefox. It's saved as a bookmark, and in the "properties" dialog for the link, I set the "keyword" value to "gv". This means I just type "gv whatever" in the URL/Address bar, and get the desired video search results.
https://encrypted.google.com/search?num=100&safe=off&...
For example: "gv knuth"
https://encrypted.google.com/search?num=100&safe=off&...
Yes, I have different "keywords" set for all the major types of searches I do, and this enables me to get rid of the "Search Box" in the browser.