Findjango, A Week of Progress
lethain.com
lethain.com
Are you sure that we need a vertical search engine? It seems to me if I put Django + other words into Google, I have myself a vertical Django search engine.
Do you have some examples where Google fails?
1. "Perfect answers" that relate to the topic at hand. For example, Findjango has an index of Django's documentation and lists related documentation at the top (above search results). For a large percentage of searches, those links will answer the search immediately (also, the links deeplink to the correct location on the documentation page, as opposed to simply dropping you at the page's beginning).
2. Because of a narrow scope, it is possible for a vertical search to be more current.
3. Because of the narrow scope, it is possible to apply custom relevancy algorithms suited to the specific niche which may outperform generic relevancy algorithms. Findjango has begun traveling this route, but is clearly not there yet.
4. The ability to collect and display targeted metadata. Although I'm not taking advantage of it yet, Findjango knows the number of followers and last commit date for each GitHub repository, and can display them in-line in the SERP. For a targeted audience this can be worthwhile to add, whereas for a main-stream search it would represented a negligible usecase.
5. I believe that vertical search engines can be monetized much more effectively--and while doing so create more added value for the user--than a generic search engine. Certainly I have no numbers to back this up as of yet.
In the end, my argument is that a vertical search can outperform a general search engine within its vertical. It is certainly not my argument that it will inherently outperform, and certainly Findjango lacks the breadth of content to compete thus far.
This is a long-term project, certainly not an overnight road to success.
Looking at your points above, in my experience, Google already provides "perfect answers," which often link the docs. I have no problem with how current the results are - they often include recent bug reports. And I think your custom relevancy algorithms are nice, but I think you'll find that Google's results are so good, because they crawl so many pages. Their algorithms aren't generic - they are highly advanced indexing and linguistic analysis, mixed with machine learning to tune the thing.
I think that if you are trying to return the most relevant web pages for a query, it's going to be a long uphill battle to beat Google. I have never thought to myself - "Google isn't doing a good job, I wish there were a Django search engine." I can always answer my Django questions using Google.
When you are looking to make money off of something too, it's important to be a painkiller, and not a vitamin. And right now, I don't have any Django search pain.
Good luck with the thing though. I'll check back later to see how you are doing. I encourage you not to try and compete with Google though, that is to get in the game of "returning lists of webpages." If you want to make a good Django search, you'll need to find a different niche. Your point about metadata sounds interesting.