Building Sourcegraph - A large-scale code search engine in Go (2014) | Hacker News Reader

Building Sourcegraph - A large-scale code search engine in Go (2014)

text.sourcegraph.com

82 pointsphilip120923 comments

This was my talk. Thanks for posting it! Happy to answer questions.

To try out Sourcegraph, sign up at https://sourcegraph.com and it'll take you on the tour.

Also, I'll shamelessly plug our Chrome extension, which "Sourcegraphifies" GitHub (adding jump-to-def and cross-references): https://chrome.google.com/webstore/detail/sourcegraph-for-gi.... It's the #1 Chrome extension for GitHub.

Finally, check out https://sourcegraph.com/beta to get early access to some new languages and super neat features we're adding soon.

Extension is cool. I know that hackernews comment is not the most appropriate place for bug report, but pressing DAG in https://github.com/apache/incubator-airflow/blob/master/airf... or set_downstream in https://github.com/apache/incubator-airflow#L37 does not work. Goto definition works with both locally with emacs-ycmd.

Sourcegraph CEO here. Filed! Thanks. I've taken bug reports and feature requests from airplane seatmates and when walking down the sidewalk in SF, so this is actually not a bad place to post them. :)

Any plans to make a Safari extension?

Thank you.

Yep, we plan to release it for Safari as well. I don't have a date, but it's planned.

I do not understand these free tiers + steep pricing. This is like you are looking to bank on those that really really need your stuff..

But I do not want to invest myself in your ecosystem just to see it die eventually. I want to pay you because I do not want the service to die. Do you plan to serve ads in the free tier ? Why not have the pricing more dynamic ?

Sourcegraph CEO here. We have a generous free tier that lets you see the value of the product (and get a lot of use out of it). Happy to help you get your company to see the value, too, if that's what you're interested in. If you want to chat further, my email is sqs@sourcegraph.com.

I find Sourcegraph really interesting as a different approach to code indexing. Opengrok, Krugle, BitBucket Server and my own poor attempt searchcode and the downloadable server version simply index the source code as is which is to just index everything thats text (with some smarts on top usually). By comparison Sourchgraph builds parse trees and attempts to understand it.

I am not sure as to which approach is better for the average user, but I do think that if you are using a language currently supported you should give it a try and compare to one of the afore mentioned solutions.

Sourcegraph CEO here. Thanks for the kind words! Ben here is just being modest...searchcode is fantastic. It has come in handy in many crucial moments when I couldn't find the code I needed anywhere else. Here's a link for those who aren't familiar with it: https://searchcode.com/.

Looks great. I've been building almost the exact same idea for fun as a side project, but initially focused on deep analysis of TypeScript and ES2015+ specifically. Runs in Docker as well, spinning up containers to do analysis work, but written in TypeScript and run via node.

Definitely going to be following the blog. Thanks for sharing!

Very cool. Would love to chat and see if we can push this stuff forward together. I can't seem to find your email address. Could you please email me at sqs@sourcegraph.com?

Is there a way of "git cloning" the https://sourcegraph.com/sourcegraph.com/sourcegraph/sourcegr...? It is linked in the article. It does not seem to be at https://github.com/sourcegraph .

    apt-get install docker protobuf-compiler
    mkdir foo
    cd foo
    export GOPATH=`pwd`
    export PATH=${PATH}:`pwd`/bin
    mkdir bin
    mkdir -p src/sourcegraph.com/sourcegraph
    cd src/sourcegraph.com/sourcegraph
    git clone https://sourcegraph.com/sourcegraph.com/sourcegraph/sourcegraph sourcegraph
    cd sourcegraph.com/sourcegraph/sourcegraph
    make
    make serve-dev

good luck.

It doesn't actually run for me, just crashes, but you'll get kind of... most of the way there with that.

(oh, you probably dont need protobuf unless you run `make generate`)

Sourcegraph CEO here. We're going to do a big source release soon. Then this should all be easier. We're growing quickly and pushing out a lot of product releases soon and have been busy on those. Email is in my profile if you really want to figure it out, and I can direct you to the right person to help.

How can one add other public repositories not hosted on github (for instance from the Eclipse foundation)?

Sourcegraph CEO here. Right now, just email us (hi@sourcegraph.com) with the Git clone URLs and we'll add them. Soon, you'll be able to add them yourself. Thanks!

Ah great, I'll do that! Thx.

Sorry for my naive question. What are the features this is adding over GitHub's source search?

Sourcegraph CEO here. Unlike GitHub, Sourcegraph actually understands the code like your IDE. Unlike your IDE, it has all the code on GitHub and some other repo hosts already loaded up and built. (We're working on adding more languages beyond Go, etc.; check https://sourcegraph.com/beta for early access.)

Try out jumping to definition and seeing hovers at https://sourcegraph.com/github.com/gorilla/mux/-/blob/mux.go.... This helps you understand code that you're browsing.

Check out global usage examples and refs of any function, such as those at https://sourcegraph.com/github.com/golang/go/-/info/GoPackag.... You can get to this page by right-clicking on tokens in the code view above. This helps you quickly figure out how to use a piece of code and see best practices.

Finally, the code search is also semantic and is ranked by popularity. Try it out at https://sourcegraph.com/search?q=new+http+request.

Code analysis and search at this scale is tough, but we're going to do it right, because it'll help developers learn more quickly, write better code, and reuse more existing stuff. (We're not totally there yet, but if you want to help by adding language support or joining our team, email me at sqs@sourcegraph.com.)

Thanks for the details. Is this available as a self hosted application or enterprise edition? Useful for enterprises.

Hey there,

James from Sourcegraph here. Right now we don't offer a self-hosted solution and won't have one for the foreseeable future. Please email me your contact details so I can let you know when we do though.

james@sourcegraph.com

Thanks, James

This seems quite limited. I went to the site and it seems to work only on a few languages. Honestly, the best code search out there is GitHub. I learn so much on there and it works across every language.

Right now I am coding in Obj-C, Elixir, Ruby, and doing web development in React. That is my day to day so this seems quite limited.

Sourcegraph CEO here. Support for more languages is our #1 request by far. We're going to have more language support very soon. If you want to get early access, head over to https://sourcegraph.com/beta and we'll be in touch.

Reply on news.ycombinator.com