GitHub switches to Rust based Blackbird code search
theregister.com
theregister.com
1. https://github.com/sourcegraph/sourcegraph/blob/main/doc/dev...
HN discussion from yesterday: https://news.ycombinator.com/item?id=35863175
They are trying to make a point about indexes, but it is overshadowed by the unit mismatch (distributed system QPS vs what sounds like single-machine ripgrep pace)
[1] https://github.blog/2023-02-06-the-technology-behind-githubs...
Sometimes both sides win.
But I think folks dramatically undervalue what GitHub(Lab) brings, and how expensive it is to host such a system for everyone.
I would describe the relationship as mutualistic: both parties benefit, but not necessarily proportionally. And they don't need to benefit proportionally, but there's also value in highlighting the disproportionality: it behooves the OSS community to remember that extremely valuable ecosystems like GitHub and GitLab derive a significant amount of their value from the continued (and mostly passive) goodwill of the community.
I would wager that the cumulative revenues of all third-party services (CircleCI, Sourcegraph etc.) that are part of the GitHub ecosystem also far exceed GitHub, value and revenues that would unlikely exist without GH (or be at an earlier phase of their evolution).
I'd even go so far as to say the value created by pull requests alone exceeds $1bln annually, for the many commercial and open source teams that use it to get work done.
The users are trapped. They've made a time (and some a financial) investment into Github, and to pack up bow and go elsewhere would kill the communities they grew there.
Github owes FOSS everything for the community and buy-in fostered there, and would be nothing without it.
This is also why I think it's unbelievably lucky we had luminaries like RMS in the beginning. To think of software freedom, what a concept! Legend!
Meanwhile everyone else is constantly trying to turn everything into a transactional interaction. "what are you going to give me for my code? How are you going to pay your debts?"
[1] I worked there
I was just posting to say Microsoft have (had?) a perfectly nice search engine many years ago. Either GitHub folks didn't want to use it, it's not good (I think it was good, but maybe it's not suitable, or maybe it isn't maintained anymore shrug) or Microsoft butchered it too much over the years.
Is this related?
"about twice as fast", yeah twice as fast but twice less effective.. what's the point
Feb 23, 2020 (https://grep.app) Show HN: Search code in GitHub repos using regular expressions https://news.ycombinator.com/item?id=22396824
Dec 8, 2021 Improving GitHub Code Search https://news.ycombinator.com/item?id=29487237
Q: How do you know if something is written in Rust?
A: The author/headline will tell you.
If it’s written in Rust, the tooling (Cargo; the Rust language server; etc) make it much easier than most other languages to jump in and fix something or add a new feature. I’ve contributed to projects written in Ruby, Python, Java, C#, C++, Haskell and many others, and every Rust project I’ve ever touched has been much easier/quicker to go from `git clone` to PR submitted.
That may not be appealing to people who never fix or extend the software they use; for me, as someone who finds bugs and deficiencies in almost everything I lay eyes on, knowing that a piece of software is written in a language that is the least likely to frustrate me and suck up my time when I inevitably have to dive in and tweak something is a huge competitive selling point over similar software written in other languages.
Or even worse, if the app needs to use unsafe, you'd be surprised how many Rust engineers think unsafe somehow "encapsulates" memory bugs inside the block itself. Which is not true. Using unsafe can cause memory issues that bubble up the call stack.
I don't feel any guarantees when a product is made in Rust. There are buggy Rust projects
I believe this real-life, 4-year, 1.5 million-line-of-code experiment disagrees with you directly [1]
> In general, use of unsafe in Android’s Rust appears to be working as intended. It’s used rarely, and when it is used, it’s encapsulating behavior that’s easier to reason about and review for safety.
[1] https://security.googleblog.com/2022/12/memory-safe-language...
unsafe fn create_null_pointer() -> *const u32 {
std::ptr::null()
}
fn main() {
let ptr = unsafe { create_null_pointer() };
// Dereferencing the null pointer outside of unsafe code
let value = unsafe { *ptr };
println!("Value: {}", value);
}
What this article is getting at is that the unsafe keyword highlights pieces of code that Google scrutinizes, not that it has functionality that "magically" encapsulates memory bugs. And yes, it's a lot better than C++. But you still need to be aware this can happen and not assume you're safe
All unsafe does is shut off the compiler for sections of the code. You can write code in an unsafe block that bubbles outside of the scope of "unsafe."*
> "you'd be surprised how many Rust engineers think unsafe somehow 'encapsulates' memory bugs inside the block itself,"
you essentially mean:
> "you'd be surprised how many Rust engineers think that they can use unsafe without regard as to whether they are maintaining Rust's safety invariants by hand in their unsafe blocks."
Yes, I do find that surprising!
No it isn't. It's not like we've been writing too much applications C-code recently and it's not like Rust offers that much fundamental security over e.g. Golang
It absolutely does, while also being more expressive. I suggest reading more of the type system literature before making unfounded claims like this.
Like what, in general? I've been working with variously typed languages of various maturity with stuff like structural typing, null-safety, sealed/enum types, inlined and dto/data types, functional types, various implementations of traiting/inheritance/prototyping, etc for years. Do you have a specific point or did you assume that being vague makes you look mysteriously smart?
Honestly, you are acting like all we ever had is JS, C and Rust.
> Do you have a specific point or did you assume that being vague makes you look mysteriously smart?
My point was what I said: you should read the academic literature in this space to understand why Rust is fundamentally safer than languages like Go. It's not just a made-up marketing ploy. The type system is specifically designed to make it very difficult to incur issues with memory, which was Rust's primary goal as a modern systems language. Yes, other languages have fancy features, and some of those features are not in Rust. But the features you mentioned don't specifically relate to memory safety, which was the topic at hand. Most of the features you mentioned are actually just about expressiveness of the type system, but not fundamentally distinct forms of type-checking.
Rust's type system is based on earlier work from Cyclone. You can find the various Cyclone papers here: [http://cyclone.thelanguage.org/wiki/Papers/]. Specifically, you'll want to look at the papers concerned with memory safety, which is where the region-based type system was first introduced.
> Honestly, you are acting like all we ever had is JS, C and Rust.
No, I'm pretty familiar with the landscape of programming languages — it comes with the job! I think you just didn't understand my point, which is probably my fault for not being clearer originally. It happens when I reply from my phone sometimes. Apologies.
That Rust is fundamentally more secure (in certain, specific ways) than other languages is a fact, not an opinion. It is a fact well-known by members of the programming languages research community, users of the language, and other sufficiently interested people. I didn't think to qualify it by citing the relevant papers partly because the fact is well-known, partly because the fact is easily researched, and partly because this is an informal conversation where you don't always need to bring primary evidence simply to talk to people.
Just because you were not aware of this fact didn't grant you license to be so dismissive at the start. Your first interaction with me was a stern, corrective "No it isn't". You were wrong, but worse than that, you were authoritative in your wrongness. If you had just asked me to explain what I meant instead of trying to correct me, this conversation could've gone differently.
Now, maybe I've misread your tone, in which case I genuinely apologize. But the certainty with which you made your incorrect statements at the beginning, followed by your kind-of condescending interactions with me in the comments since then, makes me think I am right in my interpretation, and I think that's really unfortunate.
I am honestly not. Please don't read too much into my tone, English is not my native language.
It actually wasn't clear why you've mentioned types theory initially, the semantics of "expressive types" confused me a bit, but I upvoted your latter post where you elaborated.
That said, I'm also annoyed by various navigability issues, including that one.