Addendum to “Curl is C”
daniel.haxx.se
daniel.haxx.se
This is an usual problem with people participating in discussion over the Internet. I am sure those people mean well, but often they don't put enough effort into assessing the situation and underestimate its complexity before they decide to hit the keyboard. I've myself done it a number of times, but I am working on not doing it.
Advice for the people who are making all these suggestions and: A good way to see if you're talking out of your ass—unintentionally, of course—is to try to do what you're saying someone should do. Just try and re-write some parts of curl in Rust (you can go with a hip name like rurl). It's a good exercise to learn about the project you're rewriting, the language you're writing it in, and the problem domain. So, at the very least you'll get something out of it, and if your project is successful: even better!
NO BACK-SEAT DRIVING. If you overhear people working through a problem, you shouldn't intermittently lob advice across the room. This can lead to the "too many cooks" problem, but more important, it can be rude and disruptive to half-participate in a conversation. This isn't to say you shouldn't help, offer advice, or join conversations. On the contrary, we encourage all those things. Rather, it just means that when you want to help out or work with others, you should fully engage and not just butt in sporadically.
More seriously speaking: IMO this is solid advice. It
- is actionable
- goes out of its way to be polite and respectful
- has a chance to actually improve online discussions one comment (and hopefully commit) at a time
This goes for operating systems, application software and advanced rocketry alike.
The reason we have so much C code powering the world is that (1) there wasn't anything better when that code was written, (2) time has weeded out most of the bad bugs to the point that the old codebase will be substantially more stable than anything new no matter what language you would write it in.
If someone is willing to throw a few hundred million or so at re-building an operating system from the ground up that is 100% safe and reliable and that does not have more bugs than what is available right now then they are free to do so. They're also free to donate their own time and to try to enlist their buddies. But they're not free to yell at others how those others should do their jobs.
Show don't tell.
But at the same time - we are spending those billions. We keep throwing billions at Apple by buying their new phones, yet they release lists of security issues which this time were basically all:
- malicious crafted image -> remote code execution
- maliciously crafted font -> remote code execution
- maliciously crafted audio -> remote code execution
For now c is the only thing we have that can be used so widely. Some day we may be able to use rust or something else, but that's future, not present.
Re: kernel - it's a lot of pretty simple subsystems. Each system is usually simpler (easier to verify and polish) than a lot of common libs doing really complex things (ssl, video codecs, ttf, ...). Complexity + raw pointers quickly add up to guaranteed problems.
Rust uses llvm. Use it for new development on any platform that llvm supports. A few microcontrollers might be left out.
So you're saying that Rust, at the moment, in theory, can be used in any place where C is required? Ie. writing kext on macOS, loadable kernel module on Linux or things like extension for SQLite are all possible?
In that case I wonder if there are any theoretical blockers for things like (using no_std and) mapping internally used C structures (let's say BSD kernel's rbtree) into typed, first-class Rust citizens - to avoid std overhead and reuse what's already available? In other words - is there anything there in C macros that can't effectively be expressed in Rust for example?
There are no theoretical blockers; tooling could make it a lot easier though. I was hand-porting chunks of the Ruby interpreter, which uses C macros extensively, and it's doable, but not fun.
Right, but the criticism he received for his first post was not primarily about choosing the wrong language or about refusing to rewrite in a different language now. It was primarily about downplaying the role of C in the security issues that were found.
The rest of the debate, at least as I read it, was not so much criticism as it was a generic debate about the pros and cons of various programming languages like we're having one every other day on here and on reddit.
Choosing C back then was clearly justified for a project like curl given all the constraints and considerations. And for a rewrite to be a net positive in terms of security probably takes years, not to mention the huge effort that someone would have to put into it.
There were better alternatives already available at the beginning of the 70s, and certainly when most of the software was written that we are using today. Just to name a few I am familiar of with:
- Pascal being as old as C, and later on
- Modula 2, Oberon, the Oberon System was completely implemented in Oberon
- Eiffel
and of course, today:
- Rust
- Go
So, even back then, there were more safe alternatives to C around. C gained more popularity, and consequently the tooling gave it some edge, but there would have been alternatives. Of course, when you have a nice debugged C program, that is something worth keeping. And as a consequence, for mostly historic reasons, a lot of our current infrastructure is built on top of C - but that alone does not mean that C was the perfect or even the right tool to build all of this upon.
This seems to assume that the codebase is stable and in maintainance mode. However, the reality for the vast majority of open-source projects is that they're either being actively developed, adding new code and new bugs, or they're abandoned.
Think about it: the people who design Rust don't have the time to rewrite their own Rust code to be safe.
When there's no time to rewrite the code of Rust to be demonstrably safe, why should any other project have more time than that, where the programmers know Rust much less or not at all?
Maintaining the successful project is hard and takes a lot of time, even without switching to something new.
Second point: look at the Benchmark game, a fast Rust code example also typically has an "unsafe" code. So it's still not even demonstrated that "you can have it both fast and safe" holds for these examples. Fast as in "as fast as C" and "safe" as in "provably safer than C."
Finally, I believe that the language should simply implement a language-level no-cost check for integer overflow, as the machine languages already have, and not complain that there aren't some special mechanisms in CPUs to do it differently. It's your language's failure if you can't use what already exists. The CPUs already can do it, the high level languages don't provide the high-level access to it. Which doesn't matter for C which is "unsafe" by definition (and where it's natural to call some asm when needed) but it should for Rust which goal is to produce a more provably "safe" code.
Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.
Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
I like the perspectives of what Rust could bring. But I'd like to see a real life case of its success too.
Your argument hinges pretty heavily on this. It's incorrect. The reality is that unsafe code in rust is not like unsafe code in C. First, it's bounded to a module. Second, you can grep for it (huge). Third, you still have less UB in unsafe rust than in C.
I've never written code in rust that required unsafe, after many thousands of lines.
> Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
?????????????
> But I'd like to see a real life case of its success too.
Ripgrep is a nice one to point to. There's redox too. Tons of rust code out there.
> I've never written code in rust that required unsafe, after many thousands of lines.
That illustrates the level of understanding of those who argue that curl should be rewritten in rust. Bravo.
curl has an order of 120000 non-empty lines in c and h files.
The person who made statement of "many thousand" probably made toy applications, compared to something like curl.
More realistic would be to compare something that should have been made in curl: like the "ring" library you mentioned as a Rust answer to "openssl." Which is neither an openssl nor written in Rust.
It is far from providing the functionality of openssl. The author states that:
"ring exposes a Rust API and is written in a hybrid of Rust, C, and assembly language."
"ring is focused on general-purpose cryptography. WebPKI X.509 certificate validation is done in the webpki project, which is built on top of ring. Also, multiple groups are working on implementations of cryptographic protocols like TLS, SSH, and DNSSEC on top of ring."
Actually a wrapper, and actually much smaller than openssl.
Second, "ring" has just some 9000 lines in rs files but 13000 lines of C (as it is defined by the author to be a wrapper, not surprising). So it is still just a wrapper around the C, still orders of magnitude smaller than curl, and still unable to be actually written in Rust.
On that level, I guess the Rust people would call curl the Rust application as soon as a few .rs files would appear in curl code base. Or not.
It's simply has no sense, supporting Rust with misinformation instead of being honest about its current limitations regarding what's actually implemented and what actually doesn't use "unsafe."
It's obvious that those who suggest to others to rewrite the big projects in Rust should first show that they managed to make some much smaller safe code. Especially that they managed to make some meaningful full-Rust and safe libraries.
(I don't agree with much of this but that's not the point; my point was trying to raise the level of discourse around here.)
One of the criteria for being in the standard library is "is this something which needs a lot of unsafe." This is so it can receive extra auditing, etc.
> yes, but it's an old code
This may have been true a while ago, but in my understanding (I'm not on the library team) it isn't really any more.
> why should any other project have more time than that
Because they do not need to write their own unsafe code, since it's already done in the standard library.
Generally speaking, applications in Rust should almost never need unsafe. Libraries may.
> a fast Rust code example also typically has an "unsafe" code.
This is not really true anymore; in fact, recently some programs were contributed that are 100% safe yet are faster than their previous, unsafe-using versions.
> believe that the language should simply implement a language-level no-cost check for integer overflow
This is not viable. Not enough machines have this.
> Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.
Unsafe Rust still has many, many more safety checks than C.
> Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?
Of course they have. It's a basic set of machine code instructions. Can you specify which doesn't please?
> One of the criteria for being in the standard library is "is this something which needs a lot of unsafe."
Do you honestly think a project like curl has everything it needs "which needs a lot of unsafe" already implemented in the current standard libraries?
Why don't you rewrite the curl in Rust then, it's just using what's already written? Let me know how far you come.
> https://crates.io/crates/ring ?
Does it have all the features that curl needs, or does curl still have to use a Rust wrapper around openssl, inheriting all the potential unsafety problems of openssl anyway? What is worth then, in percentage, having anyway less exposed part of the code in Rust? How many other wrappers curl has to use? Have you tried to check and estimate if it's still worth?
Once again, Rust people should first demonstrate that they can manage to make any relevant library "cleanly" safe, then complete the feature set, and only then expect big non-trivial programs consider using some Rust.
First make the basics safe, then let us use that basics. Once you have save openssl equivalent, even C programs can link to it and be much less exposed to the potential problems than they are now. It's that direction that only has sense.
That they exist is not the problem. That they exist _and_ don't provide enough performance is the issue. That is, your "no cost" is what I'm asserting here, not that it doesn't exist at all.
(This is not my area of specialty but was brought up when discussing this behavior; it's measured as a 20%-100% hit to speed, which is unacceptable.)
> Do you honestly think a project like curl has everything it needs "which needs a lot of unsafe" already implemented in the current standard libraries?
I don't think a project like curl needs much (if any) unsafe.
> Why don't you rewrite the curl in Rust then, it's just using what's already written? Let me know how far you come.
I did not suggest that curl should be re-written in Rust. I do think a functional equivalent in Rust would be good to have. I do not have the time to write such a thing myself. Luckily, many others have been working on this kind of thing.
> does curl still have to use a Rust wrapper around openssl,
ring is a port of BoringSSL to Rust, piecemeal. It still has some C code today, but at the end, will be all Rust/asm. Given that BoringSSL is a fork of OpenSSL, I would imagine that it would have enough functionality.
I'm claiming that the machine instructions exist to implement at no cost this high level construct (using C-like syntax):
if ( overflows( c = a + b ) ) {
}
try {
a huge bunch of instructions where some are additions
}
on integer_overflow_from_wherever do whatever
> It still has some C code today
Exactly my point why curl has no reason to be reimplemented in Rust now.
This is different than the claims that were made when we made this decision. Specifically, the cost angle. You can absolutely do this, but we don't turn it on by default due to its expense. See https://danluu.com/integer-overflow/ for example. From that link:
> Summing up, integer overflow checks ought to cost a few percent on typical integer-heavy workloads
That few percent matters. And the extensive discussion (in both directions) on HN: https://news.ycombinator.com/item?id=8765714
It would be reasonable to turn it on, but that's not the decision we made.
> Exactly my point why curl has no reason to be reimplemented in Rust now.
Again, I never said curl should be ported to Rust.
That's false. From[1], it looks like two submissions use `unsafe`. And one of those is for ffi to gmp.
[1] - http://benchmarksgame.alioth.debian.org/u64q/rust.html
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
2 pi-digits (ffi to gmp)
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
https://www.reddit.com/r/programming/comments/62cx5d/addendu...
There are many ways to be selective with evidence.
As-a-matter-of-fact all the contributed Rust programs I listed use unsafe.
So you say but do not show.
Please show us which of those Rust programs you count as "fast".
Please show us how many of those programs have unsafe code.
> The next time you criticize a meal, are you going to become a chef?
Well, they're not criticising the meal here, they're criticising the kitchen it was prepared in. :P
But joking aside, they don't need to rebuild all of curl in <language of choice>, just get it started, and if enough people think it's a worthwhile project, they will contribute.
> But providing thoughtful and useful criticism does not necessarily require actively creating something from scratch.
I agree. However, a lot of these comments are along the lines of "C is unsafe. Go rewrite curl in Rust!"[1] without realising how much effort and time that would take (and I'd not be surprised if some portion of this group never worked on a project in either language). I personally don't find those types of comments very useful.
> Otherwise, we would all only be starting new projects and never collaborating!
This is what the people I'm referring to are essentially asking. Going hybrid would probably introduce subtle bugs, and increase complexity and maintainability down the line.
[1]: I love Rust, and a few of my recent pet projects are being written in Rust (one of which is being ported from C). So, I'm not trying to bash Rust. I'd even encourage anyone to use Rust over C, if they were to start a new project.
For example imagine telling someone in 2000 that they should be running an automated continuous integration pipeline rather than purely manual testing. Now it's a norm. Then, we were still discussing whether Extreme Programming was a waste of time. But we also didn't have the tools to support the CI properly.
Memory safe compiled languages seem to me like the same thing. We had things like Cyclone, OOC, and other experiments which barely anyone knew. Now we have Rust which is still hard to use, but it's gaining ground. In a decade, I hope to be pointing out to this post when someone complains that you can't criticize an entire methodology. "Remember when people said the same thing about enforced memory safety?"
Curl will not be rewritten of course. But I'm happy people push this idea. The next curl will not be written in C.
But if a car manufacturer consistently uses a production methodology which causes door handles to keep falling off, again and again, even after they've been re-glued on for the Nth time...
Then I think it's fair game to say that you think there's something wrong with the methodology. People suggesting alternate approaches without putting in the actual work involved may very well be "backseat drivers of the Internet"-type commenters, but that doesn't mean they're not on to something.
My comment was meant to pertain to the suggestion floating around that Daniel ought to undertake a total rewrite of curl in Rust. That's an entirely different ball of wax. It's like telling the car maker to tear down and build a new factory because the door handles keep falling off.
We know the characteristics of glue, and have switched to better glues. All our assembly jigs work with glue. When the handle falls off it is easy and cheap to glue it back on.
If we switch to screws we need to redesign the parts and the jigs; and reorder the entry assembly line. [Not realized by the engineers at the time is if they use screws dissimilar metals of the screws corrode and result in the handle falling off as well - and it is a much more expensive fix because you have to replace the handle and the part it attaches to as both have nothing to screw into)
On the other hand if there is a project like curl rewritten from C to Rust and its author says: "We have switched from C to Rust in a year and after two years we have less new bugs", there is at least some value.
Politics: Are you critiquing the distribution of funds to the school your child attends? You probably have some insight there. Are you instead expressing a strong opinion on a complex subject that you likely don't have an iota of experience with? Well, then you should realize that you're probably not in a position to have such a strong opinion.
Meal: You're the user, so you have experience here. All you're commenting on is how it tastes to you, so that's perfectly valid, no need to become a chef.
Car: Again, you're the user. Are you critiquing a UX problem? Well, then you're in a perfect position to do so. If you're remarking about e.g. the torque to weight ratio then you should probably understand some of the engineering that goes into creating a vehicle.
Your argument is the fallacious one here. You're making an all or nothing comparison and life rarely works that way. If you're going to make a statement like "you should rewrite curl in X language" then yes, you should know what the hell you're talking about because you're making an engineering level statement.
Your objection follows a common pattern - if we took every nuanced situation and turned it into a universal black-and-white dichotomy, we would end up with half the arguments on the internet.
I don't doubt that some suggestions have come from backseat drivers... but I also don't believe that's relevant. First, people are implementing curl's functionality in Rust (e.g. hyper, reqwest) and it's working out well. Second, you don't have to be the one implementing something to push for improvements- in the general case, that's absurd.
That is good, and it's the reasonable thing to do. New tools provide the same functionality, get used where curl would have been used otherwise, and when the time is right, curl can enjoy a well-earned retirement in the hall of fame of famous ancient projects.
I believe opening our minds will be the only way to achieve meaningful improvement in software development.
I also believe this is easier said than done.
Those of us using CP/M and other systems had access to K&R C dialects like Small C, not fully compatible with C.
In those days C was just another systems programming language, looking for a spotlight.
That spotlight came thanks to universities adoption of UNIX, which given AT&T prohibition to profit from it, it was sold at symbolic prices and a few AT&T researchers did provide tapes to universities. Which then started to adopt UNIX instead of paying commercial licenses for other OSes.
So when workstation startups like Sun and SGI started to look for OSes, they based their software stack on UNIX.
Before the widespread of UNIX out of university labs into the workstation market, C wasn't that relevant.
You see the same happening nowadays with Browser == UNIX and JavaScript == C.
Also C is actually 10 years younger than Algol or PL/I, both used to implement several mainframe OSes.
C was the system programming language of Unix. Many people wanted Unix, so they started programming in C, because that was the best supported language. In the late 70s, byte code interpreted Pascal was also popular on Unix because it allowed for bigger programs on the then current 16-bit systems.
It's pity that Modula-3 never got much traction. I guess that's part do to Modula's overly verbose syntax.
C works as long as the platform is sufficiently like Unix.
For example, in the MS-DOS world with near and far pointers, programming in C becomes utterly confusing.
These days, linker scripts and ELF hide a lot of stuff C cannot express.
C can still be really "interesting" on Harvard architectures (many popular micros, PIC and AVR for example). Though it isn't that bad, since the compiler sorts constants and the like automatically out and loads them into memory (which means that by default constants that are not optimized out - like strings - will be deduced from the available memory ... 64 to 2048 bytes on typical SKUs).
This is the only thing I've seen work in the past.
For all the talk of 'a better system programming language' and replacing C/C++ there is curiously little interest or code written to prove that any of these new languages can be used to write systems from scratch and that the language and runtime can scale across architectures.
Thats before you even get to a comparison of efficacy and speed vs C/C++.
The quality of code generated by C compilers used to be pretty lame in the 80's, any junior Assembly programmer could do better.
As for ubiquity, C was a UNIX only language, with K&R C incomplete dialects available in other systems, where C was just "yet another language".
It was the adoption of UNIX, an almost free ($$) with source code available, by the 80's startups for creating the workstation market (like Sun and SGI) that lead to UNIX and C's adoption in the market.
Had UNIX been as expensive to get as the other OS of the time, and we wouldn't be talking about C's ubiquity and execution speed of generated code.
A vast amount of software in C was written for other platforms than UNIX, both SGI and SUN were companies that were focused on selling hardware rather than software. If anything the workstation market was created by companies like Xerox and commercialized by companies like Apollo.
UNIX was for a long time a very expensive operating system on X86, it only became affordable with SCO/Xenix and it became free with 386BSD by Bill and Lynne Jolitz and later Linux by Linus Torvalds.
C definitely wasn't a UNIX only language, it was available for an enormous range of computers from 8 bitters to top of the line machines. In fact that (and not UNIX) was a major factor in C's long time dominance.
Companies buying UNIX weren't buying X86 systems to run it.
We were using CP/M (on Spectrum +3 A), MS-DOS, Atari, Amiga, C64, ZX Spectrum, MSX.
C was just yet another programming language, nothing special about it.
In fact the only reason to bother having a C compiler on those systems for my group of acquaintances, was to be able to bring home the work done in expensive UNIX computers at school.
C supporters are the ones that like to rewrite history by selling UNIX and C as if they were the genesis of systems programming and OSes.
Also by the time Windows 3.1 got out MS-DOS was already at version MS-DOS 5.0 and DR-DOS at version 6.0.
During the lifetime of MS-DOS, many of us were happily coding on Clipper, Dbase, Turbo Pascal, Turbo Basic, Quick Pascal, TMT Pascal, FoxPro, NASM, TASM, Turbo C, Turbo C++, Modula-2....
There was plenty of choice.
But it takes time. Unless you're actually looking for those projects, don't expect to even know they exist during the first 5 years.
But I think Daniel's original attempt at explaining away C's security issues has been debunked. And I think he owns it well by singing a slightly different tune in this blogpost.
The lesson learnt here is that if you use C, don't try to explain away the security issues you have just opened for yourself. It will not hold up. Accept that the safety of C is a very serious issue, and that there are other reasons you choose it. Choosing C over more secure options does not mean you don't care about security, but it means that you had to make pragmatic choices. Choosing C over more secure options and subsequently attempt to minimize the security impact of this decision (not talking about Daniel here) will lead you to not addressing those issues effectively.
When you start a new C/C++ project in 2017 you can effectively and confidently address security and 'safety'. And it is done on a regular basis.
There is an entire infrastructure of tools and introspection refined over the past 30+ years for C/C++ that is part of the project from the beginning. Address sanitizes, Better compilers, Valgrind, Intel's tools etc.
All of the potential problems are known and there are tools to deal with them. But the responsibility is put in the hands of the programmer like many other things in C/C++.
1) You're stating that automated tools are a solution in a thread in a blogpost for a heavily scrutinized tool that has a zero tolerance policy for Coverity problems, uses Valgrind and address sanitizers and still owes more than half of its CVE's to various language and memory safety issues.
2) "The responsibility is in the hands of the programmer" - I think we have argued enough that putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it.
You're minimizing C's security issues again, please stop doing that.
Language designers have been arguing (mostly with themselves) since the 1970's that pointers are dangerous and memory management is bad and that programmers should not have those things.
We can go on using C/C++ and language designers can go on creating there new version of Pascal. We all know how that turned out.
> I think we have argued enough about that putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it.
System designers and those writing performance critical code program hardware. Not an abstract machine that the language designer thinks is good enough for lowly programmers.
Convince Intel and AMD, Nvidia etc, that they should make chips and instruction sets that have no access to memory, cores, vectors and so forth because 'we should do the best we can to stop it'. Lets see how many of there clients want to buy useless blobs of silicone.
A computer is a tool for computation not an adult diaper designed to prevent leaks.
In addition, you're arguing a strawman. Nobody says that direct memory access should be taken away permanently. Many 'safe' languages allow you to directly access memory when you need it.
> A computer is a tool for computation not an adult diaper designed to prevent leaks.
This attitude is why we can't have nice things. It would be great if you cared more about the security and safety of the things you make than whether you look cool while making them.
I don't think anyone was arguing any of those points. What was being explained is that C/C++ is designed to be flexible and simple and it was know from its inception that that flexibility lets the programmer 'shoot them selves in the foot'. Building out tooling to address any error patters in usage and keeping that basic simplicity and flexibility is desirable. More desirable then moving to a more complicated less flexible language that may eliminate some possible errors.
There are literally millions of C/C++ programmers and projects writing secure reliable code. Picking out a few open source projects and turning there bug reports into an internet drama is not going to convince anyone the sky is falling.
> In addition, you're arguing a strawman. Nobody says that direct memory access should be taken away permanently.
I think its fairly clear that the sentiment was.
"putting this responsibility in the hands of very imperfect programmers is precisely the problem and we should do the best we can to stop it."
This has been the justification for every misguided attempt to 'make things better' for a very long time.
A short list of unnecessary things that needed to be stopped thought the years. * Pointers * Manual memory management * Unconditional Jumps * direct access to machine words ie bytes * assembly instructions. etc..
I am fairly skeptical of arguments that suggest 'the programmer does not need that'.
I believe that there are millions of C/C++ programmers that think they are writing secure reliable code. But these there is an overwhelming amount of evidence that they don't and (for C at least) can't. There is plenty of evidence that at least C does not give you the proper tools to actually write safe code outside of trivial programs in the first place. (I cannot speak for C++)
You're still arguing a strawman. Nobody wants to take away your pointers, manual memory management, unconditional jumps or assembly, and nobody is suggesting that you don't need them. People just want to make it easier to use them responsibly without making the mistakes everybody keeps on making all the time.
In my experience the people who wring there fist at C/C++ are generally people who at one point tried C/C++ and either got frustrated and quit or got told they suck or turned down at job interviews. Or they are people trying to sell you the new hotness and have some personal gain to accomplish. People I know who did some C/C++ and now work with other languages could care less about replacing C/C++. They just do there thing.
And honestly, academics and busybodies have been whining about safety and memory and hating on C/C++ for decades. We need to get rid of it. It smells, its bad for your teeth etc. Creating a never ending stream of total crap programming languages that no one wants to use. That never actually do what there creators propose that they do. And that never replace what the say they want to replace.
Even some crusty C++ programmers still hate on C coders and how did that work out for them :)
Oh, so you say they don't use C, right? It's defined in terms of abstract machine which has very little in common with how actual hardware works nowadays...
Yes, there are reasons to sometimes disable those mechanisms. Yes, it is not a good idea to force everybody right now to abandon cars without it. Still, if you build a school bus today, I would criticize you, if you don't include the safety technology.
Just saying :-)
The video is easy to find if you want to watch it.
So apparently the tools might be there, but they are ignored by those that should be using them in first place.
Probably a lot of it is simply lack of familiarity, though. I've never seen a university course that even touches on this...
I mean, speaking as one of the 1% who fully voluntarily uses these sorts of tools (and can generally only get free ones either because the price is prohibitive for the expensive ones, or I'm working in languages that expensive ones don't exist for), I think you're all bloody loonies for not doing it too. Offload to the tools everything you can, to save your precious brain cycles for the stuff that matters. Makes you look like even more of a hero.
(The excuse I see all too often that by doing that you'll atrophy your own judgment is, by the way, itself just proof you've never used these tools, or at least not used them properly. If you use them frequently, i.e., at commit-time, rather than quarterly-release time, they train you by telling you what you did wrong, and you learn how to write better code that meets their criteria fairly quickly. They make you a better programmer. These things should be used early and often; I'm sure a lot of the reasons people tend to not enjoy using them is that they want to write a quarter million lines of code without ever consulting the tool, then run it once at the end. That is a terrible plan. The tool will scream bloody murder and especially in C you are very likely to discover that your entire architecture is fundamentally impossible to fix now, making it very discouraging. Use the tools early and often. Seriously, as soon as you've created the new project and have hello world going, hook up the tools. They both provide more value and are way easier to use that way.)
I always had to pay for my tools, just like any other professional.
And I still do pay for the tools I think are worthwhile, if not upfront, at least as donation.
But that does not change the fact that C/C++ is preferable and often the best choice in certain domains.
While one might not be able to pay for tools for static analysis, there is no excuse to ignore the type safety improvements C++ has over C.
Anyone that writes C with C++ compiler, should keep using a C compiler if he/she wants to stay in the past.
This is untrue. There are tools out there to deal with a lot of the issues, but not all of them. Due to what C allows and the halting problem, you cannot create such tools. Static analysis and dynamic sanitizers are still useful of course, but let's not pretend they're a solution.
One of my favorite examples so far: https://www.postgresql.org/message-id/CAASwCXdQUiuUnhycdRvrU...
I don't know where all the C hate is all about. I love C, it's a great tool. Can be quite sharp, be careful with your remaining fingers.
I spent the better part of last week studying rust and go, and my conclusion was 'meh, I'll have another look in another few years" and kept on trucking.
One company creates a closed-source component that produces a float, another company creates a closed-source component that receives a float. Units differ. Crash. No type-safe language will prevent that bug from happening. Let 100 programmers write these programs in your type-safe language of choice and all will be vulnerable to this.
"Oh no", I hear you say, "we'll give every unit its own type" and it still won't stop your programmers from deserializing a 4 byte value in English units into the metric value they think it is. "So we'll make all the IO typed as well!" You don't need a type safe language to do that. And why would you do it in the first place? If your API spec says you receive a float in metric units, you write your program accordingly. When you're writing mission critical software, would you really want to replace a
> read(sensor, &value, sizeof(float))
with 100k lines of even buggier code to prevent a bug from happening that shouldn't happen in the first place?
Some will be, some will define inteface as float_meters and define automatic conversion from yards, avoiding whole issue.
Example of successful compilation in C Language which is semantically wrong: http://ideone.com/Hdd4Sh
(No casts required.)
Any program in any language can be regarded as not having passed checks for some imaginable something.
A perfectly good poker game is semantically wrong because the customer asked for an accounting application. The compiler should have checked the code against a formalized version of the requirement spec, damn it!
No, that isn't quite right. I'm the one who authored that code example so even if I used opaque non-English struct tag names such as "struct s1{}" and "struct s2{}" instead of "struct meter{}" and "struct inch{}", my brain intended for s1 and s2 to not be mixed and matched carelessly. Poster mikulas_florek suggested wrapping plain ints with a struct and my example shows that that workaround doesn't accomplish what he says it does.
>It is well-defined ISO C.
Yes, and I wanted to emphasize that truism by showing a live example of a successful compilation in ideone.com!
I thought the thread was about higher-level semantics rather than the set of trivially true sequences of text input that the compiler accepts.
Successful C compilation rarely means that only higher-level semantics not encoded into type system is now violated.
The battles in C are already being lost in the trenches of lower-level semantics.
If worrying about these higher level things and getting them right were the only problem in C development, C development would be in fantastic shape.
In safe languages which don't have type systems that can express pounds versus inches in inviolable ways, that sort of thing is rarely any sort of obstacle to real work; it's a fluff concern.
If I have to program in something which has no way of enforcing pounds being added to inches, and that is the only significant drawback of that language, I'm laughing.
I didn't make that claim that UB goes away with a successful compilation and therefore "only" the higher-level semantics are worth studying. Since nobody in this thread said that, I'm puzzled why you'd think I'd have such a naive position.
>these higher level things and getting them right were _the only_ problem in C development,
But I'm not arguing they are "the only" problem. Again, poster mikulas_florek suggested a code technique that claims to address a problem but does not work. That's the limited scope of my demo code.
I don't know how theoretically fixing that tiny snippet got elevated to be the "savior to all C problems". I made no such grand claim.
>, that sort of thing is rarely any sort of obstacle to real work; it's a fluff concern.
Whether it is or isn't in relation to actual millions of lines of real-world code that may or may benefit from it is not relevant. Curl is an example of real working C Language software that may have zero code requiring type-checked units-of-measure. That's still not relevant to the previous poster mikulas_florek mis-educating C programmers.
Btw, the units-of-measure is just one variation of the "type check" issue. Many times a programmer will have a function definition such as (customer_id x, action_code y). If the programmer accidentally reverses the 2 typedefs, the compiler won't catch that. Example: http://ideone.com/2dyT9n
That type of human error happens all the time and I disagree that would be a "fluff concern".
So, getting to hyper-focused on specific type failure examples such as units-of-measure instead of considering the more generalized higher-level errors not caught by C compiler is missing the point of what the thread is about.
1. you can have the same problem in type-safe language, since it will probably have cast to int too 2. do not use builtin operators in C for that, instead create new functions, e.g. addMeters(Meters a, Meters b); 3. hide the int in *.c file
- do not use builtin operators in C for that,
- instead create new functions, e.g. addMeters()
- hide the int in *.c file
"Best practice" is a choice from among justifiable alternatives.
public class Meters { public int x; }
public class Yards { public int x; }
public class Kilograms { public int x; }
Meters m = ...
Yards y = ...
Kilograms k = ...
k.x = m.x + y.x;
- do not use builtin operators in C# for that instead create new functions, e.g. addMeters()
- hide the int in *.c file == make it private in c#Yes, you can. But at least you have a better option and better chance of other people using it, so statistically you'll have less errors.
> do not use builtin operators in C for that
No, do not add another rule to a million of rules people have to already follow, make compiler worry about enforcing that, not developer.
I highly doubt that this specific bug would not happen in any typesafe language. The only way I see this not happening is if there is no native float-like type. Is there such language? They could have make it safe in C, and they did not, probably because it's convenient to use float.
> No, do not add another rule to a million of rules people have to already follow, make compiler worry about enforcing that, not developer.
It can be implemented in C so it's forced by compiler.
What C feature forces or encourages casting Meters* to Yards*?
> You also cannot override operators for combining those units.
The lack of operator overloads has nothing to do with typesafety. There are languages which are considered typesafe and which do not have operator overloading.
What is NOT idiotic iswhat you discribe as 'ridiculous amount of verification' as it's often the only thing that will really work to catch mistakes.
But it will limit the damage they do considerably. Having bit of space or trees between lanes on a road saves a lot of lives, even if people drive as idiotically as they would without it.
> What is NOT idiotic is what you discribe as 'ridiculous amount of verification'
Designing systems that rely only on that is idiotic when there are better options.
Except that's not what the commenter said, and what the commenter did say is, although glib, not inconsistent with the motivation behind Bernstein's entreaty to just create a boringcc already.[1]
Also, generally, just don't make up fake quotes to attribute to people. It's poor form.
> I spent the better part of last week studying
Telling.
1. https://groups.google.com/forum/m/#!msg/boring-crypto/48qa1k...
And you've failed to address the parts of my comment that are most relevant to the discussion. Why?
Beside, my reply wasn't related to that particular post in detail it was paraphrasing (<-- hint) a behaviour that is pretty common across many projects, wether they are open or closed source.
Sorry to come out as a bit flippant, but I'm slightly overdosed on the 'fake .' argumentation these days.
I think Daniel Stenberg missed the point of some of the comments. (At least the prominent ones I saw.)
>We use C for a whole range of reasons as I tried to lay out there in spite of the security challenges the language brings.
The availability and practicality of old ecosystems are valid justifications but it's orthogonal to the "memory bugs caused by the language vs the human".
There's nuance and you have to distinguish multiple conversations:
1) curl should have been written in a memory safe language (past tense)
2) curl should be rewritten in a memory safe language (future tense -- e.g. maybe use Rust)
3) setting aside curl's 19 years of wide deployment challenges, D.S. could have been a better spokesman for the benefits memory safe languages
Daniel's response is mostly to #1. However, some of the criticism is really conversation #3. Understandably, #3 is somewhat harder for D.S. to adopt because it requires distancing himself from the 19 years of curl as a successful utility and instead, consider how a language's design affect programmers' bug count. Many felt that D.S. citing curl's bugs as "human logic errors" instead of "language-induced errors" undermines his point when conversation #3 is the framework.
So maybe some of the new nitpicking of counting CVE bugs is more about conversations #2 & #3. The #2 may not be realistic for another 10 years... or maybe never because of various deployment hurdles. However, we can still discuss #3.
Most programmers who attack C have seen the consequences of a memory-unsafe, type-unsafe language dominating systems programming throughout the years. I remember the period during the early 2000s when "buffer overflow exploits" were an entire industry. I also remember the strange allure of C and C++ in beginner programmer circles. The logic goes something like this: "they are powerful languages, all of the big and popular software projects are written in them, so they must be the best". Their next step was to go on to start a new generation of unsafe and vulnerable programs and libraries, and so on.
Now that we have the tools and technology to do better, we should really be making some efforts to push for them. The goal isn't change right now, but starting the process early seems like a good idea.
You just have to witness the number of "OMG, you did that in C? Wow!" that are posted every time someone posts a basic C program on a forum. There is a whole crowd that thinks C is some arcane stuff only to be used by by magicians, and if you're not a guru it's gonna kill you.
And then you have the Rust people who chime in and point a list of C problems that are in fact C++ problems.
And then you get the "but anyway modern processors do not execute the code you tell them to execute" guys. Sigh...
It goes every time the same way...
Having half-baked information on how C sucks is better than having illusions that its somehow awesome. This:
> There is a whole crowd that thinks C is some arcane stuff only to be used by by magicians, and if you're not a guru it's gonna kill you.
is actually a very reasonable starting point when deciding whether to start a substantial new project in C. Well, minus the arcane part - part of C's success can be attributed to how simple it is to learn how to shoot yourself in the foot.
It also means the number of new programmers that are fascinated with / desire to learn C continues to drop; hopefully that will be enough to break the cycle at some point in the (not so near) future.
If we then have had 40, 50 or even 60 security problems because of us using C, through-out our 19 years of history, it really isn’t a whole lot given the scale and time we’re talking about here.
I think there's a disconnect here. Given something that's has a thousand instances running and averages 20-30 security problems a year and something else that has over three billion instances running and averages 2-3 security problems a year, which one do you think I care more about? I care about the one running everywhere, and on devices which may never see updated code in the rest of their lifetime.
If curl were some unused project this wouldn't be a conversation. At the point where your code is on all those devices, you will get scrutiny. Expect it. Your choices may have some impact on a significant portion of the Earth's current population.
At the same time, it's not fair to expect curl to rewrite their project in some other language. Some other project can start with the goal of providing the same API as curl (or not) that attempts to mitigate security problems though a different code architecture or language, and maybe curl devs will contribute. But expecting curl devs to start over from scratch is not feasible.
A few years ago I looked at the API and internals and went, "gack!" No way am I integrating this dung heap at that level; let the users just wrap the process with a curl function that returns a process pipe stream if they want to use this.
So you could, for example, take a 400 line C file, convert it to Zig, and have it import the .h files that the previous C file was including, and export an .h file for the rest of the codebase to look at. The compiler can turn this Zig file a .o file that integrates with the rest of the build. The part of the codebase written in Zig could benefit from the safety features, generics, and error handling features of the language while presenting the same interface to the rest of the codebase.
Now of course, if you already have a codebase written in 1 language, it's more complicated to introduce another. This is where the Zig build system comes into play. I'm working on this now, and it's a competitor to autotools, scons, cmake, make, and all those players. It's the Zig programming language with a declarative build system API built into the standard library with some basic tooling around it, and it ships with the compiler.
So if the Zig build system gets to the point where it is high quality enough to attract users and ubiquity in package managers, then a project which already uses Zig for its build system can start using it for its actual code without the penalty of adding a dependency.
In conclusion, once Zig has reached a stable enough state, I would challenge the libcurl devs to consider, perhaps it could make sense to slowly, carefully, methodically transition to a newer language, if that language was friendly enough to the process.
Of course, I don't think that curl should be rewritten in Nim. But since we are advocating for other languages I thought I would pitch in. If anything, writing new curl functionality in Nim would be the way to go. That is assuming that curl's author is unhappy with C, which does not seem to be the case.
If you depend on Nim then you depend on Nim, a C compiler, and a linker. And I hope your C compiler supports cross compiling out of the box. It probably doesn't.
It's a small number of build dependencies.
> I hope your C compiler supports cross compiling out of the box.
It does.
Also, it might be useful to know that HN's comment syntax isn't Markdown, though it shares a few features like indenting for code blocks and asterisks for italics. Markdown-style links are not supported.
The "HN style" of citation/linking is basically the IEEE citation style. You could call it "blessed".
The reason I say that is that I could actually have written something like that a few years ago myself. I think it's a pretty common mindset amongst the "low level" C crowd. And there's a good reason for that, we've been burned before.
People have been advertising "safe" alternatives to C since forever. Use Perl! Use Java! Use Lisp! Use Haskel! Use Python! Use Go!
But none of these languages can replace C in all of its use cases. You have big runtimes, garbage collection, interpreters, you name it. You can't just bootstrap a stack pointer in 3 lines of ASM and call into them like you can with C. And even if we don't say it out loud there's a bit of snobbism to writing low level "unsafe" code. "Safe languages are for noobs and webdevs, real coders debug with oscilloscopes".
And so I had this mindset that you couldn't have your cake and eat it too: you could be safe or you could be low level. Segfaults and NULL pointers were just a normal part of dealing with low level "unmanaged" environments and we'd just have to deal with it. People who complain about it are just not worthy.
In my opinion the biggest success of Rust so far is in proving that you could be safe without needing a GC, a big runtime or complex abstractions. You could design a better C without having to compromise. Even if Rust fails in the long run, it would still be a very important "proof of concept". It showed that C was unnecessarily unsafe, that you could achieve the same feature set without throwing safety completely out of the window.
That being said if there really are folks bugging devs by asking them to consider switching to language X or Y then I think it's fair to say that these people are idiots. Might as well ask JK Rowling to rewrite Harry Potter in Esperanto because "it's totally the future!".
I mean, let's face it, at this point Curl probably has a much bigger userbase than Rust itself by at least an order of magnitude. Which one is more likely to be irrelevant in 10 years? Rust or Curl (and by extension, C)? If you answered the latter you should read less hacker news.
And even if Rust ends up being successful it's not like C will go away any time soon, or even in our lifetimes. Curl (as a C library) will probably still be useful and relevant for decades to come no matter what.
He should have just said: "curl is an established project. New languages are not being seriously considered for curl."
If you want to close down a conversation, it's counterproductive to make new dubious claims like "C is not the primary reason for our past vulnerabilities".
Note that closing down a conversation isn't bad. Sometimes they are annoying or distracting. I love rust but here was my comment when someone wanted to rewrite postgres in rust:
My personal opinion is that if he was wrong - we wouldn't even be reading his argument - it wouldn't make it to the top of HN - everybody would be using Xurl library instead (written in X-lang, where X is not C).
At the same time Rust is shaping to be fantastic language and I personally hope that soon we'll see Rurl cargo together with kernel and what not in Rust. Unless in this decade something better will appear, who knows?
Is it very difficult to transpile C into Rust?
As an aside, I not like the curl style guidelines [1]. Especially "'else' on the following line", and "No space before parentheses". That doesn't look nice at all.
Last I heard, Corrode can convert many C syntactic structures into their Rust equivalents, they usually compile and sometimes they even work. Even when the conversion works flawlessly, the resulting Rust does exactly what the original C code did—there's no safety benefit (but you can begin refactoring to clean things up).
In theory no, in practice yes. Rust is using LLVM project to generate machine code, and that only supports certain architectures. There are however other safe languages that can generate C, if this is a requirement.
> Is it very difficult to transpile C into Rust?
If safety guarantees could be added to C by some conversion or analysis, there would be no need for Rust to exist.
Rust (and other languages with similar goals) is safer for many reasons, not limited to languages itself. Stdlib and runtime play their part too. Automatic conversion that takes all that into account is either impossible or impractical. There is a tool for converting C into unsafe Rust (corrode), which might be useful for people attempting to port C code, but in itself doesn't improve code safety at all.
"C is not the primary reason for our past vulnerabilities"
So it's entirely reasonable for critics of C to challenge that claim.
> curl is currently one of the most distributed and most widely used software components in the universe, be it open or proprietary and there are easily way over three billion instances of it running in appliances, servers, computers and devices across the globe. Right now. In your phone. In your car. In your TV. In your computer.
Congratulations. So how many of those have unpatched vulnerabilities right now that were caused by using unsafe language? In my phone? In my computer? In my car? How many accidents of data or money theft it caused? Not to mention common non-security related bugs.
> If we then have had 40, 50 or even 60 security problems because of us using C, through-out our 19 years of history, it really isn’t a whole lot given the scale and time we’re talking about here.
Curl had multiple unpatched security problems for several years before they were discovered. As the author admits, it most likely still has many. dpkg -l | grep '^ii lib'|wc -l lists over 2000 libraries I have installed, lot of them written in C. An estimate of 2000 multiplied by 60 is ... scarily to many. Large amount of them avoidable by using safer languages.
> Using another language would’ve caused at least some problems due to that language
If it solves more problems than it creates, it is worth doing.
> none of the memory safe languages anyone would suggest we should switch to have been around for 19 years.
Neither have been mobile phones (at least as we know them), but they are useful, so I am using one.
> We will continue to work hard on minimizing risks, detecting problems early by ourselves and work closely together with everyone who reports suspected problems to us.
I prefer to work smart, not hard.
In the meantime, it appears rather difficult to replace the last 40 years of work overnight in a novel, untested language, with support for a tiny fraction of the targets that C has been supporting all that time, while maintaining reasonable performance.
Or in short: It's easy to work smart when you're not working hard in your arm chair.
I can point you to numerous examples of people writing and rewriting software, there is no reason for it to be my work to prove its doable and often worth the effort. And all languages that generate C code are as portable as C is. I can also point you to countless examples where people replaced their dependencies with something written in another language. Also please go back to first line of my comment and read it again.
> In the meantime, it appears rather difficult to replace the last 40 years of work overnight in a novel, untested language, with support for a tiny fraction of the targets that C has been supporting all that time, while maintaining reasonable performance.
Who suggest it can be done overnight? Why not use something that compiles to C if you need this compatibility? There is no need to replace all of it at once - you can pick particularly sensitive parts first, one file or one library at a time. There are many ways to introduce safer languages if you want to do it. Also difficult or not, all code will be replaced eventually.
As for performance - its irrelevant in large amount of networking code. I'd have no issue whatsoever with adding few cpu instructions for bounds checking, when you wait several milliseconds for IO operation, which is the case with curl.
Why? You don't have expertise needed to do that yourself? It seems you think you have enough to lecture others and write how easy it is (just generate some C code).
> I can point you to numerous examples of people writing and rewriting software
Ok, point me to this software that was rewritten from unsafe C code and was replaced with memory safe language. I don't want any hobby projects, real software used by millions. Give me 10 examples.
[1] https://cdn.infoq.com/statics_s1_20170328-0458_1/resource/ne...
Any language that wants to present themselves as a replacement for C/C++ hast to have at least a reference implementation of all those things so people can start to evaluate (the system).
No one is going to invest the time or the resources into a language to replace what we have if its not an actually up to the task. No one needs another programming language sitting on top of C/C++ that kind of makes things a little better for one class of bugs but will never really replace C/C++ and our our system interfaces.
There is a long list of languages that have made that claim and most of it has turned out to be hot air and marketing.
I do not mean to trivialize compilers, linkers and assemblers, but fundamentally they are programs that just read and write files. They don't really have to be in C, IMHO.
I am still waiting for someone to chime in with credible examples of successful C -> non-C transition.
And your right that compilers etc, do not absolutely need to be written in C/C++. But all of this is taking place in the context of the market. Open source operates in that same market.
If you have two teams working on compilers in different languages and one is using C/C++ its probably going to be much faster with fewer dependencies. 90% of the business is going to go that way and the other team is going under. And so it goes all the way up the stack.
On the other hand what about the kernel and the system libraries? Even if you replaced the whole tool chain with another language but still have a C/C++ kernel and system interface everybody is going to want to use the native interfaces and therefore work in C/C++. So whats the point of adding another layer.
Given what we have now and what peoples expectations are I just don't see 'a replacement for C/C++' ever happening until You can wipe your drive and install a new OS with a kernel and system libraries in a totally different language and never have to touch C/C++ again for the projects developed for that OS.
I just remembered reading about MirageOS recently. Almost everything is done in OCaml. I am sure it contains a bunch of low-level C code. It seemed interesting, but I don't think such projects will ever replace systems written in C.
I guess you don't do much OS X, iOS, Android ,ChromeOS or UWP programming.
OSX is a certified UNIX.A bunch of BSD people ended up over there. Android and ChromeOS use the Linux kernel and Redhats C lib. The one time I checked it looked like most of there compositor and components are also C/C++, skia etc. Windows is written in C++ and you have WinAPI. And of course vanilla Linux is C and all the GNU libs are C.
So its WinAPI and Linux system interface with a preference for POSIX. With lots of other irreplaceable Libraries that are also only written in C/C++. High performance high memory usage that needs to run stable at full load and maximum memory for days at a time.
Keep in mind that there are large industries that have usage and development patters that your oblivious to. And that claims of safety espoused by language developers based on criteria that they themselves have created are so low on the priority stack that there simply non existent in specifications.
What should we do? Should Google and Microsoft and The Linux foundation rewrite there whole stack in XYZ meme language? Or is it just all the other companies that that should switch to a language they don't need or care about just for the hell of it?
OS X, yes it is a certified UNIX, but the software world is more than command line tools and UNIX daemons.
Any OS X, iOS, tvOS, iOS application that regular users actually want to use, rely in Objective-C APIs making use of classes, categories and protocols, with some of them actually being nowadays implemented in Swift, like the new audio stack.
ChromeOS, yes it is built on the Linux kernel, however the whole stack is built to manage Chrome instances. Only web apps count. Any developer creating applications to ChromeOS only deals with the web stack.
Android, oh sure it is built on top of a forked Linux kernel, stripped out of any signs of UNIX V IPC, but regular application developers only see Java APIs. Surely there is the NDK, which exposes about 5% of Android APIs, has a very limited set of official C and C++, and any attempt to link to on-device libraries not part of the sanctioned list, terminates the application, as of Android 7.
Windows, sure the kernel is built on a mix of C and C++, both Win32 and UWP build on top of that. However .NET Native is also able to produce UWP components, just like C++/CX. Watch any Microsoft conference since the launch of Windows 10, and you will see in which direction the wind is blowing on the whole .NET Native vs C++/CX for UWP applications.
Tizen, yep they built it on top of Linux kernel, offer some native API that has already gone through three reboots, but guess what. The main APIs are their web stack, native is not even allowed on Tizen for TVs and now their are adopting .NET Core as the new API stack.
As for POSIX and mobile OS, maybe you haven't been paying attention to the news.
"POSIX has become outdated" -- http://www.cs.columbia.edu/~vatlidak/resources/POSIXmagazine...
People have been attacking POSIX almost as long as C/C++. But unsurprisingly no one has put forward anything that could replace it.
Given the fact that you don't seem to care that the OS or system libraries are written in C/C++. And you don't care that there are industries that produce C/C++ software that have nothing to do with mobile and more closely resembles system programming. Who exactly is the internet safety strike force trying to evangelize? And more importantly why do you feel the need to lecture developers in other industries on what language they should use even tho you have no experience or stake in those industries?
It just annoys people and makes them less likely to try the new meme language. And makes the internet safety strike force look like armatures with to much time on there hands.
You don't see C/C++ programmers running around the internet brigading functional programming languages with nonsense. Even tho most of those languages are total crap. Maybe we are to busy trying to write software. Maybe that's why everything is written in C/C++?
Second, my issue is with C not C++. C++ is one of my favorite systems programming languages, only spoiled by C copy-paste compatibility, which gets abused by C refugees using a C++ compiler.
I never cared for POSIX, because I always had access to rich C++ libraries, Turbo Vision on MS-DOS, OWL, MFC and ATL on Windows, Motif++ on UNIX and so on.
Windows 3.1 was already doing perfectly fine with C for Win32 SDK, with everything else being done in C++, Turbo Pascal, Delphi and VB.
It was the rise of UNIX, FOSS culture tied to C that changed the wind on C++ sails.
Thankfully Microsoft has decided that C++ is the only system programming language worth keeping and anyone that still wants to use C can keep using gcc and clang instead.
At least there is no place for C in UWP, unless for those that enjoy doing bare COM calls.
As for running around the internet, I guess you are too young, back in the day the C vs C++ flame wars were fought on BBS, USENET or even Teletext messaging boards.
Not everything is (thankfully) written in C, specially in the enterprise world where Java, .NET and C++ rule.
It is also very positive that ANSI C++ working group is turning C++ into a systems programming language with batteries, instead of relying on whatever OS APIs are available.
As for me, I have done my contribution to the world, by not writing a single line of C code since 2002.
The tools of trade are Java, .NET and C++.
One day C++ might get replaced by (Go, Swift, D, Rust, Ada), but currently C++17 as a helper language to Java and .NET, for my line of work, is good enough.
Of course by using RAII, no pre-processor beyond #include, std::string, std::vector, std::array, enum classes, references, templates, std::unique_ptr, , std::shared_ptr, and not a single line of C like coding other than as FFI to third party libraries.
You are taking code criticism to personally. Keep it professional.
> point me to this software that was rewritten from unsafe C code and was replaced with memory safe language.
From my own field - bunch of libraries for python. Mysql driver for example - used by large amount of people.
Numerous compilers and related tools: Golang runtime for example. Objective-C to Swift migration. Various parts of Firefox. Coreutils (https://github.com/uutils/coreutils). Ruby extensions: https://codeandtalk.com/v/fullstackfest-2015/full-stack-fest... remacs: https://github.com/Wilfred/remacs Whole industries run from C/C++ to Java/Python/.Net. Millions of projects there.
What? I am asking if what you are writing is based on your experience or it's just theory that you've read somewhere. How is this personal?
I am not asking for myself. I already know the answer for that question after reading your comments (the last one only confirmed that), but people that are reading your comment may not, so why don't you just answer the question ?
> Numerous compilers and related tools: Golang runtime for example. Objective-C to Swift migration. Various parts of Firefox. Coreutils (https://github.com/uutils/coreutils). Ruby extensions: https://codeandtalk.com/v/fullstackfest-2015/full-stack-fest.... remacs: https://github.com/Wilfred/remacs Whole industries run from C/C++ to Java/Python/.Net. Millions of projects there.
I am talking about projects that NEEDED C for it's time/space efficiency and cross platform support. Not any C project. Do you see OS kernels in production written in python used by millions ? Do you see video/audio codecs written in Java/python/.net used in production by millions? No. This is where C is used because it's needed.
Coreutils in rust, remacs, ruby extensions - this is not used by millions.
> Objective-C to Swift migration
What? This are different languages. It's not SAME software rewritten from C.
> Various parts of firefox
Whole software, not parts.
So the only valid example you gave is Go (Go compiler is not used by millions by the way). So you gave 1 example instead of 10 and Go compiler is now twice as slow after rewrite.
That '2000' number is not the relevant one here. The reason that security is important wrt Curl is that Curl is used to access the internet. Security in the library used to print colours to your terminal is a bit less important.
It's those widely used, public facing libraries that need a good security story (and using a memory-safe language is a good start there).
Printing colors still may provide an attack vector. Latest CVE in Ubuntu comes from a program that eject CD's (and needs root privileges to do so).
Yes, that's the point of this discussion. We are not criticising some code for having one bug a decade that requires 24 steps to reproduce, we are criticising code for having numerous bugs that are easily preventable with existing technology. Its way easier than it should be.
I have no personal need for rewriting curl. Instead I am using libraries written in safer language. You are misaddressing my criticism. I am not criticising curl author for using C - just some of his arguments for doing so the way he does. They are not valid in my opinion. There might be reasons for distributing code as C source, there is no reason for networking library to be written in C directly.
> In language that is memory safe + supports all the platforms that curl support
There are languages safer than C that compile to C, supporting all platform C does. No luck is needed.
I think you are living in theory world, but you don't have any practice. Show me 10 examples of C code (>10KLOC) that was generated by those "safe languages" that compile to C which is safe, as time and space efficient as equivalent hand written, highly optimized C code.
$ aptitude search '?tag(implemented-in::c)'|grep '^. lib'|wc -l
1895And every one of those three billion devices is vulnerable, due to the use of C.
> I feel a need to underscore the fact that none of the memory safe languages anyone would suggest we should switch to have been around for 19 years
False: Common Lisp has been around since 1994 (23 years), and in substantially the same shape for longer still. Standard ML has been around since 1990 (27 years). OCaml has been around since 1996 (21 years). Smalltalk has been around since 1984 (33 years). Each of those languages is more memory-safe than C and has facilities which help prevent other C-like errors. Each is capable of speeds approaching that of C, esp. for a problem like URl fetching (e.g. I just tried fetching http://www.google.com/ with both curl & DRAKMA — a Common Lisp package for URL fetching: curl reliably ran in about .065 seconds & DRAKMA reliably ran in about .14 seconds; I have no reason to believe that DRAKMA is particularly well-optimised; no doubt it could get even faster if desired).
I think this really is a textbook example of the Blub Paradox: someone using C thinks that for the most part it's a reasonable choice in order to achieve certain goals, while someone used to a better language is able to see that C is simply unfit for purpose: programs written in it will inevitably have security flaws which will inevitably cause harm — particularly when three billion devices, many unpatched, are running them.
$ time git clone https://github.com/edicl/drakma.git
Cloning into 'drakma'...
remote: Counting objects: 991, done.
remote: Total 991 (delta 0), reused 0 (delta 0), pack-reused 991
Receiving objects: 100% (991/991), 343.00 KiB | 0 bytes/s, done.
Resolving deltas: 100% (610/610), done.
Checking connectivity... done.
real 0m1.476s
user 0m0.148s
sys 0m0.060s
$ cd drakma
drakma $ find . -name '*.lisp' -o -name '*.asd' | xargs wc -l
868 ./request.lisp
107 ./encoding.lisp
122 ./read.lisp
281 ./specials.lisp
75 ./packages.lisp
123 ./test/drakma-test.lisp
333 ./cookies.lisp
63 ./drakma.asd
109 ./conditions.lisp
37 ./drakma-test.asd
357 ./util.lisp
2475 totalIt was already like that in the 90's.
When I learned C, I was using Turbo Pascal 5.5 and getting up to speed with newly released Turbo Pascal 6.0.
Besides the improved portability over Pascal dialects, I didn't saw any compelling feature over what Turbo Pascal was capable of.
You would have the same problems if you used an unsafe CL or rust implementation.
What we need is a meta-layer programming interface that isolates our higher thinking from lower language concerns to some degree. If you can keep the higher level understanding, then rewriting become less of a barrier. We need write pseudocode in every language.
I have been exclusively programming with a meta layer, MyDef, for quite a while, and I routinely re-write my Perl program to C, not without effort, but always a worthwhile effort trading for the goal of rewriting (speed in this case). I am a researcher, and I rarely write production code. But for production code, I think most software should consider rewriting in Rust.
Prototype in Perl/Python, develop in C, and produce in Rust -- makes good sense. It makes good sense in our current non-software industry, where researchers, developers, and producers are people with different focus and use different material and tools. This is not true for our software industry, where people who are not interested in researching or developing are forced to research and develop, and people who are only interested in research are forced to develop and people who only interested in developing are forced to research and produce.
My example: http://hz2.org/blog/send_more_money.html