Am I the only person who thinks there's something fundamentally wrong with computing if running "strings" could let someone take over your computer? (I'm not being snarky; I seriously think the whole approach to security need to be redone somehow.)
An alternative is to implement a subset of full parsing specifically tailored to each utility. In the case of strings(1) that's very simple; in the case of some other utilities it's of intermediate complexity; all the way up to some that need to parse every corner of ELF. Whether that produces a bigger or smaller attack surface depends on a lot of factors: each parser might be simpler, but there are many more of them. FreeBSD was contemplating centralizing more of that into a common libelf, so I don't think it's only GNU who think that's a good idea in principle: https://wiki.freebsd.org/LibElf
BSD tools are simpler. They're also generally less useful.
This is not, however, the actual goal or purpose of this tool. The fact that many people use Perl as nothing more than a slightly better version of sed doesn't mean that Perl's ability to write complex object-oriented software is "over-engineering". You just don't know what this tool is actually for, which is OK, but means you can't judge whether or not it is "over-engineered".
BSD, apparently going back to at least BSD 4.3, also had a strings tool, and it did the exact same thing: it parsed binary files to dump their strings table. Apple's strings tool has no code heritage from the GNU version, instead being a vague descendent of the one from BSD 4.3. This is how this tool has always worked: stop being part of the noise trying to turn this into a GNU-bash fest :/.
Not sure I agree with you on Perl in particular, but other than that I agree with what you're saying ;-) (That is, I don't think GNU being over-engineered is the (or perhaps even "a") problem here.
>> Am I the only person who thinks there's something fundamentally wrong with computing if running "strings" could let someone take over your computer?
No, the underlying library (libbfd) is an example of something that should've been fixed a long time ago. Maybe not quite the horror that was/is openssl -- but clearly an example of "old c code that sort of works" -- perhaps in some ways like bash was/is. It's old, it works, but it could use some clean-up (as evidenced by a number of related buffer under/over-flows and whatnot).
Note that parsing arbitrary binary (or otherwise) input safely, is a pretty hard problem. There was recently a (resource DOS) bug in libxml2, which have been under quite a lot of scrutiny lately (by virtue of being a brilliant injection vector for malicious code, if a bug can be found).
I read this as a two-part bug: one, a lot of people didn't know that strings did more complex parsing than a hexdump and a filter for printable strings (me included) -- and it turns out that the "smart" library isn't terribly robust (in other words: it's typical C code).
While it is possible (at least in theory) to write small C utilities that are safe, if you want them to be (wildly) portable, and have sane handling of various kinds of encoded strings, and encoded data, along with handling different endianess -- apparently most people screw up.
I think there are two basic camps wrt what should be done: those that think we need something like rust, so that we can have safety without much of a slowdown, and those that say screw it, we're no longer running on 5Mhz (or 50Mhz) cpus, we can take anything up to a 100x (10x) slow down without it really being an issue -- Security/stability/predictability is more important.
Those that can't decide between the two, continue writing C like it was still 1989, and we get lots of stuff like this.
I'm not sure if it's usually the mix of a "smart" C programmer writing a program that is patched by a "hobby" C programmer, or the fact that getting C right is just too hard -- or that people don't -Wall and don't run fuzzers and static checkers -- but whatever the reason, we keep seeing serious bugs in C programs.
I'd like to think some of it could be avoided if people wrote more "bloated" C with copious use of functions, more call-by-value, smaller loops, perhaps more computer generated code -- and other "slow" things (while still being C). But I'm probably hopelessly naive.
While I'm definitely not sold on C++ (at least not as viable "better" C for systems programming), I think the old 1998 article[1] by Strostrup on "simple" C and C++ programs illustrate quite well how hard C can be to get reasonably right, for even simple problems. Perhaps rather than waiting for Rust, a reimplementation on large parts of the backbone of our OSs/GNU in Guile, Lua, or some other higher-than-C level language could be worthwhile.
As a side note -- does anyone know of any follow up on Stroustrup's article?
I think the problem is that people don't take programming seriously. The basic flow of development seems to be to write the first thing that comes to mind as it comes to mind, and then incrementally patch things up until the code seems to work on every test case you've thought up.
This approach is at best inefficient when working with "safe" languages. With C or C++, it's nothing short of irresponsible. Part of the problem, I think, is that programmers are never taught to reason formally about their code: to catalogue their preconditions and postconditions and verify with some semblance of rigour that the code they write respects these. At best, they might be treated to a passing reference to object invariants if they happen to take a class or (God forbid) read a book on "OOP."
It's perfectly possible to write good, safe code in C and C++, but not if you're hung up on a "smart" or "hobbyist" programmer mindset, and not if you're not willing to put in a fair amount of effort into thinking before you write.
Some friends of mine are tackling this problem. You should help them. http://langsec.org
C should have been buried decades ago, it's a toy language for small pieces of code on isolated systems (and yes, I've written larger programs in C myself, 20 years ago).
Nowdays, C should not even be considered safe for implementing interpreters and runtime systems for other languages. There are plenty of reasonably portable choices available (and no, don't use Java / anything JVM based).
It can't be terribly hard to reimplement "strings" and similar software in a modern language without such deficiencies (i.e. plenty of ways to shoot oneself in the foot and overwrite the stack etc.).
So, there is a perfectly good reason C is still prevalent to this day, even if there are many security implications in doing so.
Here's a start: https://news.ycombinator.com/item?id=7882211 https://github.com/uutils/coreutils/tree/master/src
LISP is 50 years old. If it was going to become useful for high performance software in large scale distributed teams of development, it would have happened already.
We've only had Pascal and Modula-2 for what, 40-45 years?
>perfectly good reason C is still prevalent to this day
... it's not that and it's not good. It's just that security implications were largely ignored, people were lazy and innovation and making sound decisions (rather than popular ones) has never been strong in the OSS/Linux community (apart from the kernel itself).
It's a hobby horse by this point, but it makes it clear that the Pascal of 30 years ago is not the Pascal of today. The same can be said of C.
These languages have evolved to be where they are today, mostly because hardware has evolved to be where it is today. Castigating past decisions as laziness really seems to be ignorant of this process, what it involved, and why it was necessary to make the decisions that we have up until now.
Engineering, on the whole, is the art of compromise.
USCD Pascal was created in 1978, Mesa in the mid-70s and Algol in 1968.
Most of the complaints don't apply to them.
bwk's point, then as now, was that once you deviated from Standard Pascal, you had to either bet on a specific horse in the non-standard Pascal race, or develop your own Pascal-like language with its own inherent defects, similar to but different from the defects of Standard Pascal and other Pascal-like languages. Your code would likely never interoperate with anyone else's Pascal variant.
C, which had one de facto standard implementation and less genetic drift (because the standards, first K&R and then ANSI and ISO, were never as terrible as Standard Pascal's definition), didn't suffer from this nearly as much.
Mesa didn't, either, but only because it was fairly obscure, and there sure were a lot of Foogols running around for a while, of which Pascal and its semi-clones formed only one family.
The first K&R was published in 1978, ANSI/ISO C in 1989.
In 1981 C only mattered if you had access to a UNIX system, there were hardly any K&R compliant compilers.
Anyone using C outside a UNIX system was also betting on a specific horse.
Which is one of reasons why ANSI C has so much undefined and unspecified behaviours defined, as the committee didn't want to rule out any implementation.
No. Pascal and Modula-2 were perfectly usable, safe systems programming capable languages 20 years ago. Pascal was widely used in commercial product development (DOS/Windows), Modula-2 was scarce, but taught at universities with high quality development environments available for many platforms and the Oberon OS was based on the language Oberon, which was heavily influenced by/derived from Modula-2.
Everything necessary to produce safe, maintainable, fast software was available back then, but lazy/uneducated/stubborn people used C instead and wrote crappy software we still have to use today. It's a shame really.
I think your ad hominem comments about people who implemented in C are in poor taste and not based in reality.
It is still all the most horribly insecure and obtuse passing of raw pointer manipulation and bitwise logic ever. The preprocessor is still hell on software. Every variable is named something like gst_hello_world_parser_box because when you write complex software in it you always get huge name conflicts.
It still thinks the best way to manage memory is to not manage it at all. People complained for so long about having to put your deletes in destructors in C++, too bad you have no such thing in C at all - you don't have classes, after all. Guess it has to go in the procedural function logic. Like 30 years ago.
C was tied to UNIX, just as the others were tied to the OS of their vendor.
C only became portable after UNIX was available in a few American universities outside AT&T, some students decided to create workstations based on UNIX, while others started to develop C compilers, so that they could continue their work on other systems.
Now Algol, it had:
- Bounds checked arrays
- Reference parameters to functions/procedures
- Real strings (not the first version in 1960 though)
- Explicit conversions
Quote from Tony Hoare's ACM award article[1]:
"A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
[1]http://www.labouseur.com/projects/codeReckon/papers/The-Empe...
BCPL was used to write several OS's that were ported, though while BCPL predates C, I'm not sure if any of the OS's written using it predates Unix.
It'd be better if it wasn't C at all, sure, but, from a security perspective I'd take a language with a C runtime over a language where code has to be carefully written in a memory-safe manner any day.
I'd hope an LLVM compiler for a language would eventually be dogfooded, though.
This is pretty much the first thing that everyone writes in their new programming language.
Example: http://www.haskell.org/haskellwiki/Simple_Unix_tools
No, the problem comes from the other end. Strings should not be able to compromise a system. It only needs to read a user supplied file, and write to stdout. There's no reason for it to be allowed to exec a piece of code.
C is scary from a security perspective because it is both incredibly easy to write code that has subtle but very serious bugs leading to e.g. arbitrary code execution, and it is easy to exploit those vulnerabilities.
There are plenty of memory-safe languages in which you can do nearly everything you'd do in C, and much, much safer. There's no reason whatsoever a program like 'strings' couldn't be written in a memory-safe language.
Shellshock was a parsing bug, the memory safety of bash helped nothing.
Bugs are bugs. When you get an out of bounds exception that leaves your program in an inconsistent state somewhere halfway up the call stack in a code path with poor test coverage, "safe" is not the correct word.
What I'm saying is not that you can't write software that can be abused in memory-safe languages. I'm saying that you're much less likely to have extremely serious code execution vulnerabilities if you write a program in Go instead of C.
What I'm saying is that "much" less likely is overstating the difference. Most bugs in C programs are not buffer overruns, and even the ones that are would still be bugs in Go or Rust, they would just be a different kind of bug which is still plausibly exploitable under real conditions.
This is not a silver bullet kind of situation. "Rewrite everything in Go" is not actually a fix -- it's replacing code with 30 years worth of bug reports and vulnerability testing with completely untested new code, at the cost of significant resources that could better be used to fix the remaining vulnerabilities.
I'm not even saying that all the existing code is perfect. Replacing OpenSSL with entirely new code would probably do more good than harm just because the existing code is so ugly. But that's the exception rather than the rule.
> Most bugs in C programs are not buffer overruns, and even the ones that are would still be bugs in Go or Rust, they would just be a different kind of bug which is still plausibly exploitable under real conditions.
I don't follow. What's the equivalent of forgetting to check the length of an input string before chugging it into a too-small array in Go?
> "Rewrite everything in Go" is not actually a fix
I never said that, though. I'm not suggesting we rewrite the GNU tools in Go. But if you were to write the GNU tools from scratch today, C would be a bad choice simply because it's so easy to slip up with devastating effects, and there aren't many advantages to using it for simple tools like 'strings'.
I feel like "<most other languages> bugs" tend to get ignored because it's not popular to blame <language> for bugs unless <language> is C. For example an enormous number of high severity CVEs are SQL injection but I never hear anybody saying we should replace SQL with a binary interface that clearly distinguishes statements from data, even though that would make more difference in practice than replacing C with something else.
> I don't follow. What's the equivalent of forgetting to check the length of an input string before chugging it into a too-small array in Go?
In Go the program terminates, which is at best a denial of service vulnerability. If the program is anything in the nature of Fail2ban then just causing it to die is a serious problem. Meanwhile when it restarts it will have to somehow deal with whatever corrupted state the crash left behind, which depending on the context can provide the attacker with opportunities to do arbitrarily bad things by manipulating the state to be something the programmer never anticipated. Being able to induce a restart is a huge increase in attack surface.
Immediate program termination is the "take cyanide capsule" solution to serious bugs. It may be better than some of the alternatives but it's still very bad.
Using bind parameters instead of putting data directly in queries + escaping has been standard for a long time now. That is, instead of saying "make sure to escape everything", like "make sure to avoid any bugs in C code", we indeed prefer to switch to a technique which which doesn't act pathologically in the presence of small errors. In SQL's case it doesn't require replacing the whole language. It technically doesn't have to in C, either - you could have bounds checked C - but I guess once you give up the absolute-maximum-performance goal, people prefer to use different languages.
edit: Also, while program termination is not ideal, in many cases, such as this one (strings), it is basically a non-problem, and at worst, denial of service is still loads better than arbitrary code execution.
If that causes a serious problem, then the architecture is bad and unsuitable for solving that problem (remember: programs can also die because of OOM conditions, flakey hardware, admin errors. If this leaves a gaping security hole open for an attacker, or lead to DoS, then you need a better approach; fail2ban functionality should be in the process handling logins for example, not a separate entity watching logfiles, which might be broken too).
> Being able to induce a restart is a huge increase in attack surface.
Compared to what? Certainly not C code, where very common issues are easily exploitable for arbitrary code execution.
I don't even use SQL, but I've heard a similar mantra plenty enough times to internalize it.
"but one thing is clear: you should never, ever use concatenated SQL strings in your applications. Give me parameterized SQL, or give me death." ( http://blog.codinghorror.com/give-me-parameterized-sql-or-gi... )
This is an API level change, not a protocol level change - but an API change is the correct answer anyways. If you change the protocol and slap an API like the SQL injection prone ones on top of it, you'll have the same vulnerabilities no matter what the protocol.
For bonus points, use static analysis to catch and forbid query strings that can't be trivially proven static.
> For example an enormous number of high severity CVEs are SQL injection but I never hear anybody saying we should replace SQL with a binary interface that clearly distinguishes statements from data, even though that would make more difference in practice than replacing C with something else.
I see 3x more memory corruption CVEs alone, than SQLI CVEs. I'm not convinced that replacing SQL (or SQL APIs) would be higher impact than replacing C and C++. And since I actually use C and C++ in my daily bread and butter, they're significantly more relevant to me.
But SQLI may be significantly more relevant to you. If that's the case, by all means, focus on them more.
Because I am sure if had ported bash to Go, you would still have the same issue with the broken parser. I am not sure if you would still have had heart bleed, but I know you wouldn't have had heart bleed if the openssl people had used the platform libc, instead of rolling their own, so I wouldn't consider heart bleed an issue with C, but an issue with the programmer.
If you haven't seen it already, you might want to check out HiStar:
Code that is unsafe in this way is impossible to write in most languages.
I've done most of my OS-level hacking in C and a limited subset of C++. I think it works well down there, where resources can't be "magic" and it's really important to know what's going on (and you often have to tell the optimizer "hands off").
At higher levels, it's lunacy to be using C. I sure wouldn't write a compiler in one, for instance.
You sure about that? If it was a re-implementation in non-cleanroom conditions, Microsoft likely don't have that right.
Even if that's not the case, I'm curious what commercial gain Microsoft believe they would have be keeping it private.
One wonders how much other code of public interest is hidden behind closed doors (and not just Microsoft's).
As for the provenance of the code, it's quite probable it was a clean-room reimplementation. Writing such tools isn't exactly rocket science. I wrote a PDB-oriented addr2line for VC++ a few years ago, and it didn't have any of the original addr2line code in it at all. In fact, I've never even looked at the addr2line code. I just ran addr2line --help and copied the command line options I saw there. I suspect a reimplementation of strings would be just as straightforward.
This was driven by Microsoft's hostile policies towards running anything open source inside the company (you have to get special dispensation to install Linux, for instance). Actually looking at outside source code is a big no-no.
There has been at least one case of a contractor including GPL3 code in a project, and MS responded by (a) letting the contractor go, and (b) releasing the source for the product in question. [And no, it wasn't Windows]
It's certainly not the right solution for everything, but there are plenty of cases where I'd think it could be an excellent choice. Most of Android (except the kernel) is written in Java, as an example.
The amount of C++code decreases with each release.
For example, the version 8 of the OpenJDK had quite a few code rewritten in Java, thanks to the work introduced with invokedynamic.
There are plans to eventually replace Hotspot with Graal and SubstrateVM, in some future version of the OpenJDK, thus reducing even more the C++ surface.
This is why project Sumatra is using Graal and now jRuby is also playing with it.
And as other commenters note the amount of C++ decreases each release because its less and less useful to have any code in the JVM as C++.
In graal calls to the OS are going to more limited in surface area than in Hotspot C2.
For all reasons removing a class of errors e.g. buffer overflow leading to executable code is going to give more secure code.
Not that I think Java is a silver bullet that will solve all problems. I do think that a language like Java or Rust is a major improvement over C with minimal performance issues.
Is it really so bloody hard to ensure that e.g. the fields of the executable header, if they're offsets, are actually valid values? I've written tools that work with PEs, many of them reading the entire file into a buffer first, and "make sure you're inside the file" was one of the points I always kept in mind.
I say we need to fix how programmers think, not the language, because the same mindset that leads to bugs like these in "unsafe" languages will also lead to (maybe less severe, maybe more severe but also more subtle) bugs in "safe" languages too.
Citation needed. The plethora of over overcomplicated C code is a strong argument against this claim.
There certainly is plenty of overly complicated C code, but I doubt there is more of it than there is the amount of overly complicated code written in some of the other popular "safe" languages: Java and JavaScript.
I agree, but unfortunately, changing the language or replacing it, is a lot easier than changing the way programmers think.
Why not? There are meta-circular implementations of Java, with native code generation on their toolchains.
Even OpenJDK is going to get a AOT compiler as discussed on JVM Language Summit 2014.
It is common to blame C for being unsafe but the real issue seems to be trusting user input, without sanitising it first. You could use a typed language to enforce sanitising by having a special type for user input (whether read from a file or received from the system) and have the converters sanitise it.
If you are disregarding Java when you are looking for portable options, then you are IMHO limiting yourself quite a bit and could be missing out on a good solution to your problem.
It's also a basic problem that so much complicated systems software is written in C, but that's better known.
Knowing which programmer signed the code is nice, writing stuff in a "more secure" language is nice, but sandboxing beats it all.
It does attempt to parse a.out format, but any weird values in the header won't cause reading/writing beyond the end of a buffer.
Solution? In my opinion, either don't bother parsing any binary formats (who actually needs that functionality?), or use a safe language.