Some dark corners of C
docs.google.com
docs.google.com
My main take-away from this is that Google Drive seems like a nice way to put presentations online :-)
Don't. I've been trying to access the presentation for 10 minutes and it won't allow me:
Wow, this file is really popular! Some tools might be unavailable until the crowd clears.
#define struct union
#define else
But, apart from that, it's a really nice compilation. I didn't know about the compile time checks of array sizes, but I have a doubt. What if I pass to a method declared
int foo(int x[static 10])
int* x = (int*) calloc(20, sizeof(int));
EDIT: Funnily enough, in Mac it doesn't give any warning, neither for pointers nor for undersized arrays (ie, foo(w[5]) doesn't give a warning). And I've compiled with -std=c99 -pedantic -Wall.
Edit: While we're talking about dark corners, please stop casting functions that return void * . If your code lacks the declaration of the function, the compiler will assume pre ANSI-C semantics and generate code returning an int.
On machines where pointers do not fit ints (basically all 64bit machines), you just silently (due to the cast there is no warning) truncated a pointer. Worse, it may work depending on the malloc implementation and how much memory you allocate.
We have to fix these kinds of bugs on OpenBSD a lot, please help by typing less and let the compiler warn you about silly mistakes :-)
And yes, C++ fucked this up for C. I'll leave it to Linus to say something nice about that..
btw; I wrote this talk.
And by the way, nice talk, it's great learning these dark secrets of C.
= Bitfields =
Not even once.
;)
int b:1;
Thanks, I will take that into account.
People tended to confuse C++ with C more in the past, as they had not diverged as much as they have now.
struct a { int a;}
struct b { int b;}
struct c : a, b { };
c myc;
c* p1 = &myc;
b* pb1 = p1; // correctly points to myc's base b, which is offset
void* p2 = &myc;
b* pb2 = p2; // Would not point at instance of b, compile error
#ifdef __cplusplus
#define STATIC_CAST(T, EXPR) static_cast<T>(EXPR)
#else
#define STATIC_CAST(T, EXPR) (EXPR)
#endif
Better, please help by compiling your C code with -Wimplicit-function-declaration (included in -Wall), and fixing all the problems it reports. Then you won't have to worry about this problem, or a bunch of other problems.
-Wall -Wextra -Werrortypedef void ( * fptr_t)()
and cast to fptr_t instead of void * But it's bigger problem how to force users to cast it back to proper prototype, because casting it back to something else will give UB.
Wow! Ugly! Scary! Another good reason to know in fine details just what cast does. At one point, my Visual Basic .NET code actually calls some old C code, and in time I will need to convert to 64 bit addressing. So, I will keep in mind that with 64 bits I have to be especially careful about pointers and C.
It's a pretty easy thing to leave out, missing a required header, but if you always compile under -Wall it'll catch this and many other problems as well.
The problem is, if I want my C code to compile with MSVC, it has to compile as C++ - and even if I abhor Windows for development myself, a lot of developers are using MSVC.
I just wish Microsoft would update their C compiler, at least to C90. But then I suppose the standard has only been around for 23 years, and nobody really uses C anyway.
As someone who has compiled a lot of .c files that do not cast void pointers with cl.exe, I'd say no, this is not true... Maybe what you're trying to say is that your code relies on C99 features that are also present in C++? IMO if that's what's holding you back it's much easier to just write C89, maybe with the occasional ifdef, than to suddenly write some crummy C/C++ hybrid. That or just use mingw. (Unless you're doing SEH. I'm not aware of a good way to do SEH with mingw.)
I think casting void pointers is very much the lesser of two evils. And of course I can use MinGW, but that doesn't mean the majority of Windows developers don't insist on using MSVC.
[0] http://en.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%2B
Definitely writing C99 in that I compile the exact same source files with -std=c99 -Wall -pedantic when using a real C compiler.
I don't think this is as big a deal as you're making it out to be. Maybe I'm just used to it, but this is one area where I consider the C89 way to be better on stylistic grounds. (I wrote about this on a stack exchange site some time ago: http://programmers.stackexchange.com/questions/75039/why-dec...)
Realistically I don't think C99 is all that revolutionary, and the fact is there are plenty of people writing C89 in 2013. Let's look at some things C99 adds over C89:
* Mixed declarations and code
IMO not a huge deal, for reasons I give above.
* VLAs
A nice feature, but people have been using malloc for similar purposes for ages, so it's hard to say it's any better than "nice to have". I'd also argue that something that leads me to potentially consume arbitrary amounts of stack space based on runtime decisions is probably not universally good, and I'm pretty cautious about using VLAs even when given the choice.
* Last field of a struct can be variable sized array, eg. struct foo { int num_data; int data[]; };
I like this pattern and have used it a lot. Including with VC++ by having a zero-sized array at the end of the struct. This is once place where I use an ifdef to bridge the gaps because recently I noticed LLVM started doing weird things for this if you don't do it the "legit" C99 way.
* Cosmetic issues like // comments
VC++ already supports this in C89 files as a non-portable extension.
* Portable typedefs like uint32_t, etc.
These are also pretty handy. But if you're building with VC++ you're probably using the non-portable Microsoft ones anyway. (I think VC++ 2010 added the C99 versions too.)
* bool type
Again, every platform and even reasonably sized library works around this with a typedef and some macros.
* Slightly different behaviors for some libc functions
I notice this most for snprintf, where the pre-C99 return value [followed by Microsoft] means something different [and better] in C99. Again there are non-portable solutions for this that you can use ifdefs for.
* Named initializers for struct members
Cool feature. I'd hardly say I couldn't live without it, though. The only place I've really seen it used extensively is in the Linux kernel.
* Stuff that no one uses, such as "static" as used in these slides, or complex numbers, or whatever.
Not a problem since these don't see wide use.
Then there are things like the portable integer types - which you decided aren't necessary because everyone is writing Win32 code (???), the designated initializers which I use every day to make my struct initializing code safe from changes in the struct itself, for loops, a consistent bool type ... heck, I could go on all day if you hadn't already dismissed the rest of the standard as "stuff that no one uses".
I really don't mean to come across as aggressive, but this kind of ignorant and dismissive attitude is so rampant in the tech scene at the moment. And it's getting us nowhere.
And then after all, you initially explained the need for this in terms of Windows-specific code (or at least that's how I read it), so that's the angle I took...
Sound static analyzers fall in the first case, but require a lot of work to become precise enough to be used (ie, to reduce the number of false alarms). Compilers fall in the second case in the sense that they don't have to honor such a clause. And in the C99 norm it's actually a "shall" (it just couldn't honor a "must" in that case):
"If the keyword static also appears within the [ and ] of the array type derivation, then for each call to the function, the value of the corresponding actual argument shall provide access to the first element of an array with at least as many elements as specified by the size expression."
You can not have a general procedure, but with the help of the programmer / user of the compiler, you can prove all kinds of things.
Both of them would be a real pain to debug.
#define memmove memcpy
#define strncmp memcmp
#define rand fork
#define free(x)
#define strerror(n) strerror((n)+1)
#define memset(addr, byte, len) memset(addr, ~(byte), len)You can redefine keywords in your code, but they must not be defined when including standard headers (for obvious reasons).
I don't believe C has any such restrictions, though.
int foo(int x[static 10])
The real purpose is to allow the compiler to optimise the compilation of the foo() function itself, under the assumption that x will always point to the first element of an array of at least 10 elements.
"... if the keyword static also appears within the [ and ] of the array type derivation, then for each call to the function, the value of the corresponding actual argument shall provide access to the first element of an array with at least as many elements as specified by the size expression."
char main[] = { 0xf0, 0x0f, 0xc7, 0xc8, 0xc3 };
My gcc compiles it with only this warning:
foo.c:2:6: warning: ‘main’ is usually a function [-Wmain]
[23] .got.plt PROGBITS 0804954c 00054c 000014 04 WA 0 0 4
[24] .data PROGBITS 08049560 000560 000010 00 WA 0 0 4 <---
[25] .bss NOBITS 08049570 000570 000008 00 WA 0 0 4
66: 0804840a 0 FUNC GLOBAL HIDDEN 14 __i686.get_pc_thunk.bx
67: 08049568 5 OBJECT GLOBAL DEFAULT 24 main <---
68: 08048278 0 FUNC GLOBAL DEFAULT 12 _init
unsigned char code[] = { 0xf0, 0x0f, 0xc7, 0xc8, 0xc3 };
int main(void)
{
((void (*)())code)();
return 0;
}
char main[] __attribute__((section(".text")));
(You get a warning from the assembler.)
I didn't know gcc attributes included that kind of thing. I've really gotta dig though the manual some time.
int x = 'FOO!';
I'm sure, though, that someone sooner or later will be bitten by code like
int x = 'é';
int x = 'RIFF';
.. if you were packing a WAVE file header.
int x = 'A';
Nevermind, it would take more than the Lord of the Rings triology.
I remember hearing that the disallowal of pointer aliasing was the main reason that it was possible for a Fortran compiler to produce code that could outperform code from a C compiler: It allows the compiler to perform a new class of optimizations.
It would appear that the restrict keyword lets C programs regain that class of compiler optimizations.
So... not that telling then.
http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-de...
string create_function ( string $args , string $code )
"Creates an anonymous function from the parameters passed, and returns a unique name for it."
So there you are. Of course you can choose to be an anonymous value, but you'll get a name assigned by the state, for free. :-) Fascinating logic, captain.
create_function was a way to
A) not having to define a function separately
B) not getting problems with the function being re-defined, since each call would create a new function
C) fake closures by generating code.
All this should be moot points by now, since PHP has real anonymous functions with closures,
But I suppose that goes naturally hand in hand with the cargo cult approach to language design.
That's why it would be a fake closure.
$foo = someNumber();
create_function('', 'return '.$foo.';');
> talks casually about "returning the name of an anonymous function". It shows just how much twisted the logic of these people is.
I think you read too much into this. The point of an anonymous function isn't to make it not have a name, but to be able to define it where it is needed instead of referring to some specific function name in your code.
It also fits well with the way PHP handles "pointers", by storing the name of a variable in another variable.
$foo = 42;
$bar = 'foo';
print($$bar); // Prints 42.Actually, its point is to make it a value that can be referenced from any number of bindings (associations between names and values) in any number of scopes. What you're saying is just a consequence of this.
"It also fits well with the way PHP handles "pointers", by storing the name of a variable in another variable."
Which only shows the deficiency, since any such indirect reference should never refer to a name, but to a binding instead.
But about those dark corners, I guess the point wasn't to present any particularly nasty gotchas, but rather some precious little lesser known tricks. C has plenty of very well known features you can be bitten by (mostly related to memory management, of course). While the presentation reiterates over some of them, the most valuable parts are about various _good_ parts of the language which are rarely heard of (viz. the usage of `static` inside brackets).
Of the C "dark corners" that are problematic, it'd be extremely rare to run into them in most real-world code. You'd have to intentionally go out of your way to write code that will trigger them, and this code often looks obviously suspicious.
It's very much the opposite with JavaScript and PHP. A world of pain and danger opens up the moment you do something as simple as an equality comparison. The problems that can and will arise are well documented, so I won't repeat them here, but it's a much worse (and unavoidable) situation than when compared to C, C++, Java, C#, Python, Ruby or other mainstream languages.
The automatic conversions in JavaScript and PHP seem pretty harmless by comparison.
But "restrict" is a low-level micro-optimization, those tend to be tricky. I don't think a sane C programmer would sprinkle that keyword all across the source base, because as you have pointed out it can cause hard-to-diagnose errors.
In contrast, the automatic conversions in JavaScript and PHP are an "always on" feature you cannot avoid.
This allows, for example, compilers to replace a `memcpy()` call that has a constant size argument with direct loads/stores.
Author got this wrong, that would be an empty file, which is what won the IOCCC for smallest self-replicating program once.
http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp...
(Sorry for of offtopic)
You can actually get a firm grasp of the basics just by reading chapter 3 from Computer Systems: A Programmer's Perspective (http://csapp.cs.cmu.edu/public/samples.html) and practice writing some simple command line programs.
First, what are malloc() and free() doing? That is, what are the details, all the details and exactly how they work?
It was easy enough to read K&R, see how malloc() and free() were supposed to be used, and to use them, but even if they worked perfectly I was unsure of the correctness of my code, especially in challenging situations, expected problems with 'memory management' very difficult to debug, and wanted a lot of help on memory management. I would have written my own 'help' for memory management if I had known what C's memory management was actually doing.
'Help' for memory management? Sure: Put in a lot of checking and be able to get out a report on what was allocated, when, by what part of the code, maybe keep reference counters, etc. to provide some checks to detect problems and some hints to help in debugging.
That I didn't know the details was a bummer.
It was irritating that K&R, etc. kept saying that malloc() allocated space in the 'heap' without saying just what they meant by a 'heap' and which I doubt was a 'heap' as in heap sort.
Second, the 'stack' and 'stack overflow' were always looming as a threat of disaster, difficult to see coming, and to be protected against only by mud wrestling with obscure commands to the linkage editor or whatever. So, I had no way to estimate stack size when writing code or to track it during execution.
Third, doing data conversions with a 'cast' commonly sent me into outrage orbiting Jupiter.
Why? Data conversion is very important, but a 'cast' never meant anything. K&R just kept saying 'cast' as if they were saying something meaningful, but they never were. In the end 'cast' was just telling the type checking of the compiler that, "Yes, I know, I'm asking for a type conversion, so get me a special dispensation from the type checking police.".
What was missing were the details, for each case, on just how the conversion would be done. In strong contrast, when I was working with PL/I, the documentation went to great lengths to be clear on the details of conversion for each case of conversion. I knew when I was doing a conversion and didn't need the 'discipline' of type checking in the compiler to make me aware of where I was doing a conversion.
Why did I want to know the details of how the conversions were done? So that I could 'desk check' my code and be more sure that some 'boundary case' in the middle of the night two years in the future wouldn't end up with a divide by zero, a square root of a negative number, or some such.
So, too often I wrote some test code to be clear on just what some of the conversions actually did.
Fourth, that the strings were terminated by the character null usually sent me into outrage and orbit around Pluto. Actually I saw that null terminated strings were so hopeless as a good tool that I made sure I never counted on the null character being there (except maybe when reading the command line). So, I ended up manipulating strings without counting on the character null.
Why? Because commonly the data I was manipulating as strings could contain any bytes at all, e.g., the data could be from graphics, audio, some of the contents of main memory, machine language instructions, output of data logging, say, sonar data recorded on a submarine at sea, etc. And, no matter what the data was, no way did I want the string manipulation software to get a tummy ache just from finding a null.
Fifth, knowing so little about the details of memory management, the stack, and exceptional condition handling, I was very reluctant to consider trying to make threading work.
Sixth, arrays were a constant frustration. The worst part was that could write a subroutine to, say, invert a 10 x 10 matrix but then couldn't use it to invert a 20 x 20 matrix. Why? Because inside the subroutine, the 'extents' of the dimensions of the matrix had to be given as just integer constants and, thus, could not be discovered by the subroutine after it was called. So, basically in the subroutine I had to do my own array indexing arithmetic starting with data on the size of the matrix passed via the argument list. Writing my own code for the array indexing was likely significantly slower during execution than in, say, Fortran or PL/I, where the compiler writer knows when they are doing array indexing and can take advantage of that fact.
So, yes, no doubt as tens of thousands of other C programmers, I wrote a collection of matrix manipulation routines, and for each matrix used a C struct to carry the data describing the matrix that PL/I carried in what the IBM PL/I execution logic manual called a 'dope vector'. The difference was, both PL/I and C programmers pass dope vectors, but the C programmers have to work out the dope vector logic for themselves. With a well written compiler, the approach of PL/I or Fortran should be faster.
It did occur to me that maybe other similar uses of the C struct 'data type' were the inspiration for Stroustrup's C++. For more, originally C++ was just a preprocessor to C, and at that time and place, Bell Labs, with Ratfor, preprocessors were popular. Actually writing a compiler would have permitted a nicer language.
Seventh, PL/I was in really good shape some years before C was started and had subsets that were much better than C and not much more difficult to compile, etc. E.g., PL/I arrays and structures are really nice, much better than C, and mostly are surprisingly easy to implement and efficient at execution. Indeed, PL/I structures are so nice that they are in practice nearly as powerful as objects and often easier and more intuitive to use. What PL/I did with scope of names is also super nice to have and would have helped C a lot.
Eight, the syntax of C, especially for pointers, was 'idiosyncratic' and obscure. The semantics in PL/I were more powerful, but the syntax was much easier to read and write. There is no good excuse for the obscure parts of C syntax.
For a software 'platform' for my startup, I selected Windows instead of some flavor of Unix. There I wanted to build on the 'common language runtime' (CLR) and the .NET Framework. So, for languages, I could select from C#, Visual Basic .NET, F#, etc.
I selected Visual Basic .NET and generally have been pleased with it. The syntax and memory management are very nice; .NET is enormous; some of what is there, e.g., for 'reflection', class instance serialization, and some of what ASP.NET does with Visual Basic .NET, is amazing. In places Visual Basic borrows too much from C and would have done better borrowing from PL/I.
Everything is about tradeoffs. Fortran uses space-padded strings with no null terminator. On the positive side, this forces everyone to explicitly pass the length they mean instead of relying on more work at runtime to figure out when to stop by looking for the null sentinel. Passing explicit lengths is good practice in C anyway because you usually avoid having to scan the contents multiple times / multiple calls to strlen at different levels in the stack. While everything should be better in the Fortran case, the class of bugs that persist are even more hard-to-find bugs because poorly written code mis-calculates the length, ignores it, etc., stomping over adjacent memory. This probably won't crash, and since other code has to use an explicit length when accessing the buffer, you usually won't notice the problem at the source of the issue. Contrast that with C, where you're more likely to see an issue immediately as soon as the string is used or passed to something else.
tl;dr Poor programming is poor programming in any language.
With PL/I the maximum length of the string is set when the string is allocated, usually dynamically during execution. The length can be given as a constant in the source code or set from computations during execution. There is also a current length <= the maximum length. When passing that string to a subroutine, the subroutine has to work a little to discover the maximum string length, but, by in effect 'hiding' both the current and maximum length from the programmer of the subroutine, the frequency of some of the errors you mentioned should be reduced.
In Visual Basic .NET, the maximum length of any string is the same, as I recall, 2 GB. Then having the strings be 'immutable' was a cute approach, slightly frustrating at times but otherwise quite nice and a good way to avoid the problems you mentioned.
But, of course, the way I actually used strings in C was close to the way they were supported in Fortran.
And, of course, likely 100,000+ C programmers wrote their own collection of string handling routines where use a struct to keep all the important data on the string, say, allocated or not, pointer to the allocated storage, maximum allocated length, current length, etc. (multi-byte character set anyone?) and then pass just a pointer to the struct instead of a pointer to the storage of the string; in this way, again should reduce the frequency of some of the errors you mentioned.
I've written some assembler in the machine language of at least three different processors. On one machine I was surprised that my assembler code ran, whatever it was, 5-8 times faster than Fortran. Why? Because I made better use of the registers. Of course, that Fortran compiler was not very 'smart', and smarter compilers are quite good at 'optimizing' register usage. I will write some assembler again if I need it, e.g., for
R(n+1) = (A*R(n) + B) mod C
where A = 5^15, B = 1, and C = 2^47. Why that calculation? For random number generation. Why in assembler? Because basically want to take two 64 bit integers, accumulate in two registers the 128 bit product, then divide the contents of the two registers by a 64 bit integer and keep the 64 bit remainder. Due to the explicit usage of registers, usually need to do this in assembler.
But at one point I read a comment: For significantly long pieces of code, the code from a good compiler tends to be faster than the code from hand coded assembler. The explanation went: For longer pieces of code, good compilers do good things for reducing execution time that are mostly too difficult to program by hand which means that the assembler code tends to be using some inefficient techniques.
REF INT i = HEAP INT; # sort of like C++ "new" # REF INT i = LOC INT; # allocates from the stack #
or the shorter forms
HEAP INT i; LOC INT i;
Since heap is the word used in heap sort, it's fair to say that the second use of that word was a misuse. I don't know which use was second and don't really care but did want to know the details of the dynamic memory allocation used by the C malloc() and free(). I just would have appreciated an explanation of malloc() and free() were doing so that could write some code, as I described, to 'help' me monitor what my code was doing with memory. Sure, now writing a good system for 'garbage collection' complete with reference counts and memory compactification is difficult, but what malloc() and free() were doing was likely not very tricky. I just wish K&R had documented it.
Maybe that's what was being used in the common versions of C. If so, then for whatever reason I missed out on that. I kept seeing in the book where they kept saying that malloc() allocated storage in the 'heap' without being clear on what they meant by a 'heap' although in this thread is an explanation that 'heap' was also used in Algol 68. Whatever, when they said 'heap' with no explanation, they blew me away.
Once I was one of a team of three researchers that did a new language. Eventually it shipped commercially. We needed a lot in dynamic storage allocation. Our approach was to start with an array of pointers, say, for i = 1, 2, ..., 32 or some such, s(i) was a pointer to the start of storage for chunks of size 2^(i-1) + 1 to 2^i. So, allocate 10 bytes of storage from 16 bytes where 16 = 2^i for i = 4. That is, i = 4 handles requests of size 9 through 16. Etc.
So, right away at the start of execution for relatively large j, have the operating system allocate a block of storage of size 2^j. Then for i < j, if need a block of storage for allocations handled by i, get that from storage handled by i + 1, etc. up to j where actually get some storage. That is, if i = 4 needs storage, get that from i = 5 that handles requests up to size 32 = 2^5.
For each i, the allocated blocks are chained together in a linked list and so are the free blocks. So, for an allocation, look first at the end of the linked list of free blocks.
In principle, after enough uses of, call it, free(), could return some storage for i to the storage for i + 1 but we didn't bother doing that.
It always seemed to me that on a virtual memory system where the page size was a power of 2 (aren't they all?), this approach to dynamic memory allocation would be quite good.
Later I was using an old version of Fortran, got a big block of storage from the operating system as just an array (right, as a common block known to the linkage editor), and wrote code such as above to have versions of malloc() and free().
If what is there in K&R in 8.7 is what was actually being used in the versions of C I used, then I blew it by not writing some code at least to report on storage allocated, freed, 'fragmentation', when allocated, etc. Basically I was highly concerned that in a relatively complicated program with just malloc() and free() I would make some mistakes in storage allocation and get some bugs that gave symptoms only occasionally and that would be a total pain to diagnose. "Last Tuesday after running for four hours we got some strange data in the file and then it blew up." Great! It reminds me of one of those arrangements of a few thousand dominoes on edge where when one tips over they all go, a house of cards, an electric power system with no circuit breakers, a dense city built with no fire safety codes, etc.
Even today, the GNU C compiler (which is what I mostly use) has non-standard extensions to malloc()/free() that allow you to obtain information that is otherwise not mentioned in the C Standard (GNU defines mtrace() and malloc_hook, for instance, which trace each allocation, and allow you to peek into malloc).
From your descriptions, it sounds like you wrote C back in the 70s or 80s. It's changed a bit since then.
For the C documentation, I recall only one point: Due to all the different addressing options on x86, the Microsoft C compiler had a crucial but nearly secret command line switch we needed. I found the switch only by some strange efforts. It was not in the documentation. None of the documentation I had was much beyond just K&R.
You seems to have found some that works well for your needs so everything is good.
The last time I had to write some C, I just refreshed my C 'skills' with K&R and reading some of my old code.
For your
"You seems to have found some that works well for your needs so everything is good."
I agree: I looked at Java early on and didn't like it. From some of the comments and links here at HN, I see that Java has made progress since then. Indeed, some of what I like in Visual Basic .NET (I say ".NET" because there is an earlier version of Visual Basic that is quite different and less 'advanced') seems to have come from Java. So, now I'm glad to have the progress of Java and/or Visual Basic .NET and will return to C only when necessary.
Actually, the last time I worked with C, I wrote only a few lines of it! Instead, I took some Fortran code, washed it through the famous Bell Labs program f2c (apparently abbreviates 'Fortran to C') to translate to C, slightly tweaked the C, compiled it into a DLL, and now call it from Visual Basic .NET.
Maybe what will be waiting for me in the lower reaches is C programming on an early version of Unix without virtual memory and without a good text editor on a slow time sharing computer using a B/W character terminal, 24 x 80!
It's particularly useful for bugs related to memory.
So, someone else dug into the details of how C manages memory and wrote some code to help people find problems; makes good sense.
But how C programs behave regarding memory is well known and understood. However, it does require understanding basic memory concepts related to the operating system itself.
On your
"K&R and other good C references describe their public interface well and that's all you need to know to use them effectively."
I want more. By analogy, all you need to drive a car is what you see sitting behind the steering wheel, but I also very much want to know what is under the hood.
Generally I concluded that for 'effective' 'ease of use', writing efficient code, diagnosing problems, etc., I want to know what is going on at least one level deeper than the level at which I am making the most usage.
Your example of putting a 100,000 byte array on the stack is an example: Without knowing some about what is going on one level deeper, that seems to be an okay thing to do.
2) My remark about the stack is either not quite correct or is not being interpreted as I intended. For putting an array on a push down stack of storage, I am fully aware of the issues. But on a 'stack', maybe also the one used for such array allocations (that PL/I called 'automatic'; I'm not sure there is any corresponding terminology in C), there is also the arguments passed to functions. It seemed that this stack size had to be requested via the linkage editor, and if too little space was requested then just the argument lists needed for calling functions could cause a 'stack overflow'. A problem was, it was not clear how much space the argument lists took up.
Then there was the issue of passing an array by value. As I recall, that meant that the array would be copied to the same stack as the arguments. Then one array of 100,000 bytes could easily swamp any other uses of the stack for passing argument lists.
But even without passing big 'aggregates' by value or allocating big aggregates as 'automatic' storage in functions, there were dark threats, difficult to analyze or circumvent, of stack overflow. To write reliable software, I want to know more, to be able to estimate what resources I am using and when I might be reaching some limit. In the case of the stack allocated by the linkage editor for argument lists, I didn't have that information.
3) Sure, I could make use of the strings in C as C intended just as you state, just for textural data, but also have to assume a single byte character set.
I thought that that design of strings was too limited for no good reason. That is, with just a slightly different design, could have strings that would work for text with a single byte character set along with a big basket of other data types. That's what was done in Fortran, PL/I, Visual Basic .NET, and string packages people wrote for C.
The situation is similar to what you said about malloc(): All C provided for strings was just a pointer to some storage; all the rest of the string functionality was just in some functions, some of which, but not all, needed the null termination. So, what I did with C strings was just use the functions provided that didn't need the null terminations or write my own little such functions.
As I mentioned, I didn't struggle with null terminated strings; instead right from the start I saw them as just absurd and refused ever to assume that there was a null except in the case when I was given such a string, say, from reading the command line.
It has appeared that null terminated strings have been one of the causes of buffer overflow malware. To me, expecting that a null would be just where C wanted it to be was asking too much for reliable computing.
3) On casts, we seem not to be communicating well.
Data conversions are important, often crucial. As I recall in C, the usual way to ask for a conversion is to ask for a 'cast'. Fine: The strong typing police are pleased, and I don't mind. And at times the 'strongly typed pointers' did save me from some errors.
But the question remained: Exactly how are the conversions done? That is, for the set D of 'element' data types -- strings, bytes, single/double precision integers, single/double precision binary floating point, maybe decimal, fixed and/or floating, and for any distinct a, b in D, say if there is a conversion from a to b and if so what are the details on how it works?
One reason to omit this from K&R would have been that the conversion details were machine dependent, e.g., depended on being on a 12, 16, 24, 32, 48, or 64 bit computer, signed magnitude, 2's complement, etc.
Still, whatever the reasons, I was pushed into writing little test cases to get details, especially on likely 'boundary cases', of how the conversions were done. Not good.
Sure, this means that I am a sucker for using a language closely tied some particular hardware. So far, fine with me: Microsoft documents their software heavily for x86, 32 or 64 bits, from Intel or AMD, and now a 3.0 GHz or so 8 core AMD processor costs less than $200. So I don't mind being tied to x86.
On PL/I: Thankfully, no, it was not nearly the first language I learned. Why thankfully? Because the versions I learned were huge languages. Before PL/I I had used Basic, Fortran, and Algol.
PL/I was a nice example of language design in the 'golden age' of language design, the 1960s. You would likely understand PL/I quickly.
So, PL/I borrowed nesting from Algol, structures from Cobol, arrays and more from Fortran, exceptional condition handling from some themes in operating system design, threading (that it called 'tasking' -- current 'threads' are 'lighter in weight' than the 'tasks' were -- e.g., with 'tasks' all storage allocation was 'task-relative' and was freed when the task ended), and enough in bit manipulation to eliminate most uses of assembler in applications programming. It had some rather nice character I/O and some nice binary I/O for, say, tape. It tried to have some data base I/O, but that was before RDBMS and SQL.
In the source code, subroutines (or functions) could be nested, and then there were some nice scope of name rules. C does that but with only one level of nesting; PL/I permitted essentially arbitrary levels of nesting which at times was darned nice.
Arrays could have several dimensions, and the upper bound and lower bound of each could be any 16 bit integers as long as the lower was <= the upper -- 32 bit integers would have been nicer, and now 64 bit integers. Such array addressing is simple: Just calculate the 'virtual origin', that is, the address of the array component with all the subscripts 0, even if that location is out in the backyard somewhere, and then calculate all the actual component addresses starting with the virtual origin and largely forgetting about the bounds unless have bounds checking turned on. Nice.
A structure was, first-cut, much like a struct in C, that is, an ordered list of possibly distinct data types, except each 'component' could also be a structure so that really was writing out a tree. Then each node in that tree could be an array. So, could have arrays of structures of arrays of structures. Darned useful. Easy to write out, read, understand, and use. And dirt simple to implement just with a slight tweak to ordinary array addressing. So, it was just an 'aggregate', still all in essentially contiguous, sequential storage. So, there was no attempt to have parts of the structure scattered around in storage. E.g., doing a binary de/serialize was easy. The only tricky part was the same as in C: What to do about how to document the alignment of some element data types on certain address range boundaries.
Each aggregate has a 'dope vector' as I described. So, what was in an argument list was a pointer to the dope vector, and it was like a C struct with details on array upper and lower bounds, a pointer to the actual storage, etc.
PL/I had some popularity -- Multics was written in it.
For C, PL/I was solid before C was designed. So, C borrowed too little from what was well known when C was designed. Why? The usual reason given was that C was designed to permit a single pass compiler on a DEC mini-computer with just 8 KB of main memory and no virtual memory. IBM's PL/I needed a 64 KB 360/30. But there were later versions of PL/I that were nice subsets.
It appears that C caught on because DEC's mini computers were comparatively cheap and really popular in technical departments in universities; Unix was essentially free; and C came with Unix. So a lot of students learned C in college. Then as PCs got going, the main compiled programming language used was just C.
Big advantages of C were (1) it had pointers crucial for system programming, (2) needed only a relatively simple compiler, (3) had an open source compiler from Bell Labs, and (4) was so simple that the compiled code could be used in embedded applications, that is, needed next to nothing from an operating system.
The C pointer syntax alone is fine. The difficulty is the syntax of how pointers are used or implied elsewhere in the language. Some aspects of the syntax are so, to borrow from K&R, 'idiosyncratic' that some examples are puzzle problems where I have to get out K&R and review.
To me, such puzzle problems are not good.
I will give just one example of C syntax:
i = j+++++k;
Right: Add 1 to k; add that k to j and assign the result to i; then add one to j. Semi-, pseudo-, quasi-great.
I won't write code like that, and in my startup I don't want us using a language that permits code like that.
anyway, if you think you can prevent bad code by using restrictive languages, you're gonna have a bad time. Any language can be abused. Just don't abuse it, treat your code with respect.
Also I'm pretty sure j+++++k has undefined behavior so you should be shot if you write it.
> in my startup I don't want us using a language that permits code like that.
Well I hope you don't run unix or windows, python or ruby, Firefox, chrome, ie, safari, or opera, or use a smartphone.
"No other language will reliably run on many systems that fast with that much existing code." Yup, and just such reasons are why at times I used it. It's in effect also why I'm using some of it now although mostly my code is in Visual Basic .NET (VB): The Microsoft VB documentation is fairly clear ('platform invoke' or some such) on how to call C from VB. Well, I have some old Fortran code I want to call from VB, do have the old Watcom Fortran compiler, but do not have details on how to call Fortran from VB. So, I used the old Bell Labs program f2c to convert the Fortran to C, used the Microsoft C compiler to compile and link to a DLL, then call the DLL from VB. And actually it works. And in effect the reason I can do this is what you said: C is so popular, for the reasons you gave, etc., that Microsoft went to the trouble to say how to call C from VB. Microsoft didn't do that for Fortran, PL/I, Algol, Ada, etc. You are correct that the popularity of C is important.
"anyway, if you think you can prevent bad code by using restrictive languages," Right. Each such restriction eliminates only some cases of bad code.
> Any language can be abused. Just don't abuse it, treat your code with respect.
Right. There is "When a program is written, only God and the programmer understand it. Six months later, only God." Well, so that I could read my code six months later, I wrote only easy to read code. So, I would write
n = n + 1
and not the short version, and would never write i+++++j.
> > in my startup I don't want us using a language that permits code like that.
> Well I hope you don't run unix or windows, python or ruby, Firefox, chrome, ie, safari, or opera, or use a smartphone.
You lost me: I'm using VB and find nearly all the syntax to be fine, that is, easy to learn, read, and write. And the main reason I'm not using C# is what it borrowed from C/C++ syntax. I'm using C only when really necessary. Sure, I use Windows and Firefox; if they are written in C/C++, that's their issue. But by staying with VB, I am successful with my goal of
> in my startup I don't want us using a language that permits code like that.
> Also I'm pretty sure j+++++k has undefined behavior so you should be shot if you write it.
As I recall, I actually tried it once, and it compiled and ran as I explained. And, as you explain, if it works in one heavily used C compiler, then it should work the same in all of them. If look at j+++++k, I suspect that it parses according to the BNF just one way with no ambiguity. So, don't have to write, say,
(j++) + (++k)
"The C specification says that when there is such an ambiguity, munch as much as possible. (The "greedy lexer rule".)"
So j+++++k turns into:
j++ ++ + k
Which is clarified on the next slide.
I would have guessed that j++ ++ was not legal syntax.
So, I was wrong: There are two ways to parse that mess. So, there is ambiguity. And the way they resolve the ambiguity is their 'greedy' rule! Wow!
Net, that tricky stuff is too tricky for me.
There was a famous investor in Boston who said that he only invests in companies only an idiot could run well because the chances were too high that too soon some idiot would be running the company.
Well, I want code, or at least language syntax, that any idiot can understand, for now, me, and later some of the people that might be working for me!
You are way ahead of me on C, and you leave me more afraid of it than I was. But then I was always afraid of it and, in particular, never wrote ++.
So, my first issue was the statement for C
i = j+++++k;
Platform: Windows XP SP3 with recent updates. And apparently somehow I have
Visual Studio 2008 x86 32 bit
I don't use IDEs or Visual Studio and, instead, apparently as a significant fraction of readers at HN, write code with my favorite text editor (e.g., KEdit) and some command line scripts (using ObjectRexx, which is elegant but for better access to Windows services, etc. likely I should convert to Microsoft's PowerShell).
So, I typed in some C code and tried to compile it. Then I encountered again one of the usually unmentioned problems in computing: Software installation and system management. Several hours later I had a C/C++ 'compile, load, and go' (CLG) script working, but my throat was sore from screaming curses at the perversity of 'system management' -- a project of a few minutes with a prerequisite of several hours of system management mud wrestling.
For the mud wrestling, the first problem was, since my last use of C, I had changed my usual boot partition from D to E. Next the version of C installed on E was different from that on D. And the installation on D would not run when E was booted. Bummer.
Next, the C compiler, linker, etc. want a lot of environment variables. Fine with me; generally I like the old PC/DOS idea of environment variables.
However, apparently Microsoft was never very clear on just what software, when, could change the environment variables where. At least I wasn't clear.
So, booting from my partition E, the C/C++ tools want environment variables set as in
E:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools\vsvars32.bat
If run the BAT file from a console window, it changes the environment variables as needed by C/C++. But, in console windows I run a little 'shell script' I wrote in ObjectRexx. I has a few nice features for directory tree walking, etc. But when run the BAT file from the command line of a console window that is running my little shell script, after the BAT file is done and returns, the environment variables have been restored to what they were before running the BAT file. If use a statement, say,
set >t1
So, sure, there is a 'stack' of invocations of processes, applications, or whatever in the console window and its address space, and, somehow, since my shell script was in the stack, when the BAT file quit the stack and its collection of environment variables was popped back to what they had been.
But eventually I relented, gave up on this little project taking just a few minutes, slowed down, thought a little, read some old notes, discovered that I should change the environment variables within my ObjectRexx script, using an ObjectRexx function for that purpose, as needed by C/C++ CLG, found the needed changes, implemented them, and, presto, got a C/C++ CLG script that works while my shell script is running and while I am booted from my drive E.
On to the C question:
For 'types', the test program has
int i, j, k;
i = j+++++k;
i = (j++) + (++k)
i = j+++++k;
Not now! With the C/C++ tools with
Visual Studio 2008 x86 32 bit
i = j+++++k;
error C2105: '++' needs l-value
So, it wasn't clear how the compiler was parsing. So, I tried
i = j++ ++ +k;
error C2105: '++' needs l-value
So, I tried
i = (j++)++ + k;
error C2105: '++' needs l-value
i = j++ + ++k;
Then I tried
i = j+++k;
So, I long concluded that to understand some of the tricky, sparse syntax of the language, not clearly explained in K&R, have to write and run test cases as here. Bummer. But, as below, here I'm significantly wrong.
Possible to make sense out of this?
Maybe: If start reading
Brian W. Kernighan and Dennis M. Ritchie, 'The C Programming Language, Second Edition', ISBN 0-13-110362-8, Prentice-Hall, Englewood Cliffs, New Jersey, 1988.
in "Appendix A: Reference Manual" on page 191, then hear about 'tokens' and 'white space' to separate tokens.
Okay, no doubt + and ++ are such 'tokens'.
Continuing, right away on page 192 have
"If the input stream has been separated into tokens up to a given input character, the next token is the longest string of characters that could constitute a token."
I would have said "up to and including a given input character", but K&R are 'sparse'!
So, with this parsing rule, in
j+++k
j
++
+
k
(j++) + k
j+++++k
j
++
++
+
k
(j++)++ + k
So, my remark that
j+++++k
Basically I was assuming a 'token getting' parsing rule like I've implement a few times in my own work: There are tokens and delimiters, and a 'token' is the longest string of characters bounded by delimiters but not containing a delimiter. The delimiters are white space, (), etc.
K&R seems to have a point: My parsing rule would have trouble with just
j>=k
j >= k
Generally, though, the C syntax is sparse and tricky, so tricky it stands to be error prone.
Back to writing Visual Basic .NET.
My personal view is that C offers a perfect tradeoff between simplicity and capability, it has a magical quality that has made it the most important single computer language for nearly 40 years and on into the forseeable future. Increasingly its importance is as a layer that more programmer friendly technology sits upon, but it's no less important for that.
I've read that the difference between chess and go (the oriental game, not golang) is that on Alpha Centauri if little green men play a game that resembles chess, they will almost certainly play a game identical to go. Go is simple enough it is almost inevitable. For me it's almost the same thing (I stress almost), with computer languages and C.
One final point; C syntax is ultimately a matter of taste. If you find this to be a completely obvious, correct and straightforward way of doing a non-overlapping C string copy;
while( *src )
*dst++ = *src++;
*dst = '\0';
Yes, for your code example, I 'get it'! It's cute! No doubt it's cute.
So, to get 'full credit', dst and src are 'strings', that is, essentially just pointers. Since C pointers have a data type with a length, here the data type of these pointers is byte, or character, or some such with length 1 byte.
Starting off, since src points to a C string that obeys the C standard of null termination, we know that the string has exactly 1 null byte, its last byte. So, for any byte except the last one, it is not null. So, if the string has length more than 1, then as we enter the While, src points to the first byte of the string (or at least the first byte we want to copy); * src is that byte; and * src is not null, that is, is not 0, that is, tests as True for the While (need to know that 0 tests as false and anything else tests as true). So we execute the statement following the While.
That statement says copy byte * src to byte * dst and, then, before considering the semi-colon and before collecting $100, increment both src and dst by the length of their data types, that is, by 1 byte. So, now src points to the next byte in its string and dst points to the target of the next byte in its string. Then we return to the While and do its test again. If we have more bytes to move, then we just rinse and repeat the above. Else src points to the last byte of its string which has to be the null byte so that the byte itself, * src, is the null byte, that is, 0, that is, False in the While statement. Then we leave the While statment, that is, move past its semi-colon. So, net, the While statement moves all the non null bytes we want moved. Of course when get to the next statement, dst points to the last byte of its string, that is, the byte that is to be its null byte (just why that byte is not null already is possibly an issue). At any rate, we want that last byte to be the null byte, so that last statement so assigns that last byte.
For more, 'src' abbreviates 'source' and 'dst' abbreviates 'destination'.
So, yes, it's possible to describe this stuff in English.
So, there's a problem, a significant problem: I 'documented' your code. Okay, but your code is for a string copy. There should be some documentation, but it should be in documentation of a string operation for the language. That is, even just for the documentation, the strings and the copy operation need to be 'abstracted' to a higher level where they can be documented and learned there and set aside the need to document in the code.
Of course, I would write such a copy loop, assuming I wanted to use a loop, using Fortran's Do-Continue, PL/I's Do-End, VB's For-Next. I don't recall the Algol syntax. The PL/I syntax is the same as in Rexx which I use heavily.
Of course in PL/I and VB, I would use the substring function instead of a loop.
And I would fear that too much usage of syntax as sparse as this example here would be more error prone. And for more complicated operations, I would fear that neither God nor the programmer understood the code. I can understand that on some processors with some compilers, that C syntax could lead to faster code, but I'm not thrilled about digging into x86 assembler enough to be sure. Also now it's tough to know what fast code is due to out of order execution, speculative execution, parallel execution, pipelines, three levels of cache, the cache set associative, and cache line invalidates when have several cores. But, computers are so fast now I don't much care, and if I did care I would notice that making such code faster would not make the code in the runtime library or the operating system faster and, thus, might not be able to do much for actual performance of my application.
I don't really mind your example; it's actually not sparse enough to be a serious problem. But I'm not thrilled by the example because I don't take pleasure in that clever sparsity and, again, I'm afraid that it could result in bugs that could hurt my business.
Then there's the issue, I regard as significant, that with that code the C compiler is very short on the 'meaning' of what is being done. Or, you and I can look, read, guess, and agree that we are doing a copy of the tail of one C string to another (or possibly the same if initially src equals dst) string. Okay, you and I can guess that. But the poor compiler can't, and if it tries then I might get torqued when in that loop actually I'm not working with C strings but doing something else. So, then, the compiler will have a heck of a time checking string lengths and not writing on the wrong storage. So, I'd rather have strings as a higher level construct, than just a pointer to some storage from malloc(), so that the language can help me debug my code. I've programmed so many errors in array bounds and string lengths that I don't want to be without some good checking at runtime, at least in some 'debug' mode.
Next, for being 'fast', on at least IBM 360/70, etc. computers, actually that code sample would be slow! Why? Because that instruction set has a single instruction to copy all the bytes in a range of sequential virtual addresses. So, if the compiler knows it is working with a string and knows it is compiling for that instruction set, then it can replace the whole loop with just one instruction.
There was an old remark in the IBM PL/I program logic manual on execution speed: People could complain about PL/I being slower than Fortran. But if PL/I were used at all carefully, then it was faster than Fortran, and one of the main reasons was that PL/I then but not Fortran had strings in the language. So, Fortran programmers wrote collections of string handling functions/subroutines, and the internal logic was much as in your example, move one byte at a time. And as in the standard C library, do this by calling a function/subroutine with its overhead. PL/I's compiled code for strings was in-line and blew the doors off anything in Fortran. For today, and for similar reasons, Visual Basic .NET has a chance to be faster in string manipulations than C code such as in your example. Further, it's super tough to do something with VB strings that would mess up memory.
I don't find your example "a confusing monstrosity", but I greatly prefer to bet my business on VB instead of C/C++.
By little example is basically the strcpy() standard facility. Maybe a better example would be a construct that (roughly) could replace memcpy();
while( n-- )
*dst++ = *src++;
The equivalent to my memcpy() snippet on the original x86 machines was simply this;
rep movsb
But for an entrance exam, it's slightly hardcore :)
Unless you know the actual parsing rules, there's no way to know if a real parser would be greedy or not (or perhaps it might try to be clever?). This is nothing more than a trivia question, which does not test aptitude or intelligence.
I expect they asked some other questions too.
I had no idea how the C parsing algorithm worked for +++ et al, and I'm an expert C programmer. Then again, I'd also never use such ridiculous constructs in production code.
Even if you don't know that much about the subject, you can still have an interesting reasoning about it. Seeing that it is ambiguous is already a good observation. You then can propose way to resolve the ambiguity and touch (willingly or not) upon the topics of operator precedence, associativity, greedy matching.
Those topics are not only relevant in parsing either, for instance associativity is an important concept for list operations such as folding (a right fold is different from a left fold).
By the way, shouldn't the right hand side text on slide 7 (the final part of slide 7) talk about the pointers z and x, instead of the values pointed at? (Aside: How do I write "asterisk x" on HN without getting an italicized x?)
Took me awhile to understand this; single quotes define single characters, and for some C decided to allow multiple character character constants but leave their value as implementation-defined. Discussion: http://zipcon.net/~swhite/docs/computers/languages/c_multi-c...
Lots more ambiguities in C++. But a challenge to find them in C. My favorite: [] are just '+'
I don't know why the author chose to change the syntactic structure of the loop though, since it hides the point.
You have to be careful when counting down though. If you're accessing an array, you might be tempted to do this:
for(size_t i = bar_len - 1; i >= 0; --i) {
foo(bar[i]);
}
Overall, the presentation is very weak, like from a yesterday's graduate.
By the way, I'm an expert C/C++ programmer, so my opinion matters.
Sure, the other stuff was either UB or less interesting to experienced C programmers, but I'm not sure why that should be a problem. If you already know all of these, then you're not the intended audience. You can comment from a more experienced position, or just move along.
Bragging about knowing all of those and even worse - claiming they are just silly UB, all in a condescending tone as dakimov did, is a very stupid thing to do.
I've got quite a bit of experience with C, and I haven't heard of the "static" array size feature before, which seems extremely useful.
So I am glad it was posted, it helped, me, and this comment page was also, something to both smile/laugh and learn something from. Thanks
Screw this 'community'. It sucks ass.
Also, the design of this site is awful. And the engineering skills of the Mr. PG The Greatest apparently suck as well.