Hacker News Reader

Comment by oconnor663 | Hacker News Reader

My favorite take on the severity of the UB problem, https://blog.regehr.org/archives/1520 :

> Tools like [Valgrind] are exceptionally useful and they have helped us progress from a world where almost every nontrivial C and C++ program executed a continuous stream of UB to a world where quite a few important programs seem to be largely UB-free in their most common configurations and use cases...Be knowledgeable about what’s actually in the C and C++ standards since these are what compiler writers are going by. Avoid repeating tired maxims like “C is a portable assembly language” and “trust the programmer.” Unfortunately, C and C++ are mostly taught the old way, as if programming in them isn’t like walking in a minefield.

View on HN

> Be knowledgeable about what’s actually in the C and C++ standards

... turns out to be an extremely tall order; the 2017-11-17 working draft of the C++ standard is 1,448 pages in PDF format.

At some point, I wonder when programmers who are required to care about correctness throw up their hands and say "This language isn't reasonably human-sized for a user to know they're using it correctly." At which point the immediate next question should be "Why don't you use something else?"

(In my experience, the only sane counter-answer to that last question is "I'm programming on an architecture where the {C/C++} compiler is the only one with any real expressive power that anyone has ported to this physical system." This is an answer I understand; most other answers will raise an eyebrow from me).

> I wonder when programmers who are required to care about correctness throw up their hands

I did that over 20 years ago and have never looked back.

> the 2017-11-17 working draft of the C++ standard is 1,448 pages in PDF format.

To be fair, that contains both language documentation and library documentation. The language specification portion itself is only ~400 pages, which is smaller than the Java Language Specification and JS's specification (both about 700-800 pages, although JS does include its [meager] standard library in there).

When it comes comprehensibility, the C++ standard is orderS of magnitude more difficult than the JLS due to the numerous interdependencies and layers upon layers of cruft. There is no single, cohesive model to understand. That is the fatal flaw of C++ and I guess what led to Stroustrup's couple-of-decades-too-late "Remember the Vasa!" proclamation.

As someone that has been using Java, .NET, C++, OpenGL, DirectX, Win32, Linux and Web since their early days, you will never have a cohesive view in any of them, specially if not being there when that decision X was discussed in the community and there are no written traces of why it was taken that way.

C++ should only be used as a last resort. It's complex, development is slower, the tooling is poor, some universities have stopped teaching it. It's very hard to hire good C++ devs, and they are typically not cheap.

I manage a c++ team btw. Our app has to be fast.

As your parent said, this is also an answer I can understand.

Still, have you guys looked at alternatives -- and seriously evaluate them? Rust, Zig, Nim, D, others?

If you tell me "we can't afford to, we have too much work" then that's also a valid answer (for a while at least).

The bespoke libraries etc have god knows how many man hours invested, it would be possible but gargantuan.

> C++ has a standard that's too long and hard to understand, so let's use a language without any standard whatsoever instead!

Good lord, if that's the kind of human capital that's involved in making next-generation languages, then I have no doubt what will still be used to write serious software in 2080.

(Hint: not Rust.)

You'd be more helpful if you specified what else would you deem appropriate -- as opposed to resorting to snarky sarcasm that brings nothing interesting to the discussion except supposedly degrade people whose tech choices you disagree with.

The phrase 'undefined behavior' only makes sense in the context of an ISO standard.

For any particular combination of compiler, OS and computer architecture there are no 'undefined behaviors'; you only care about the concept if you want portability to a different compiler/OS/architecture.

So the idea that C++ sucks because of 'undefined behavior', so let's use some random language without an ISO standard or any portability guarantees at all is completely and utterly insane.

Our alternatives are Java and .NET languages, with native bindings to C++ libraries.

C++ alternatives still need to grow up to this kind of mixed language development, where I can have .NET code with C++ libraries and easily debug across them on the same Visual Studio session.

Same applies to Java and C++ development experience.

Not sure how fair to newcomers (as in languages) this requirement is.

F.ex. when coding in Erlang/Elixir I can use an excellent bridging library between their VM and Rust. There's also an excellent support for working with C libraries. Not sure what else can be expected.

Maybe I am not reading you correctly and I apologise if so. It just kind of sounded like "the newer languages must be compatible with 20+ different ABI standards if they want us the older programmers working in C/C++ to adopt them"?

Back on the original topic, I completely understand if a project has too much baggage and sunk cost so as to make even a partial migration (and hardening via using memory-safe languages and/or professional paid code analysis tools) unrealistic.

Naturally it isn't fair, but one cannot expect teams to drop velocity and diminish their productivity only on basis of adopting a new language that doesn't support existing workflows, IDE tooling and libraries.

It is like telling someone to delivering games in Unity and Unreal that they should use Amethyst instead, without understanding what it means to actually having a team working in those ecosystems.

Naturally Rust will get there, however it took C++ 30 years to be where it is today, and that is what any replacement attempt should take into consideration.

This is probably a much larger and not very relevant to the topic here discussion but I feel a lot of the newer breed of languages don't emphasise IDE tooling very much (not in the sense of Visual Studio / Eclipse and the like at least; VS Code gets a lot of attention though). So I fear if that's an expectation from a certain group of devs that this might already be a generational difference that's impossible to overcome.

That being said, I agree -- existing teams can't be expected to just run away to the new thing, that much is true.

Depends where you look, Swift, Kotlin and TypeScript live from their tooling.

My favourite language -- Elixir -- is the same. I also quite admire Rust's and Golang's tooling.

CLI tooling is the best it ever was in history (from where I stand at least). But IDE tooling isn't a first class citizen for many languages these days, is what I was saying.

I beg to differ, I rather enjoy languages that pursue the dream from Xerox PARC than working like when I was in high school during the late 80's.

Just for reference, my first UNIX was Xenix.

Don't see the point why people buy expensive computers to just use them like we did at the university lab with IBM X terminals.

The main difference was that instead of Slack we had xterms dedicated to run talk sessions.

What are you comparing it to? I think C++ tooling is fairly decent. (By the way, what do you work on?)

I think java has very good tooling.

Java's main issue is the barrier to entry is lower because it has good tooling, which means really bad, inexperienced programmers can write bad code, and have more tooling to bail them out, allowing them to continue down that path until they have a huge ball of mud.

There ought to be some kind of phrase for this "the tooling curse".

GDB isn't bad, IMHO, but the inability of IDEs to deterministically parse templates / index code without huge problems is problematic.

> the inability of IDEs to deterministically parse templates / index code without huge problems is problematic.

More like the inability of text editors to parse templates; most IDEs rely on a full compiler backend that can usually figure things out.

But it shouldn't require a full compilation.

The ultimate result, given the slow compile times of c++, is that you cannot know in your editor if your code is correct.

The IDEs I have used, or IDE-ified editors, run compilation ("indexing", or whatever) in the background to make this work.

Maybe you want to share which IDEs you've been using? It sounds like a wonder if there is one that can handle C++ in a way comparable to, say, IntelliJ for Java, or VSCode for TypeScript.

My experience is frankly more that the usual IDEs I know of can't even handle syntax coloring 100% correct when it comes to C++…

Xcode for well-structured (read: Xcode) projects. Sublime Text with clangd for random editing (not an IDE, but it gets good autocompletion and syntax checking from it).

This is problematic on large code bases. Huge mistakes were made on C++ design.

How large are we talking about? I've been able to use these tools on codebases such as WebKit and LLVM…

LLVM has 2.5 million lines of code, but I think it also depends upon the number of templated methods etc, not just LOC.

I'm surprised you haven't seen this point made many times. It would be a full-time job on the internet asking people why they have code completion issues with C++ IDEs.

No, I know what you're talking about; this is an issue in general if you'd not using tooling that's aware of the ins and outs of C++. The point is that the IDEs I have used essentially run the compiler on the file, so they can actually understand what the templates are doing and get through them.

Templates can inherently not be understood. It's duck typing 3 degrees of freedom with 3 unknowns. You need concepts which adds 1 known into the equation, (the final missing degree of freedom is the dreaded ctrl-click-into-an-interface-instead-of-the-concrete-class which is hard to avoid). Tell me one IDE that will give any meaningful information about buzz in the example below:

    template<typename T>
    void foo(T bar) { bar.buzz(); }

Other than that, finding foo<T> is usually easy, i've had good experience with qtcreator or clangd, you just have to be very very sure that your include paths are correct, qtcreator is good in that it uses CMakeLists as project-files so all this is done automaticaly.

I completely forgot about this aspect of C++. I remember being surprised the first time I tried compiling a C++ library on my workstation and found I couldn't do so because my workstation ran out of memory. The library was Panda3D; I can't remember what kind of workstation I was running on at the time.

It turns out if you use enough layers of templates, the compiler has to carry around an awful lot of state to figure out what program it should actually output.

This is another downside to a 1400 page language specification; whether the user can hold most of it in their head or use a reliable, safe subset to avoid sharp edges, the compiler and tools still have to be aware of all of the layers of complexity in both the code a developer is writing and often whatever tricks and quirks developers chose to usein the libraries the developers code is depending on.

Not in my experience. IntelliJ can't even figure out where a make_shared call goes.

Does IntelliJ actually support C++?

Probably talking about CLion, their C/C++ offering.

Yes, CLion, my mistake, sorry.

Hmm, that's a bit disappointing…might I suggest trying out a LibClang-based IDE, if CLion isn't using it already? It can do wonders even to "stupid" applications. For example, my Sublime Text (by itself, with only basic knowledge of C++ keywords and such) with clangd tells me that std::make_shared comes from this code in <memory>:

  template<class _Tp, class ..._Args>
  inline _LIBCPP_INLINE_VISIBILITY
  typename enable_if
  <
      !is_array<_Tp>::value,
      shared_ptr<_Tp>
  >::type
  make_shared(_Args&& ...__args)
  {
      return shared_ptr<_Tp>::make_shared(_VSTD::forward<_Args>(__args)...);
  }

Oh no, what i mean is that it can't help me find the constructor of the thing i'm creating. Resolving a make_shared call to the implementation of make_shared is completely useless; the only thing you ever want to do is find the constructor it's forwarding to.

Lots of languages suffer there -- eg there's no reason you _couldn't_ be a really excellent PHP developer who specializes in WordPress sites, but it's an uphill battle to prove you're a good dev because there so many bad ones in that space.

For that matter, most of the so-so devs resumes that come across my desk will list "HTML and CSS" as skills. Even among experienced devs, being legitimately good with HTML or CSS is extremely rare. I'd give my eye teeth to have a dev on staff who was an actual HTML expert (knows the w3c docs, can write clear, correct, semantic html using the right tags, organizes and expresses their intent, etc.), but since the barrier to entry is barely above basic literacy, so finding one is tough.

I find modern c++ tooling to be great compared to even 10 years ago. I love the language. Between it and python I stay very busy with both.

Wouldn't something like Rust meet your performance concerns?

The sunk cost is huge. My main point was: if you don't have to do it in c++, for the love of god do it in something like java. Yes java is not heaven on earth, but still...

> most other answers will raise an eyebrow from me

Portability. A C library can be trivially linked with any other language. But one will be hard pressed using a python library form Ruby, for example.

Indeed. C++ is relatively portable too (or at the very least, can be made to hide behind a C ABI in many cases) and it seems like Rust is as well. Whereas, if you have a library written in python that you want to leverage, you're forced to include a python interpreter. This sucks.

Julia lets you run Python, R and Fortran libraries.

Only if you have a complete Python/R environment installed.

Inter-linking is a superpower, as is having a very thin runtime that ships with most operating systems.

I realized recently that Python could have saved a decade by having a means to "interlink" modules allowing the use of python 2 modules in python 3. Much easier in the C world; there have been incompatible syntax changes and incompatible link changes BUT not both at the same time.

Embedding Python 2 inside of Python 3 (or vice-versa) is not very hard to do on Linux. Simply `dlmopen` the .so in a new linker namespace and write a little bit of bridging code to interface the two object layouts.

Python has a very rich runtime, so there are some tricky problems to solve if you want this to be perfect. e.g., circular references between the GCs. However, there are simplifying assumptions that can dramatically reduce the difficulty, and these assumptions might not significantly hinder the language-upgrade use-case.

We've known about this approach to embedded interpreters for a couple of years, but we've not found a sufficiently compelling use-case for anyone to develop it beyond a simple proof-of-concept.

In general, it seems like there most compelling use-case for inter-language interfaces is writing core libraries in something fast, (mostly) runtime-free, and portable, especially for codes that no one wants to write twice.

Though I spend most of my time writing codes in slow, rich runtime languages, I've been on the lookout for a better technology to replace C. There are quite a few interesting options, and some have even less of a runtime than (dynamically-linked) C!

> Embedding Python 2 inside of Python 3 (or vice-versa) is not very hard to do on Linux. Simply `dlmopen` the .so in a new linker namespace and write a little bit of bridging code to interface the two object layouts.

Where can I pip install this from?

Or is there a reason that none of the huge numbers of python users delaying their transition as much as possible until all their libraries were updated built it?

> Where can I pip install this from?

A couple of signatures have changed, but the general approach looks like: https://gist.github.com/dutc/eba9b2f7980f400f6287 or https://gist.github.com/dutc/2866d969d5e9209d501a

The above will launch ("embed") a Python 2 or Python 1.5 interpreter from within a Python 3 interpreter. If you use `dlmopen` and `LM_ID_NEWLM`, the guest interpreter will have its own linker namespace. In other words, the guest interpreter (and any DSOs it opens) will be totally isolated from the host interpreter.

The above shows the use of `PyRun_SimpleString`. Since `PyRun_String` accepts a `PyObject* globals` and `PyObject* locals`, you could build a very basic bridge in <30 lines of code by using serialisation to copy and convert. (For user-defined classes, you would need something better.)

I've given a many talks about this at various Python and PyData conferences. (I've mentioned it a number of times on HN, both in response to complaints about the Python 2→3 transition and in response to comments like yours suggesting this solution to that problem.)

Though presented as a joke, the approach could be made to work with some effort. I can't speculate on why no one has ever followed-up on it, and I can't speculate on why the Python 2→3 transition has been so difficult for some users. Perhaps in some places, financial or organisational arguments are more influential than technical arguments.

This is literally the reason I most recently wrote some C, but - do you really need linking? You can pretty easily use a Python library from Ruby by writing a little loop in Python that accepts JSON input and produces JSON output, and calling that as a Ruby subprocess.

It's fairly rare that you actually need to be in the same process. (The thing I wrote was a wrapper for unshare(), so it did strictly need to be in-process, but that's an unusual use case.) Even if you want to share large amounts of data between your process and the library, you can often use things like memory-mapped files.

And there are some benefits from splitting up the languages, including the ability to let each language's runtime handle parallelism (finding out that the C library you want to use isn't actually thread-safe is no fun!), letting each language do exception-handling on its own, increased testability, etc.

> You can pretty easily use a Python library from Ruby by writing a little loop in Python that accepts JSON input and produces JSON output, and calling that as a Ruby subprocess.

If you're a Ruby application, sure, you can use a Python library by forking off a subprocess.

Just make sure to document the installation requirements - in addition to having Ruby and the correct gems installed, you also need Python (which version?) and the correct pip libraries installed.

If you're a Ruby library, instead of an application, forking off a Python process is a nonstarter, unless you want to propagate those requirements out to every single application using your library.

Sure, but there's an equivalent problem in the C world - go back in time a couple years before we figured out how to do precompiled binary-compatible C extensions in Python and try running 'pip install numpy'.

I'm mostly thinking that if you already have a Python library you really like, chances are you already have a Python environment capable of running it. (The environment doesn't have to be related to your Ruby environment at all! If you want to get Ruby through your OS and Python through a Docker container, or whatever, that should work fine.)

> At some point, I wonder when programmers who are required to care about correctness throw up their hands and say "This language isn't reasonably human-sized for a user to know they're using it correctly."

That's a bullshit argument very close to FUD.

You don't need to know the 1400 pages of the C++ standard to use safely a subset of it. Specially with proper tooling to help you.

Do you really think that every web frontend developer knows every W3C/ECMA standard every time they create a website ? Or that they have even an idea of feature the complexity of the Web browser they use. One tips: no they don't.

Those standards are built to be default-sound. It shouldn't be possible to do wild memory access in JavaScript or via novel application of CSS or DOM structure.

Most importantly, if someone does find a way, it's an error in either the spec or the browser implementation. It's not flagged as "undefined behavior" and we go on with the assumption some web pages just break your browser or the OS hosting it.

I can tolerate a thousand-page spec for a system that can't crash unsafely; it's a much bigger risk for one with unchecked pointer indirection as a feature.

> It shouldn't be possible to do wild memory access in JavaScript or via novel application of CSS or DOM structure.

This is a narrow view where you focus only on memory safety and buffer overflow. Undefined Behaviours goes way wilder than that.

Many problems coming by UBs do not goe into any crash, just wrong results and wrong behaviour which is something even worst.

And currently, JavaScript is full of that. Almost everything that JS can not handle (or do not know how to handle is undefined) or purely implementation specific.

And for JavaScript, that space doesn't include buffer overruns or memory safety, unless you do something very exotic and implement your own indirected memory access in JavaScript (in which case, you've sort of dug your own grave). because undefined behavior literally means undefined behavior, C++ gets to bring to the table all of the failure modes JavaScript brings to the table in addition to the plethora of ways unchecked memory indirection can go wrong.

There are circumstances where you want the features C++ brings to solve a problem. But solving a problem with dynamite is still solving a problem with dynamite; there's a lot more ways it can go wrong than solving the problem with a steam drill, to torture an analogy a bit.

> But solving a problem with dynamite is still solving a problem with dynamite; there's a lot more ways it can go wrong than solving the problem with a steam drill, to torture an analogy a bit.

Excepted that a language is not absolute, is not dynamite or not dynamite. It is what you do of it depending of the subset you use. C++ is not exception.

Agreed. But I won't reach for a language that lets me treat all my program's working memory as an undifferentiated integer-addressable array of bytes if I don't need that to solve the problems I'm trying to solve.

... which goes back to the topic of the HN post; that's only one way to look at the state of a running program, and it's a way that has strengths and weaknesses. The utility of it comes at the cost of the program failing in ways programs in other languages structurally cannot fail (unless you do something truly exotic, like implement a subset of a C++ compiler or interpreter in that language).

C++ has always been the language that "has a safe subset to use but nobody can agree what that subset is".

JS is ... well, JS

This is at least for me the first time I've seen them discussed in the same breath, which is revolutionary: C++ the language bringing a blue screen to a desktop near you; JS the language bringing your startup to its knees.

I guess I never realized that while they are polar opposites, there are probably some subtle cultural similarities that would make for some hilarious unearthing

  Why don't you use something else?

Because, most (all?) of those other languages fall into one of three traps, they also have a lot of undefined behaviors, they define the behaviors in ways that don't reduce actual programming bugs (javascript!), or they have so rigorously defined the language with underlying assumptions (say a really strong memory model, signaling NaN, overflow exceptions, etc) that the performance is sub-optimal on any platform that doesn't exactly fit the expected machine description.

Really, C isn't hard if you burn your copy of K&R and stop trying to be so damn clever. It also turns out to be a lot more readable, if a bit more verbose, if one ignores a lot of its "features" and pretends its pascal with a single statement, without side effects per line. That includes using pointers in any kind of arithmetic (or type casting), instead using them only as though they were C++ references. (and a few other basic rules).

So, while I don't think rust is a particularly good language, I'm also starting to think that everyone should be forced to use it early in their career so they are forced to consider object ownership and lifetime in a rigorous way. Then when they move to C/C++/etc they wont be foot-gunning themselves at every turn.

Static analysis tools should be the standard part of any build process, but especially with C/C++.

I’ve mentioned before here, but when I found a reasonable number of bugs in critical library used throughout one of the FAANGs written by their most senior and brilliant engineers by running a fizzier on it for a few hours, my opinion of those languages changed.

Integrating Valgrind and Clang’s checker, and running AFL for many CPU years was critical in unearthing some really funky crashes, hangs, overflows. Without those tools it’s not clear that those bugs would have been fixed. In fact, one of them affected me a few months prior and AFL helped uncover the codepath that was failing.

I don’t think that this bugginess is an inherent property of these languages, because there are other practices that could lead to reduction in total bug count and severity, apart from integrating additional tooling.

Out of curiosity, how many bugs you found by using these tools could have been avoided by using a “watertight” memory management system [0], with strong decoupling of pointer and object lifetimes?

[0] https://floooh.github.io/2018/06/17/handles-vs-pointers.html

I'm curious who the target demographic is for this, people who think they need bare metal performance in a language that gives them access to bare pointers, know that most humans aren't capable of following Uncle Ben's adage[1], and then voluntarily give it up while insisting they need it?

I think almost any JITed memory safe language will be faster than using handles for all object access. At least Java, .NET, JS, etc under the hood can avoid "double dispatch" of memory access. And you can use things like arenas to ensure same objects are allocated adjacently, etc.

[1] With great power comes great responsibility

The solution presented solves "memory corruption" from the point of not having undefined behavior, but it doesn't really protect against "I accidentally created an index that has nothing to do with memory I manage, but is still 'in bounds' to the code that handles lookup".

Every language has some expression weaknesses, and C/C++'s are such that the languages make static analysis tools mandatory for correct code (much like Python makes every-line-is-executed unit tests mandatory for correct code).

What is profoundly pissing me off are bunch of hipsters on the internet that are using xyz language and are constantly trying to compare everything to C. We are faster, we are better, we are more portable, we have JIT, we have reflection... and so on.

Who cares what your tool for solving the problem is. Use Java, use JS, use python, go, rust... whatever suits for the task. Why do you want to compare yourself to C. Just stop it. No one with enough expirience in coding will ever care what language you have used to acomplish the task. Unless the task is going to be purely written, slow, buggy, CPU consuming, will have security issues and so on. This can be acomplished with half-baked (I hate the expression, but sorry) coding monkey in any language. Why do you even care about C. It doesnt matter, all that are complaining about it in whatever way possible or compare to it, will probably never be involved in a task where C excels, so why bother?

---

jeffdavis: the problem is that people would like to use wrong tool for the job. And the fact that they cant frustrates them, blaming everything else except the fact that they picked the wrong tool. As you need expirience for picking the right tool. A lot of expirience or rather beeing fluent in all the tools. Which makes it a problem if you dont dedicate a lot of time to this.

---

jeffdavis #2: > choosing the wrong tool really is a great way to understand your tools and the problem more deeply.

YES! Exactly! I did mistakes, a lot of them, like pragma packing the structure and send it to system on other side of network in other endianness. Overwritting EIP number of times before I understood it. Or best one, translating command.com to my language using edit :D Used wrong call convention. But my goal was always to understand assembler. To understand C. To understand C++. And I have made quite a lot of code in all 3. Then learnt any language that crossed my way and it looked like interesting (Rust is still on my list) and used each in at least one project, hobby or not. Now everyone learns one or two languages, without any background knowledge and then starts to preach it, prove that language written in X is faster than language that X is written in and so on. Who cares! It makes me puke.

> all that are complaining about it in whatever way possible or compare to it, will probably never be involved in a task where C excels, so why bother?

I don't think that's a fair assumption. You may be underestimating the number of developers who have a need for a systems programming language.

Sure, I do. I am just hooking linux kernel calls, I will never take JS for this. Pure C. Valgrind? I have a header with #define MALLOC/FREE/REALLOC/... which returns larger buffer with guards at beginning and the end while after 25 years of C I hardly do anything more problematic that this. But seriously, would you give JS developer making kernel module? Ruby? PHP? No. So I wont compare with them and I would expect they would stop comparing to C. It is not the same task they are made for. I am sick of constant moaning about who is faster and who is better and who is cutter,...

System language? Make it. I dont care, if it will work better than C for the task, I will take it. If it is just a newage nonsense wasting resources for the sake of deallocating and taking care about buffer boundaries as it is so hard to take care and track your memory I will skip it. I dont care if I take a hammer or brick for using a nail, I will use what works best and my requirement is that I dont need 20 people to hammer in one nail for the sake of no one getting hurt due to putting its finger between hammer and nail. Someone would rather have a safety inspection. Also a fine decision as far as I care. Not my choice but I couldnt care less.

---

Anyway, kids, downvoting me to censor what you cant handle, I am over 40 years old, developing for my whole my life. I have seen hypes, new "revolutionary" ideas, repacking old technologies as new, stealing ideas (I see this lately 24/7) at least 20 new "revolutionary" languages, cpus, mainframes, clouds, any possible way of another human trying to rip you off (google, fb,...), lies, deceit, preaching, evangelism,... Do you really think I will care about it? It is just sorry truth that you will have fun of enjoying kernels written in JS on a browser, on a linux kernel that no one develops any more. Have you seen movie Idiocracy? You really should. I cant wait for it to happen. And then will I ask you if you are sorry, that 20 years back you didnt rather decide to learn instead of copy/pasting and spitting over everything you are unable to handle. (your parent should told you that, but they didnt, I am sorry)

----

saagarjha: sure, ignore the part about censoring, that is not part of debate. About everything else. You get used to it in a same manner as using floats in js. But this is my expirience. It is completely ok, to have different one. Some people are memorizing decks of cards, i consider that something difficult. They probably dont. They have learned to do it. And I am so sorry if dont feel like I need to prove it to you. Anyway, thank you for the cycript.org link, the only problem is that i cant use userspace solution, but it might come handy some day.

I'm just going to respond to the top part of your post, as it's at least somewhat relevant rather than being a disconnected rant about the kids censoring you.

> I am just hooking linux kernel calls, I will never take JS for this. Pure C.

Why not? Hooking native code in other languages is already fairly common: http://www.cycript.org

> Valgrind? I have a header with #define MALLOC/FREE/REALLOC/... which returns larger buffer with guards at beginning and the end while after 25 years of C I hardly do anything more problematic that this.

Valgrind does significantly more checks than your solution.

> System language? Make it.

"Systems language" is a very ill-defined term, often used to gatekeep people.

> If it is just a newage nonsense wasting resources for the sake of deallocating and taking care about buffer boundaries as it is so hard to take care and track your memory I will skip it.

Keeping track of your memory is hard. If you say "no, it's easy for me", I'd like to see a non-trivial sample of your code that doesn't have memory safety issues.

It is easy for engineers, because engineers are trained to make things correctly even when that is inconvenient.

Engineers make bridges that don't collapse (unless not maintained for decades) and planes that don't fall from the sky (unless overruled by management). By the evidence, it is not easy for people who just can't be bothered to take the time to make anything correctly.

So, there are languages for engineers to use to make things where it matters if they are right, and languages for everybody else, where apparently it doesn't. Now, all we need is for engineers not to need to call into libraries not written by engineers.

> it is not easy for people who just can't be bothered to take the time to make anything correctly

Laziness may make the problem more likely, but even a disciplined team working on security-critical software can make mistakes[0].

[0] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-0777

> Use Java, use JS, use python, go, rust... whatever suits for the task.

A primary factor that determines whether a particular language is suitable for a task is its runtime performance.

> constantly trying to compare everything to C.

C has little runtime overhead and incredibly mature compilers, so it is an excellent target to use when measuring another language's runtime performance. "Close to C" is simply a synonym for "the language adds little runtime overhead".

The "right tool for the job" is a myth (EDIT: exaggerating; clearly it applies in some cases). Languages don't work together very well, so you need huge, thick boundaries between them, such as serialization/deserialization. Data types don't match up, GCs require special data formats and don't line up with the GCs of other languages, other runtime weirdness, etc. At minimum, you need lots of copying/transformation of the data.

Have a gnarly problem in a Python program that Haskell would be perfect for? Too bad. You'll spend more time figuring out how to transform the data and get the Python code to call into Haskell than you'll save by using the right tool. And in the process, the overhead, complexity, and bugs introduced in this process turn the right tool into the wrong tool.

And that's the reason why C is so important. It actually does work with pretty much any language, because it has a defined ABI, simple data types, minimal runtime, can work on any data formats unmodified, no GC. Very few languages have this superpower.

C++ mostly has that superpower, but some features don't entirely work with unmodified structures (RTTI and virtual functions both require adding magical fields to the class structure). Rust seems to have this superpower (does dynamic dispatch a different way than C++, so can always work on unmodified structures).

The JVM is an interesting case and the ecosystem of languages built on it do work better together because they have the exact same runtime (GC, etc.).

To summarize: we'd all like to use the right tool for the job. But we can't, except for the languages that go out of their way to make this effective, and that list of languages is very short.

This really needs to be said more. For those of us not in web devel or microservices, the lower level of coupling between components really makes using multiple languages a pain. Plus, you have to learn all those languages -- sometimes I just want to actually get stuff done.

I consider it significant code smell when a project has multiple languages, particularly when each language seems tied to a particular developer.

> Rust seems to have this superpower (does dynamic dispatch a different way than C++, so can always work on unmodified structures).

This isn't really true.

Can you explain?

Rust has both static (based on monomorphization, similar to a template instantiation in C++) and dynamic (based on vtables, similar to inheritance in C++) dispatch: https://doc.rust-lang.org/1.8.0/book/trait-objects.html. Both Rust and C++ are moving in the direction of encouraging static dispatch.

It may be similar, but it is implemented very differently, in exactly the "so can always work on unmodified structures" sense.

Trait objects are a double pointer: one to the vtable, and one to the data. C++'s dynamic dispatch uses a single pointer to a structure that has the vtable and the data. The memory layouts are very different.

"The memory layouts are very different."

Clarification: the memory layouts between C++ dynamic dispatch and rust dynamic dispatch are very different.

Rust imposes no requirements on the structure layout, whereas C++ adds some magic data. The magic data means it's awkward to make C++ do dynamic dispatch on a C struct pointer; but rust can do dynamic dispatch on a C struct pointer with no problem.

A really thought-provoking argument. I suppose that when it comes to choosing "the right tool" the important thing is to optimize globally across the problems that your organization will be solving.

But if you want to be able to optimize both globally and locally, well that's just an engineering problem!

Two examples spring to mind:

1. Lisp. Lisp's metaprogramming facilities allow you to build the language into the specialized tools that you need, while still having the common runtime and base language.

2. Unix. There a hundreds (possibly even thousands) of little programs running on my Linux machine written in probably dozens of languages. For the most part, I have no idea (nor do I care) what language they are written in. That's because the Unix model puts a strong emphasis on the protocols for communicating between programs.

Lisp is an interesting point. I guess the argunent is that it's the right tool for every job?

Unix is a reasonable example of a polyglot environment, but at a high cost. Lots of serialization/deserialization through text. That has a high cost in terms of bugs, complexity, lines of code, and inefficiency.

> Lisp is an interesting point. I guess the argunent is that it's the right tool for every job?

Yup, that's more or less the argument, though the argument need not only apply to lisp.

> Unix is a reasonable example of a polyglot environment, but at a high cost.

I'd rather say that Unix is an example of how to effectively drive down the cost of a polyglot environment to a point where it is outweighed by the benefits.

Now, of course the cost is still there, but I would argue that beyond a certain scale you cannot optimize globally on a single runtime (perhaps not even with something like lisp) and so the cost of global consistency is outweighed by cost of being unable to optimize locally.

To put it more succintly, you could not build a system as complex as Unix in a single language[1]

[1] Although Ala Kay's work at VPRI suggests you can, if you choose a sufficiently expressive base language. But even that may have it's limits.

Those programs communicate by serialization and deserialization, usually in bespoke, poorly documented data formats.

Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful.

> Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful.

OK, I probably misspoke by claiming it put an emphasis in protocols, but I think it's fair to say that Unix does emphasize component integration by sharing data instead of trying to integrate through a common runtime.

I do think that having everything be (mostly) text is helpful though. It's true that the data formats are often poorly specified, but that is compensated for by the fact text is a format that has a very rich set of tooling. text editors, regular expressions, parser-generators, etc. all make it possible to capture, analyze, and manipulate the text data exchanged.

Perhaps a better example of protocol-based integration would be the internet. It has it's flaws as well, but it's also enabled collective engineering projects on a vast scale.

Modern unix gives you mostly low-level mechanisms, not inter-application protocols. It is better to leave it up to applications/administrators to figure out the best protocol (policy) for their use case, to make the best use of those mechanisms.

Those mechanisms aren't textual, they are byte-sequence based.

It turns out that many text-based protocols are more popular than binary/object alternatives. The penalty for textual redundancy is negligible/acceptable for many of them. Where not, Unix allows people to use unix/create new mechanisms to implement specialized protocols (rpc, protobuffers,...).

You got it yes. And very few Software Engineer understand that unfortunately.

The reason C is so successful is named ABI.

C (and C++ to some extend) can be used to produce library that can be used from any higher level language in existence. And this is strictly required in the heterogeneous world we live in.

If you have to write a system components, long lifetime, implementing a protocol / database / format / any-low-level-staff: You do not have choice, safety or not. It is going to be C or C++, because it is the only thing reusable in most other environment.

Python has pybind11, Java has JNI, Lua operate beautifully with C/C++, Node is itself C++, go has cgo, Ruby is in C, any proper programming language can interface with C or C++.

They are the only languages allowing that, and that is why they are still alive, very well alive, even if they are unsafe.

As long as new language authors will be more obsessed about supporting new fancy feature instead of providing a well defined ABI (C compatible). The situation will not change.

Rust might be the new comer in this area that has its chance. But for the time being, it is still too young: you have to map every Rust API to a C one manually to make it exportable.

To be precise, C does not have a defined ABI (the spec goes out of its way to avoid this). Platforms (OS+Arch) define a C ABI for their own tools. Since this ABI is required to interop with the OS APIs, this will usually become the de-facto standard for the platform.

This might seem like nitpicking but it's an important distinction because you can't assume that C data structures can be passed between different platforms (e.g. over a network or even stored in a filesystem).

What is the right way to say that without sounding too pedantic every time? "Most platforms define a stable C ABI"?

The only platforms that define a stable C ABI are those whose OS is written in C and exposes a C API to userspace.

Symbian did not had a C ABI for example, nor do mainframes, in ChromeOS or Android you also don't have a C ABI exposed to userspace, in Windows a C ABI also won't help you much if you need to speak COM or now UWP.

"people would like to use wrong tool for the job"

That's actually a great way to learn. I do that intentionally sometimes to force myself to understand a problem better. Sometimes it even works[1]!

I'm half serious here. Clearly when something is important and needs to be timely, choosing the right tool is important. But my comment is not sarcastic, either: choosing the wrong tool really is a great way to understand your tools and the problem more deeply.

[1] https://github.com/jeff-davis/postgres-extension.rs

C is the lingua franca of the programming world. Of course people are going to compare xyz language to C.

Reply on news.ycombinator.com