Comment by deutschepost | Hacker News Reader

Comment by deutschepost | Hacker News Reader

I think Conan and Modules will make c++ more popular in the future. It is a behemoth of a language. Very hard to master and it is still too easy to produce difficult to debug errors. So it will be very important, that the language changes to address these.

But in working with c++ day to day I feel the most annoying things are indeed dealing with dependencies and headers. It can be a pain to set up a complex c++ project on a new machine. Even with Conan it can still be a pain because configuration management is a mess in c++.

As for headers I can’t wait to see them rot in hell. Some people say that they can help you reason about your Program, but this is a very small positive point. Too many times I had problems with includes which only worked because a Translation Unit included some other header beforehand. I once read a blogpost of the developer of SumatraPDF where they describe that they never include anything directly[1]. This can be an improvement for compile time. But if someone else is working on your code or has to refactor it, it can be impossible to read yourself into it or track any compile time error. If one would only use modules these errors would simply not exist.

[1]https://blog.kowalczyk.info/article/96a4706ec8e44bc4b0bafda2...

It's crazy to see how much work header files make the C++ compiler do just to build hello world:

    $ gcc -E hello.c | cloc --force-lang=c -
    419
    $ g++ -E -std=c++11 hello_iostream.cpp | cloc --force-lang=c++ -
    20707
    $ g++ -E -std=c++20 hello_iostream.cpp | cloc --force-lang=c++ -
    30876
    $ g++ -E -std=c++20 hello_std_format.cpp | cloc --force-lang=c++ -
    42757

Just bumping the C++ std version can add thousands of lines of code.

iostreams are more or less just a straight conversion of everything that C streams were (but nobody ever used) in to polymorphic interfaces and templates.

It's a shame they add so much header bloat, but there's a lot of functionality in there and it's hard to see a way it could have been avoided if you want type safety.

Most of the sin of iostream is having a humonguous overload set for operator<<. AFAIK it's not too bad to just compile the stream headers, but the overload resolution for `std::cout << anything` is expensive.

And after all that effort, an operator overload is still the "wrong" API for the job anyway, because: it's missing a crucial argument! Any binary (infix) operator can only relate two things, but typically when you want to print something you actually want to relate three things: the stream, the object or value to print, and the way you want values represented.

Oh, sure, you can manipulate this by inserting magic formatting objects, but that's not different from saying all functions take one argument a la Category Theory or Haskell without syntactic sugar. Or you could write a class or function that allows you to tag your object or value by changing its type to a different one so a different operator<< overload is called. In the iostreams design, you're only indirectly selecting which print function to use by manipulating either the stream or the printed object. But in actuality the types of your two arguments are not different, so either you must munge global settings on the stream or "hack" which operator<< overload is selected.

And that's the crux of the issue; what you "really want" is a stream API with three arguments: the stream, the object or value, and a pointer to a function (or function object) that takes the value and transforms it to a stream of characters. Why not just pass that function directly and explicitly? Of course that requires something that's not just a single chained infix operator; my thesis being that starting with that quirky choice leads to down a path that ends in a bad design.

> what you "really want" is a stream API with three arguments: the stream, the object or value, and a pointer to a function (or function object) that takes the value and transforms it to a stream of characters.

Or maybe a format object that returns a string in the way described by some directive, the way printf() worked in C or format conversions work in Python? You could call it something like std::format and make it available through a standard header like <format>.

Also taking us back to 1990's, anyone coming from Object Pascal, Smalltalk or Common Lisp would feel right at home with iostreams architecture.

As for the header bloat, fear not, import std; for the complete C++ standard library, is faster than a plain #include <iostream>, as per VC++ team measurements.

> Just bumping the C++ std version can add thousands of lines of code.

And yet none of this heavily influences compile times. At least I have not found so on MM LoC projects I've been working so far. Not sure what the fuss is around modules, I honestly don't think it will solve C++ build times. Happy to be proved otherwise.

Try compiling a small source file that includes a handful of STL headers, and compare to one that doesn’t. Even just including without instantiating any of the templates, the difference is a substantial fraction of a second. That can be a heavy influence, though maybe not in a huge codebase which also has plenty of large headers of its own.

I tried couple of times to workaround the problem you're describing through PCH on some of my ex projects. All of them were on the MM LoC scale and build-from-scratch was taking roughly 45min to 1hr on a developer machine. I never got to make a substantial cut in the build time through PCH, only perhaps by a very few % but YMMV.

Given that modules are PCH in disguise, I don't believe modules are going to have a dramatically bigger, and better, effect on C++ build times. Whether or not they are going to cut the build time from 45 seconds to 30 seconds I also couldn't care less - small projects don't even have the build time problem worth solving. Mid and big projects is where it will really count and if modules are going to have a substantial effect in the range of at least 20-30% in build time cut, I will be all ears.

Yeah, how much modules/PCHs would help probably depends a lot on your code base. I read about clangs work on modules and they seemed a bit uncertain if you'd get much benefit at all once you switch on optimizations. https://clang.llvm.org/docs/StandardCPlusPlusModules.html#ho...

How many of those lines are blank after #ifdef?

I used cloc so blank lines and comments should be ignored.

I don't really see many codebases using iostreams anymore, it's all either cstdio or fmt

When you consider that most popular modern languages long ago abandoned the idea of a separate header file, you see that the advantages far outweigh the disadvantages.

The problem isn't separate header files, it's textual inclusion. With C++ modules you still have separate module interface units. Separation between interface and implementation is important for enabling circular dependencies (mutual use) between implementations.

Except for C, no popular language (modern or ancient) has ever used header files.

So it's not a story of "abandoning", more like an insane C idiosyncrasy that wasn't ever used anywhere else.

In Python I sometimes miss "header files". Quite often you see:

    import x

    class Foo:
        def bar(self):
            import y  # break circular import
            y.something()

Would be nice to be able to import `Foo` without pulling in also `y`, or moving `y` inline.

Can be solved in different ways, but you see inline imports everywhere.

That's not a problem with modules, per se, but a problem with Python's module system. Importing a module in Python just executes the whole file. JavaScript used to have this issue with `require` years ago. Each `require` statement essentially executed that import right then and there, and would bug out if circular dependencies existed. However, they fixed it with `import`.

Almost every other module-based language does not have issues with circular dependencies. Python, in theory, could follow their lead, but they won't.

EDIT: I wasn't as clear as I could be. The issue again isn't modules, but scripting languages that allow top-level statements at all. Intermixing types and method declarations with executable code makes circular decencies an issue. Compiled languages don't allow top-level statements, so they don't have the dependency resolution problem.

> Each `require` statement essentially executed that import right then and there

That still happens.

It takes a lot of magic trickery to make cyclical require/imports work for JavaScript and a lot of times they don't/can't.

That's true. I guess I misspoke and wasn't as clear as I could be. What I meant to say was: the problem with Python and JavaScript's module system isn't modules themselves, but top-level statements. You are correct that `import` doesn't solve such a problem; It solves the synchronous dependency resolution problem.

Realistically, that's just what happens when a language allows top-level statements, as they are executed when a file is loaded. As such, scripting languages tend to fall victim to the problem, but compiled ones don't.

I'll update my comment.

Worth noting that ES-style imports do improve the scenario for cyclic dependencies though, assuming you do not need to reference the symbols imported during the execution of the top level statements. This is because `import { bar } from './bar.js'` makes `bar` a "live binding" -- even if `bar.js` also imports `foo.js` and thus `bar` may initially be undefined due to the cyclic dependency, later when `bar.js` completes its initialization of `bar`, the identifier `bar` in `foo.js` will be updated to match.

So if you just need to call that binding within, say, a function or a method call, its typically fine to do so assuming that by the time that function or method call is executed, both modules will have completed their initializations.

And that is similar to require's behavior for properties on the returned object.

   const foo = require("foo"); // starts out as empty object

   exports.bar = function() {
     return foo.fn();
   }

Yes but the difference is that this:

let x = require('foo').x

of course, cannot do that. It's a small difference but it does make it easier to fall into the happy path in more cases.

> Realistically, that's just what happens when a language allows top-level statements, as they are executed when a file is loaded. As such, scripting languages tend to fall victim to the problem, but compiled ones don't.

Lots of languages allow initialization code (e.g. Java static initializer).

But in the case of scripting languages, usually the classes and functions themselves are initialization code....i.e. everything is initialization code.

Go forbids cyclical imports, and, in most cases, I really appreciate it. Makes the code much simpler to reason about

As someone who doesn't use Go, how would one handle a situation where two modules actually depend on each other? For example, a parent object owning a child object is normal, but what if that child object sometimes needs a (weak) reference to the parent?

Just put them in the same module?

What if the child is an user-implemented piece of code and the parent is in a library?

Then there wouldn't be a circular dependency. If the parent is in a library and was written independently of the child, then the parent could not possibly import the child.

> Almost every other module-based language does not have issues with circular dependencies.

I recall having seen a circular dependency compiler error in Go.

wait, is this even possible? I just presume that all import statements are executed first.

Ironically you can solve this with header guards in python as well... Old problem requires old solution

It was only ever a thing in the first place because, almost half a century ago, some people decided to write a one-pass compiler due to storage/memory limitations. Even 40+ years ago, C was the only popular language that required them - to never address the issue is practically inexcusable.

I personally like headers very much and do not understand the hate: They are super simple and efficient.

They do not work well C++ though because it puts the implementation into the header for some reason.

The reason being the compiler needs to know the template code to be able to generate a specialization for it.

ELF files based on the C type system can't provide such information on object files, and binary libraries.

In the modules world it can read it from the symbol table on the BMI, like in any other module based language.

Not sure what ELF and C type system has to do with this. C type information can be stored in ELF (as well as many other things). This is how a debugger can know the types.

It is simply a language design flaw of C++ that class implementation (not just templates) ended up in the headers.

so modules should have actually been one of the first feature added to C++? Because in C the usage of header files is still more or less sane, it's only C++ that drives it too far...

Impossible given the original design requirements for C with Classes, while being developed at Bell Labs, require nothing more than a standard UNIX C toolchain.

Hence name mangling, the only way to add some linking type safety, while using a bare bones UNIX linker that only knows C and Assembly.

One justification I've seen is that the header defines the interface of a library and allows its implementation to change; the argument reads remarkably like that for pure abstract base classes or what other OOP languages call "interface". The problem, of course, being that C's version of this idea is a bastardized compile-time-only polymorphism.

But from a developer user experience standpoint, what is the difference between having to write things twice for this reason:

// header.h

int foo();

// library.c

int foo() { return 42; }

Versus writing things twice for this reason:

// interface.cs

interface IBar { int foo(); }

// library.cs

class Bar : IBar { int foo() { return 42; } }

Granted in C it was for dumb reasons whereas in a modern language it's for better (?) reasons, but: You're still writing things twice!

> writing things twice

Not in Oberon.

The ada concept of "package spec" and "package body" does headers the right way.

Expose the types, and the public procedures. Users only need to see the spec during compilation.

Pascal has include files which work very similarly. Cobol has the COPY statement which is commonly used to include shared data structure definitions (“copybooks”).

I think headers were a great tool from a time before intellisense where developers and teams would include function + comments on how to use it.

On the flip side, I think there are still a lot of developers who don't use language servers or use the equivalent of notepad who do rely on separate header files as a means of documentation.

That said, it's always been painful to structure inter-related headers and source files to avoid circular dependencies, and if modules truly resolves that, I'm happy to see it.

conan2 breaking backwards compatibility with conan1 is... rough. My first conan experience was everything breaks because of that...

>If one would only use modules these errors would simply not exist.

In exchange for reducing parallelism because you are not using forward declerations to break dependency chains up.

+1

I once refactored part of my code for headers to only #include forward declarations (and few generics, of course), except where really needed (implementation)

It made compile times 2x faster, probably could get 3x if fully done.

It is going to be a struggle. Headers work well for full recompilation as they provide significant parallelism. But are very bad for incremental compilation.

Only for the mindset stuck in the classical UNIX tooling approach, module based languages have used parallel compilation for a long time.

The big difference is that those communities embrace compiler and tooling are part of the same story.

Thankfully now we have a tools working group trying to bring C++ community into the modern world of module based compiler toolchains.

Show me a module based language that maintains the same parallelism of compiles.

C#, compiling in parallel since the C++ compiler was replaced with a C# written one, beating C++ traditional compile times hands down.

Go, compiling code in parallel since Go 1.19. One is able to compile the whole toolchain from scratch faster than many C++ codebases.

Active Oberon, compiling in parallel via the Paco toolchain since 2003, a full graphical workstation OS, built in a couple of minutes.

Ada, parallel compilation available in most toolchains since 1989. Also available in GNAT.

Delphi, yet another example.

It is C++ that needs to get up to date with modern toolchains and away from hacks like unity builds.

Go compiler does incomparably less work than GCC. Arguably, GCC could have been faster, but this is somewhat apple to oranges comparison. The closer one possibly is comparing .NET Native AOT and GCC, where the compilation times and memory use start not to differ that much because of significant amount of work required to properly optimize and link all assemblies/compilation units.

Otherwise, I agree that LLVM and GCC are not an end all and be all, and have significant issues with compilation performance.

Notice how you ignored the existence of .NET Native, Ada, Delphi.

While I can pick other examples, I can't be bothered providing an exaustive list of every single language with compiler toolchains doing parallel compilation.

.NET Native is dead and when Ada or Delphi were in their prime I did not even have a computer which is why they got no mention :)

Reply on news.ycombinator.com