No:
* Setting up an isolated build env
* Pulling 1600 dependencies and building them from source
* installing a filesystem layer with 1600 .so libraries
* running docker, npm, etc?
_AND_ it compiles in a fraction of a second?
Amazing what this new language C can do.
No. You need make and a C toolchain installed.
If you have those installed, you almost certainly also have Python, Perl, and possibly even a reasonably recent Java (not on MacOS, yes on most Linux distros, i think) installed, and all of those are also capable of doing this with zero dependencies.
I've seen embedded C code that requires Python AND Ruby to build.
Linux is still using makefiles (+ config). Your project cannot possibly be more complicated
The Guix project recently hit a massive milestone in untangling this, by creating a package set which starts by building a 357-byte program and works its way up to the utilities you'd expect: https://guix.gnu.org/blog/2023/the-full-source-bootstrap-bui...
I had one embedded project that I handrolled the makefile for, and it was all in pure C, no OS. That was fine, but ensuring that the versions of the hardware abstraction layer, and other drivers I wanted to use from the same vendor was a source of annoyance. Keeping track of which source files were to be enabled and disabled in the makefile was also annoying.
Another project I am working on uses platformio on an RTOS (and a much bigger micro). Platformio is written in Python to handle all of that 'seamlessly' and it does work pretty well from my perspective. And I don't have to manually install separate toolchains to /opt (or wherever) and keep track of them; it 'just works.'
I like the makefile approach. With only a little work, it's possible to make clean build directories without .o files littered around like mouse droppings. I know exactly what code is being built, and, because I usually have to write the linker script, where it is being put. But once it gets more complicated with different toolchains and compatibility problems (and it usually does if you have an RTOS), it sure is nice to have something like Platformio.
So I could go to all the trouble to have a cascading compilation all in pure c or c++, so the git SHA is written into a given source file, which is in turn built and flashed onto my target device. Or I could use an interpreted language for all the off-the-device operations, completely skip over configuring a separate development environment, and do it all in a fraction of the time.
The only reason I want to be compiling c or c++ for the host is because I'm also writing a driver at the same time, and even then...
Just curious, but based on your experience, why does it “usually” get more complicated in those ways with an RTOS?
The `cryptography` library (I'd say quite an important one in the Python ecosystem) requires Python, C and Rust to build, since v3.4.
A full text search for "python" reveals only a single match hidden at the bottom within the FAQs section; ironically within the context "Why are there no wheels for my Python3.x version?".
Most of the other FAQs add to it:
- I cannot suppress the deprecation warning that cryptography emits on import
- cryptography failed to install!
- Why does cryptography require Rust?
- Installing cryptography produces a fatal error: 'openssl/opensslv.h' file not found error
- cryptography raised an InternalError and I’m not sure what to do?
- Installing cryptography with OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 fails
- Installing cryptography fails with error: Can not find Rust compiler
- I’m getting errors installing or importing cryptography on AWS Lambda
- Why can’t I import my PEM file? What happened to the backend argument?
- Will you upload wheels for my non-x86 non-ARM64 CPU architecture?
Python is by far the most platform-dependent, brittle programming environment, and the prime example that the "interpreted languages run everywhere!" trope is increasingly dead.
Build nonce, fail everywhere.
I’m pretty sure that was a typo, but it was an excellent one and I hope it doesn’t change! :)
I meant: "Python *has by far the most platform-dependent, brittle programming environment".
> You are choosing to use a c/rust accelerated library.
Especially with something as fundamental as `cryptography`, I don't choose it - it is a given through transitive transitive dependencies.
> If your project were 𝗽𝘂𝗿𝗲 c or rust, you would still have to build everything.
That's the whole point: when developing in programming language XYZ, then stay within XYZ's environment: As much as possible, implement stuff by writing in XYZ, and import other stuff following this principle.
Whereas, the moment you or one of your dependencies delegate to other environments, your program inherits and requires additional environments.
So many projects and libraries in Python World have taken the latter approach, resulting in so many Python applications being a cobbled-together Frankenstein diva. It will refuse to work on your users' system, until they have the correct rubyenv, and autoconf version, and npm.
"Huh, why does the user report a problem with npm when installing my Python application?". Well, that could be because unbeknownst to you, whenever you yourself developed, updated, and ran your program, there was always this tiny dependency of a dependency that needs it - you just didn't know about it until this issue report, because your system already had Node and npm set up correctly. That reporting user's not!
Of course they did. But they made it sound like the language is the deciding factor, and that's not true.
C can have horrific-to-install dependencies, even worse than you see with binary python packages.
Like, a ton of our ops-scripting, things have 2-3 direct dependencies and maybe 10 transitive dependencies. And the latter is an actual flask application with pydantic, so a bit outside of "scripting". We've settled on a set of these versions, put them into a venv on a system and that's it. Update every few weeks, get accustomed to the libraries. Nice and stable base to work with.
And then we have some of our python-based ML teams. I'm not joking, but I have entire classes of servers which - including their dataset - are smaller in storage footprint than some of their python containers including tensorflow and models and panda and all manner of things. Feel free to accuse me of comparing apples and oranges, but that feels very, very strange.
I come from an embedded systems / microcontroller background though, so the bloat does feel completely insane.
For me things like Django feel "worse" overall in terms of developer QoL than the ML universe does, but maybe just because I don't know it as well and my last real webdev experience was before PHP4/HTML5.
The snark is mostly arising because I recently wrote a little tool to do an inventory of our private docker registry, because storage was getting difficult to manage. After a whole bunch of messy data collection and setup, it sets up a graph of the layers in the registry, assigns images and tags to owners. From there it backpropagates "ownership" of layers - if a layer is only referenced by images attributed to one team, that layer belongs to the team, else it's marked as mixed.
Before doing that, people were enthusiastic about making image builds more efficient, tagging smarter, collecting faster. A great atmosphere I'm finding myself enjoying more and more in the company, which is great.
However, doing the analysis pretty much showed us that 90% - 95% of the storage used in the private registry was used by these python/ml docker images. We found several projects whose CD images were not being garbage collected by the setup for 2 years or so. This wasn't great, but the entirety of their images took up less storage than like 2 base images of the ML stack.
It's the thing we have to do, but I reserve my right to make fun of it.
The upside of that copy-and-paste coding is that you don't have dependencies. But I'd hardly call it a virtue.
My entire system is built on a staggering amount of C libraries.
Until recently, literally the only way to have your code reused from different applications in different languages was to use C, or use C++ with the interface butchered to be C compatible.
Other languages code reuse is "well, first you have to go all in into our platform", which is great from the POV of someone already on that platform, but not so great if you just want to reuse a small bit of code from that other language.
12 year old me would choke seeing NPM, pip, or bundle these days... or Electron!
We could do so much with so little!
$ gcc foo.c; cp a.out ~/bin/foo
$ foo
$ go run main.goI shouldn't have to specify a ton of flags to g++ something, it should find them within the project directory and pull in dependencies automatically.
Header files should be done with. Poof. Gone. There is zero reason for them to exist. I know exactly why they exist, but it's an artifact of a system where preprocessor, compiler, linker are separate. I should be able to do "import<foo/bar>" multiple times and it should pull in all the classes and functions in "./foo/bar.cpp" or "./foo/bar.so" (whichever is available), just like Python, and cache the ".so" files just like Python caches ".pyc" files.
Improving C++'s build experience would have absolutely zero impact on execution speed.
Yet they're busy adding some stupid spaceship operators and other things people almost never use.
Wait... you didn't get the memo? They did, and renamed the language in the process... it's called Rust. ;)
The reason they still exist is due to how useful they are.
The majority of Android applications probably wouldn't exist if C had done away with headers.
You also can pass env vars to it:
$ CC=clang make foo
And (with care) define your own in: https://www.gnu.org/software/make/manual/make.html#MAKEFILES...
He did not make it past me.
Not that we don’t care about those things, but we have better uses for our time and money than implementing an in-house version of React. Also, our other engineers would probably have strangled him.
Using stdlib is almost free on most systems. It is usually preloaded so it doesn't even use up ram. Unlike functions you write yourself. And with static linking, linkers will only take the functions you actually use, not the whole library.
A js library needs to be downloaded and compiled in its entirety by the user browser. There are techniques to make it less bad, none of them ideal.
The way I interpret GP post is that 12 year old didn't want to bloat his code but had a misguided idea of what causes bloat. You are saying that bloat is fine if it saves on dev costs. Two very different ideas.
Please give me a little credit! It wasn't that I'm against writing code in-house. That's my favorite part of my job! But someone insisting that they want to write everything themselves is a huge red flag. I mentioned React to give it a contemporary context. The interview was a few years ago, and they were ranting against the bloat in jQuery.
That also wasn't the only reason I said no, and I won't go into detail about the rest.
Yeah sure, that's what must be broken. Not my code.
A while later I got smarter, and progressed enough to hook into DOS interrupt handlers with my C code and some assembly glue code. I decided to make a program that would click the PC speaker every time the hard drive is accessed (not a new idea, but a good exercise), by hooking into INT 13h (in DOS/x86 in general, sw interrupts were basically used as system calls).
To my delight, it worked, and it suddenly seemed like I had a very loud hard drive. But DOS would also spit out strange I/O errors when trying to list the current directory.
Huh, weird, I wonder if the same happens in the root directory? So, "cd \", followed by "dir", and, yep, I/O error.
Okay, let's reboot and... doesn't boot?
It dawned on me pretty much immediately what must have happened: I didn't restore all registers correctly before calling the old hard drive sw interrupt handler (again, system call in modern parlance). It must have been unlucky enough to convert some reads into writes. And by trying the same thing on the root directory, I probably converted some reads to the sectors making up that root directory into destructive writes... oops.
I mean, fundamentally, you weren't completely wrong.
This one needs some unspecified versions of gcc, make, and ncurses installed.
That's not what happened for me, and it seems to be a very odd way of presenting a very different message.
Turns out the minimal dependencies needed involve several OS packages and also the special "dev" versions of those packages installed, and I need to use one of the right subset of terminals.
Having the implicit knowledge to get the "magic", doesn't have anything to do with minimizing dependencies. Sounds like the real thing being said is: "It's neat when people use a toolchain and environment I'm familiar with". I don't think that extrapolates to "minimal dependencies are what we want".
Consider the set of dependencies required to recreate this in language / framework X.
I assert that this set of dependencies is larger (in some cases much larger) for probably all languages and frameworks, except those that are built using the same language, compiler, and libraries used by, and shipped with, the target operating system. I would also assert that the speed to build, install, and deploy the package would be worse. Larger here means "includes more lines of code", as well as "requires more libraries" and "requires more executables, probably from interpreters". "Worse" here means "more time required to resolve", "requires more downloads", and/or "requires more CPU cycles to solve the multiple levels of indirection from interpretation".
It just so happens I have npm, nodejs, cargo, g++, python3, pip, and gcc floating around, and I almost always find that 'make' with `gcc` is the fastest gun in the west, as long as you stick to some basic OS libraries, as this project does. Cargo is real, real close, and probably equal if you ignore the long dependency chains that result for almost any project.
But that's much more verbose, and I think the original message implied this for most readers.
I would assert, that this could be done in C, python, rust, and go - no libraries just whatever ships with installing the compiler package. Probably others too but I don't know them well enough to say this about them with the same level of confidence.
I will concede that some of the packages to install those languages may be bigger (in the more bytes sense) than gcc + dev versions of the c libraries.
Literally it just does a few syscalls to read from input and write to output, all those syscalls are available in the stdlib of the languages I mentioned. Everything else is just a few simple loops and some table lookups (there is the ncurses version that does it a bit different, but it includes the raw vt100 codes too - I'm ignoring the ncurses version to be kind to your assertions about dependencies).
For the languages I listed you need the same number of binaries, or fewer. For go - you just need the go binary. For python, the same. For rust you need cargo, rustc and ld (vs C's make, gcc, ld). (note ld here is shorthand for all the tools ld invokes too).
As for cpu cycles, you may be right but that is also unrelated to the minimizing dependencies message - although I don't know that the binaries created by rust or go would actually use more CPU cycles, and likely not enough to matter on any computer made in my lifetime.
As for compile times - I don't think python or go would have a noticable difference (python is compiled to bytecode to run on a vm at startup, its not a true interpreter so what I mean is that the python text -> bytecode part will not result in a noticeable startup delay).
Looking at the project code, I can see several ways to make it have lots of additional dependencies in C or any other language. For instance we could consider the ncurses version in the repo itself, that adds a dependency which must be downloaded (ncurses is installed often by default on linux, but ncurses-dev or whatever is an extra install). But why stop at ncurses, we could make it use glib and getopt and so on.
At best, the dependency thing seems orthogonal to the language.
As for packaging - well I just don't believe it would be worse.
It would be the same set of tools to create the package for each of them - for go and rust you wouldn't even need additional OS dependencies to install the package - really it's just a change of the build command and the binary path to copy into the package, though you would for python - fortunately that is included as part of the default for many OSes that don't include make and C.
But if you want a really lightweight language, you'll want something like Lua. It's like 0.2MB in a single binary which has a (very small) stdlib already built in it.
There is a reasons old-style routers/embedded devices with megabytes of flash came with lua (and rarely python/perl), and almost never with gcc or other languages.
We build on the shoulders of giants. We pull in other peoples code because we don't want to duplicate work, ours is likely not to be as good, any code is a liability, and sometimes _we just aint got time for that._
The problem is that most of the time said giants are actually sand golems and it is giants all the way down.
Actually, C is worse: it could all be any compiler/version, make version, or ncurses lib version - nothing is fixed, not even documented. At least in other language there is some standard way of pinning dependencies.
Yes there is something special about C (on Unix/Linux). C is the mother tongue of Unix so it's more intertwined with the system than anything else (other than shell scripting also). You'll nearly always have cc installed. The libraries you need are often in /usr/lib as part of the OS install.
Python, no way. Now you have to start pip installing dozens of packages and then you run into dependency mess so then you reach for some tools to manage multiple environments and suddenly you're a few hours into what's become a project when all you wanted was to run a script.
Aaaaand then there's all the arcane configure script nightmares.
I have and it wasn't a problem. I ran 32bit libs alongside 64bit libs for literal years without a problem.
I have to say ... no. And I've worked on some very large C projects at times.
Part of it is cultural. In the systems library space, there is a much stronger (nothing is perfect but I'll say much stronger) culture of compatibility awareness. Libraries don't break compatibility randomly without any planning unlike in less mature ecosystems (coughnpmcough). So you don't need to depend on libfoo 3.17.42 because neither 3.17.41 nor 3.17.43 work for you. That's not a thing.
Part of it is how the system works. If you look under /usr/lib you'll notice libraries are versioned and have symlinks from major/minor version to them. Different versions coexist and you can link accordingly. Even if you have (rare) badly-behaved library authors who break compatibility, it's not an issue to have multiple versions and each program links to the one it needs.
They don't think on the big picture the efficiency on the resulting binary size and build time (implied by the number of dependencies).
Another thing that is ignored is the external dependency that the user would need to run the program (whether none with a native compiled binaries or a whole runtime environment to run your python program)
Python used to be fantastic. It actually used to just work. That stopped once everyone caught on to the language, Nvidia got involved, and tensorflow / pytorch /ML ruined everything with so many script kiddies writing nonsense that had no interest in making their packages compatible with anything. The community is the best, but also very very much the worst part of the language.
C is not special in this regard. Special part is an OS itself that takes the role of a single system-wide virtualenv or node_modules/.. with all common trouble packaged in.
Python and others didn’t create this trouble, they ejected it from your system into a single rm-able folder.
This project built fine, though.
Python is actually smaller, 82MB and 1461 files, and has a much better stdlib.
If you never wrote zero-dependency python scripts, give it a try! It's a very nice experience: download file / clone repo, run a file and that's it. Not even a build step.
I wonder how minimal a giant web app written in C would be? Would it still compile in a fraction of a second?
And probably also contain a bunch of RCE vulnerabilities.
Took me hours to get Audacity building on Windows. Actually, I might have given up and installed Linux instead, and built it there. That's usually way easier!
But the other day I built an old MP3 encoder (BladeEnc) in a split second. It brought a tear to my eye. That's with Visual Studio 6. Which somehow builds an entire project before the new VS even begins compiling!
git clone https://github.com/bartobri/no-more-secrets.git
cd no-more-secrets
make
./bin/nms
It worked!
One well-maintained C project in isolation. I've also seen nightmares that are difficult to build on certain platforms or require undocumented build dependencies. Let's not forget how horrible the automake/Cmake/Scons/etc. ecosystem can be.
Everything is awful when there's no attention to the dev experience.
> make: The term 'make' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Everything else is weirdo legacy crap.
/s
I'll note that this project does come with an `uninstall` Makefile target, but not a lot of projects do/did.
An effect that I could write in high school using Pascal and that would require only builtin library and few loops, in proposed source code requires so much *stuff*. Given that this is a fun demo and not something that one needs to maintain as enterprise software, I totally would accept such trade off and remove dust of something simpler.
Oh, memories.
Using Crt;
GotoXY();To be fair it's a very old language, but what's crazy is that decades later engineer still have not come up with a way to execute the C directly, but have to rely on a process called "compilation". I mean how hard is it to make a CPU that understands C? /s
The dynamic C listener from symbolics C could do this decades ago.
It's a C program running under UNIX/X11 that was last updated in 1993. Last I tried, it still compiled with absolutely minimal changes (I think I had to change an #include directive or something) after typing `xmkmf; make` (which was the way to compile X11 programs at the time), and runs, on macOS. Provided you have Xquartz installed.
/s in case anyone is confused.
Second: usually docker, npm, etc are not needed. If the code use 3rd party C libraries, you need to install it. On Debian, again you can use `apt` to make life easier.
How do you know which is the wrong version of GCC? That's the neat part, you don't.
* For this C program: Let's say a default install of a Debian system, with GCC and Make installed. Nothing else. Just clone and run make && ./bin/nms.
* For an hypothetical JavaScript implementation: Same Debian system, with Node.js installed. Nothing else. Just clone and run node ./src/nms.js.
Now tell me that the second point would ever happen, of course without the obvious trick of vendoring tens or hundreds of dependencies in the repo itself. Given the current trends and ecosystem incentives in the JS development world, I highly doubt it.
These trends only favor mindless composition, of which the latter is good, but the former is bad. IMHO most devs would probably not even consider the idea of writing a compact, self-contained piece of code and have their own termio [1] or charset [2] implementations, to begin with.
[1]: https://github.com/bartobri/no-more-secrets/blob/master/src/...
[2]: https://github.com/bartobri/no-more-secrets/blob/master/src/...
Also, there are plenty of non-C languages (js may be one of them, but it's been half a decade since I touched it and longer since I used it in any meaningful way) that you could transliterate this program into - its a handful of lookup tables, a couple loops, and a couple syscalls available in every stdlib I'm familiar with. It seems disingenuous to assume that there are only C and JS (or being generous that those extremes are the only options).
> assume that there are only C and JS
Not so much ignoring other languages, as just talking about the one which coincidentally (or maybe not) is the one that usually attracts most conversations about dependency ballooning, at least around here on HN.
> if those exact same devs would write C, they would somehow magically stop seeking out libraries to solve the problem for them?
I'd posit that an ecosystem which doesn't encourage to add a third party library for the tiniest of needs, yes, does indeed discourage from using libraries except for the most egregious needs.
E.g. you wouldn't implement a whole object-based oo-style programming paradigm in C, it would be wiser to just use GLib with its GObject implementation.
But it would be uncommon to use a stupid library like is-even. Something that is very common to do in JS.
And actually this leaves the space of being opinion based and is more supported by evidence. You'd hard pressed to find this kind of mindless lazy usage of libraries in well consolidated and popular C-based software. But I bet you wouldn't need even 30 seconds to find some popular JS package that in its transitive dependency tree ends up using a left-pad [1] level of library.
[1]: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/