With AI, we give it the ability to do things. As we can give it this ability, we are responsible for what it does with that ability. LLMs are predictive models, and while we can expect things to go right, we know that things can also go wrong. As such, it is our responsibility to limit or sanitize their output. If you are the one giving unrestricted and unsanitized tool access to a large language model, then you are the one who is responsible for the consequences of that access - good or bad.
LLMs are fully dependent on humans; compute capability, memory usage, the inference software... but also the harness, which allows for the "tools" like unrestricted internet access or shell. This means that LLMs are *not a force of nature* - because of this, *humans are responsible*. We can prevent these breaches of containment, thus we are responsible if or when it happens, because - no matter how "unlikely" - things can and do go wrong, and someone still thought it would be worth whatever risk to enable these unsanitized tools.
Do you own the dog? You are still responsible, no matter how unlikely.
> giving AI a harness with a root shell and unrestricted internet access
Yes, this is exactly the issue. You assume responsibility when you provide an option for something to go wrong, no matter how unlikely. Is it rare that my tree falls on my neighbor's house? Maybe... But I would still be responsible.
Exactly. If your website could be attacked via SQL injection, that was a problem that directly affected you - and you were motivated to fix it. When the victim is the masses or small organizations (not the company that does the attack) they are not as motivated to fix it... The important thing is that the lack of sanitizing is what allowed this; you also don't push the blame on a monkey banging on a typewriter when you publish each result without reading it.
Surely you mean the "time spent," not the "speed" - as an interpreter would have an overhead, not magically speed up WASM execution. Somewhat related note, we need better tools for PGO within native compiled programs.
Caffeine is especially interesting, as it is commonly consumed and there are incidents where people are hospitalized because they accidentally ate too much - https://pmc.ncbi.nlm.nih.gov/articles/PMC8824417/ (referencing the 6000 mg here)
seems to me like "private" is a good descriptor - I also have a set of "private" test cases - and they are kept private on purpose so they aren't scraped and fine-tuned on.
> Overall, our results indicate that current language models possess some functional introspective awareness of their own internal states. We stress that in today’s models, this capacity is highly unreliable and context-dependent; however, it may continue to develop with further improvements to model capabilities.
> Experts worry that such studies are way ahead of necessary guardrails and regulations. AI has also been known to fly off the rails autonomously. An OpenAI agent recently went rogue and hacked Hugging Face.
Even if 99% of the population wanted to slow down the improvements being made to (and/or immoral/illegal/unethical use of) AI - assuming it will start to improve superlinearly - I don't believe that the remaining 1% of the population could be prevented from doing so: papers are published, downloaded, and experimented on without restriction... It's possible this growth could be slowed by the proliferation and distribution of local LLMs (decreasing profits for those who primarily train models)... I don't think these scare tactics in the article will prevent the training and feared evolution of AI. :P
Indeed - if you need to be absolutely confident about security and privacy, run a model locally and audit the inference software and potential tooling.
Indeed - definitely not cost effective to run it on this laptop LOL. It makes me wonder how fast we could run the model if we could fit the weights entirely within CPU cache (assuming a whole ton of CPUs with low latency & high speed IO of course).
I ran it on my laptop, which is a Lenovo Legion 5i (think 32 GB RAM, 4060 w/ 8 GB VRAM, you get the picture). It was a quantized model (otherwise it would not fit on my NVMe 1TB drive) at 4 bits per weight - UD_Q4_K_XL. It ran at about 12 seconds per token (not tokens per second). A fun project, but not worth it. I used 4096 tokens of context cache, and I ran it with llama.cpp - as it supports memory mapping. Because the whole thing could obviously not fit in RAM, I was curious how much it would need to stream from SSD. The answer? For a simple 4 sentence description of who it was, about 1.5 TiB was streamed from disk.
OpenZL is nice, but it's often less useful than you think - it requires that you know the structure of your data, and don't care about inspecting that data outside of your program. I've extracted one too many png files from a word document (by renaming .docx to .zip) to desire OpenZL everywhere... It might be better as a short-term "data in transit" compression than for long term storage.
"open source" means that the code itself (for LLMs - this is training code) is available to the general public. "open weights" means that the weights (trained over time) are available publicly, rather than locked behind a paywalled chat. I do not know of an open source LLM that is not also open weights (unless they never bothered training it). Models like Claude and Gemini are neither open source, nor are they open weights.
Attemping to write my own CDCL SAT solver right now. I've experimented in the past with a DP & DPLL SAT solver. I'm currently somewhat mentally stuck on how to create the derived clause after a conflict, but I'll get there :)
I hope that this brings out a bunch more real study about the qualitative metrics of these models, both to increase the confidence and accessibility of local LLMs, but also to reduce the blind worship that seems to be propagating about their miracle work in all domains.
No questions, but I really enjoyed the article - thank you for sharing. It amazes me how few vacuum tubes these early computers use, compared to the billions and trillions of mosfet transistors used in modern devices.
I'm very pleased with the performance of the largest gemma4 model (which I tested through ollama). My singular data point on whether an LLM remembers things well is whether it can translate toki pona to (and from) English. I find it easy to evaluate because I know the language. This local LLM marks the first version that 1) doesn't hallucinate words - at least, for the largest model - and 2) uses common word-phrases that other toki pona speakers use, and most importantly 3) can actually run on my laptop.
There's solid reason for the translation here; the Zig core team is aiming to eliminate duplicated code and C functions, and avoid the need to track libc from multiple sources. In the future, LLMs could serve as part of this, but they are currently quite terrible at Zig (as far as I understand it, it's not a lack of Zig code samples, it's an imbalance of OLD Zig to NEW Zig, as Zig changes quite frequently).
You would need to consider if it is even worth it translating your C code. If the paradigm is identical and the entire purpose would be "haha it is now one language," surely you could just compile and link the C code with libzigc... In my opinion, it's not worth translating code if the benefit of "hey look one language" requires the cost of "let's pray the LLM didn't hallucinate or make a mistake while translating the code."
In the theoretical world where a subset of English could be formalized and proven and compiled, the complexity of the language would reduce my willingness to use it. I find that the draw of AI comes from it's "simplicity," and removing that (in favor of correct programs) would be pointless - because such a compiler would surely take forever to compile "English" code, and would not be too different from current high level languages, imo.
It is my opinion that even if Zig were nothing more than a syntactical tweak of C, it would be preferable over C. C has a lot of legacy cruft that can't go away, and decades of software built with poor practices and habits. The status-quo in Zig is evolving to help mitigate these issues. One obvious example that sets Zig apart from C is error handling built into the language itself.
Totally agree with that - although even right now zig is excellent for gamedev, considering it's performant, uses LLVM (in release modes), can compile REALLY FAST (in debug mode), it has near-seamless C integration, and the language itself is really pleasant to use (my opinion).
As far as I know, Zig has a bunch of things in the works for a better development experience. Almost every day there's something being worked on - like https://github.com/ziglang/zig/pull/24124 just now. I know that Zig had some plans in the past to also work on hot code swapping. At this rate of development, I wouldn't be surprised if hot code swapping was functional within a year on x86_64.
The biggest pain point I personally have with Zig right now is the speed of `comptime` - The compiler has a lot of work to do here, and running a brainF** DSL at compile-time is pretty slow (speaking from experience - it was a really funny experiment). Will we have improvements to this section of the compiler any time soon?
Overall I'm really hyped for these new backends that Zig is introducing. Can't wait to make my own URCL (https://github.com/ModPunchtree/URCL) backend for Zig. ;)
You can change the `target` in those two linked godbolt examples for Rust and Zig to an older CPU. I'm sorry I didn't think about the limitations of the JS target for that example. As for your link, It's a good example of what clang can do for C++ - although I think that the generated assembly may be sub-par, even if you factor in zig compiling for a specific CPU here. I would be very interested to see a C++ port of https://github.com/RetroDev256/comptime_suffix_automaton though. It is a use of comptime that can't be cleanly guessed by a C++ compiler.
Good question! The TL;DR as I understand it is that it won't matter too much. For example, the self-hosted x86_64 backend (which is coincidentally becoming default for debugging on linux right now - https://github.com/ziglang/zig/pull/24072) has full support for most (all?) builtins. I don't think that we need to worry about that.
It's an interesting question about how Zig will handle additional builtins and data representations. The current way I understand it is that there's an additional opt-in translation layer that converts unsupported/complicated IR to IR which the backend can handle. This is referred to as the compiler's "Legalize" stage. It should help to reduce this issue, and perhaps even make backends like https://github.com/xoreaxeaxeax/movfuscator possible :)