Hacker News Reader

ingenieroarielon Arcan 0.6.3 – I, PTY, The Fool [video]

Associated blog post: https://arcan-fe.com/2023/11/18/a12-visions-of-the-fully-net...

Arcan FE Discord Server: https://discord.gg/pjZeG3KPgc

Arcan builds with CMake and needs Freetype2, OpenAL, SQlite3, Lua and (SDL | Mesa, libdrm), (Wayland | Xorg).

ingenieroarielon Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

thanks! I am still curious about the sensor inputs part.

I am trying to replace the 2.4ghz controller on my electric skateboard to make 0 to 5kmh and braking more pleasant and maybe use gyroscopes to do away with the controller altogether. What would tokens be in that case? Do you create a CAN style representation and feed that to the llm? What kind of throughput do you foresee being possible on which hardware?

ingenieroarielon Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

Pretty cool!

> My research goal is to train models using various hardware telemetry data with the hope that the models learn to interpret sensor inputs and control actuators based on the insights they glean from the sensor inputs. This research direction may open up exciting possibilities in fields such as automation, space, robotics and IoT, where L2E can play a pivotal role in bridging the gap between AI and physical systems.

This part was easy to miss but quite interesting, could you expand a bit here? What does L2E stand for?

ingenieroarielon Instead of collaborating or supporting me, Google stole my idea

I have a lot of respect for Oz's ability to ship and skills / imagination.

Only thing I can think of is for people to think thoroughly the license they set. BSD in this case, 4 years ago. And to not take any contributions unless they sign a CLA so the license can be changed in the future.

As for my own point of view, I screen apps/libraries that are not MIT/BSD/Apache2 and I may want to link statically to or sell services for and try to sponsor on Github projects I rely heavily on.

[1] https://github.com/ozramos/handsfree/blob/master/LICENSE

ingenieroarielon Llama 2

I filled the form about an hour ago and got the download link 15 mins ago. Download is ongoing.

Direct link to request access form: https://ai.meta.com/resources/models-and-libraries/llama-dow...

Direct link to request access on Hugging Face (use the same email): https://huggingface.co/meta-llama/Llama-2-70b-chat-hf

Direct link to repo: https://github.com/facebookresearch/llama

Once you get a link to download on email make sure to copy it without spaces, an option is to open it in a new tab and then download. If you are using fish or another fancy shell, make sure you switch to bash or sh before running download.sh from the repo.

I am not sure exactly how much space is needed but it is likely north of 500GB given that there are two 70B models (you are given the option to download just the small ones in a prompt).

Edit: The_Bloke on HF already has them in GGML format available for download.

https://huggingface.co/TheBloke/Llama-2-7B-GGML https://huggingface.co/TheBloke/Llama-2-13B-GGML

ingenieroarielon I Built Linux from Scratch

I have used NIXOS_LUSTRATE (wipes the other Linux on the next reboot) on several systems where it was not supposed to be used. Once you build nix, you should be able to boostrap NixOS from there.

ingenieroarielon Apple Introduces M2 Ultra

$3,999.00 is the entry price for the M2 Ultra. The M1 Ultra is supported by Asahi Linux and I have to say the memory bandwidth makes things like running llama.cpp way faster than Intel CPUs. The rust GPU driver is also a joy to use.

Compared to the price of Two very good nvidia cards and the PC to host it I think the price is reasonable for running Linux on high end Apple Hardware.

Another thing is that the memory consumption is under 20W when idle and 100W when compiling the Linux Kernel. I think you make up the money just in electricity after a few years.

ingenieroarielon Don Knuth plays with ChatGPT

I suggest playing with the unfiltered LoRA versions of LlaMA like Wizard Vicuña/ Mantecore, essentially GTP4 was lobotomized during the months before we could play with it so that it was "nice", but the raw version (non fine tuned) would have probably satisfied your use case.

ingenieroarielon ChatGPT app for iOS

Same, I tried Whisper and received a call right after and it was unusually hot.

ingenieroarielon Open-Llama: Complete training pipeline for building large language models

WebGPU would be a way to shift that cost back to each device. RedPajama 3B could become useful for some tasks and run quite fast on most hardware available. Then as users have better computers, they can get access to better models?

ingenieroarielon Are We Sixel Yet

I have been trying out a kakoune arcan frontend [1] with the intention to use it over the network along with cat9 [2].

[1] https://github.com/cipharius/kakoune-arcan [2] https://github.com/letoram/cat9

ingenieroarielon DIY Adapter: Using an e-bike battery to run a laptop

I am doing a similar thing but for a Mac Studio running Asahi Linux (13 W idle, 215 W max), since DC/DC was not an option I am using https://www.amazon.com/CAT-Professional-Station-Starter-Comp... (200W max) which was half price at Costco. A power adapter that can also inflate tiers and jumpstart other veihicles.

Would anyone know if I can charge it while the computer is plugged? I usually assume that is a no / no, but if I were to try, could something explode?

ingenieroarielon Datomic is Free

Today I looked up pgvector's NixOS availability. For the past 15 years I have relied on postgis source being available and improved by the community for my day to day business.

My point is that the option to modify the source results in software bein available and community maintained in a way that binary only isn't. Even if I change the source myself just twice a decade.

ingenieroarielon Datomic is Free

I saw Fabrice Bellard do it this year (MIT): The CPU version is released as binary code under the MIT license. The GPU version is commercial software. Please contact fabrice at bellard dot org for the exact terms.

https://bellard.org/ts_server/

ingenieroarielon Databricks Releases 15K Record Training Corpus for Instruction Tuning LLMs

Thank you, thank you, thank you!

If possible, could you share how Dolly v2 compares to RWKV-4 14B ctx 8019?

ingenieroarielon Databricks Releases 15K Record Training Corpus for Instruction Tuning LLMs

Ditto, this is release early release often without necessarily meaning move fast and break things. Other teams can do the equivalent of Alpaca to Llama and we can all learn for the next round.

ingenieroarielon Terence Tao on GPT-4

But chatgpt can crawl a semantic web and use it without us knowing.

I have asked a langchain bot about wikidata ids for specific places, links to the page, to read it and then to answer facts about places and got very good results instead of made up numbers.

Wikidata links to FIPS codes, OSM ids, GeoNames and that gives us an opening to link against the cool datasets from Flickr, Foursquare and others who have created gazetteers.

To me, Semantic Web was dead on arrival because of its UX, but now a semi-smart agent can help us get past the UX problems and jump from plain text to json output.

ingenieroarielon The Coming of Local LLMs

tinygrad

https://github.com/geohot/tinygrad/tree/master/accel/ane

But I have not tested it on Linux since Asahi has not yet added support.

Same machine but OSX, llama.cpp runs at 18ms per token (7B) and 200ms per token (65B) on CPU using float16.

ingenieroarielon The Coming of Local LLMs

They are comparable to the first ChatGPT proof of concept from say early 2022. The reason many of us are excited is because we may be a year or two away from being able to run a ChatGPT if the open source models follow a similar curve.

ingenieroarielon The Coming of Local LLMs

In order to run large language models, we should all be buying a fully loaded Mac Studio (128GB of ram, 20 CPU cores, a lot of GPU and Neural cores.) and putting Linux on it to remove the artificial restrictions.

Yes, we will be running them soon in low end hardware, but we need to get at least to GPT-3.5-turbo level of inference speed and quality before we try to make it small.

I already started.

    $ neofetch
                   -`                    x@decpti 
                  .o+`                   ------- 
                 `ooo/                   OS: Arch Linux ARM aarch64 
                `+oooo:                  Host: Apple Mac Studio (M1 Ultra, 2022) 
               `+oooooo:                 Kernel: 6.1.0-rc6-asahi-4-1-ARCH 
               -+oooooo+:                Uptime: 4 hours, 23 mins 
             `/:-:++oooo+:               Packages: 177 (pacman) 
            `/++++/+++++++:              Shell: bash 5.1.16 
           `/++++++++++++++:             Resolution: 1920x1080 
          `/+++ooooooooooooo/`           Terminal: /dev/pts/0 
         ./ooosssso++osssssso+`          CPU: (20) @ 2.064GHz 
        .oossssso-````/ossssss+`         Memory: 717MiB / 129540MiB 
       -osssssso.      :ssssssso.
      :osssssss/        osssso+++.                               
     /ossssssss/        +ssssooo/-

ingenieroarielon Humans in Humans Out: GPT Converging Toward Common Sense in Both Success/Failure

https://gist.github.com/ingenieroariel/39bc74f56d7adef437ed4...

Linus Torvalds (Creator of the Linux Kernel) - https://github.com/torvalds John Carmack (Co-founder of id Software) - https://github.com/ID_AA_Carmack Bjarne Stroustrup (Creator of C++) - https://github.com/BjarneStroustrup Fabrice Bellard (Creator of QEMU, FFMpeg, and Tiny C Compiler) - https://github.com/fbellard Andrei Alexandrescu (C++ expert and author) - https://github.com/incomputable Chandler Carruth (LLVM and Clang developer) - https://github.com/chandlerc Daniel Lemire (Computer science researcher, focuses on performance) - https://github.com/lemire P.J. Plauger - A renowned author, and contributor to the C Standard Library - https://github.com/pjplauger Peter J. Weinberger - Co-creator of AWK and a contributor to Unix - https://github.com/pjw Keith Packard - A prominent contributor to the X Window System, and the Linux graphics stack - https://github.com/keith-packard Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker Julia Evans - Systems programmer known for her work on C projects and educational materials - https://github.com/jvns Tony Finch - Author of several C projects, including the Knot DNS server and other infrastructure tools - https://github.com/fanf Sergey Bratus - A security researcher and expert in low-level programming, with a focus on C - https://github.com/sergeybratus Herb Sutter - A prominent C++ expert, author, and chair of the ISO C++ standards committee - https://github.com/hsutter Sean Parent - A software engineer at Adobe and C++ expert, known for his talks on C++ best practices - https://github.com/sean-parent Kate Gregory - A C++ expert, author, and Microsoft Regional Director - https://github.com/gregcons Eric Niebler - A C++ expert and author of the range-v3 library, which inspired C++20's ranges - https://github.com/ericniebler Jason Turner - A C++ expert and host of the C++ Weekly YouTube series - https://github.com/lefticus Titus Winters - A software engineer at Google and C++ expert, known for his work on the Google C++ Style Guide and Abseil - https://github.com/tituswinters Jonathan Boccara - A C++ expert and author of the book "The Legacy Code Programmer's Toolbox" - https://github.com/joboccara Richard Stallman - Founder of the GNU project, which aims to create a complete, portable, and free Unix-like operating system - https://github.com/rms Samuel L. Miller - A core developer of the SQLite project, a self-contained, serverless, zero-configuration, transactional SQL database engine - https://github.com/slmiller Frank Denis - Creator of the libsodium project, a modern, easy-to-use software library for encryption, decryption, signatures, password hashing, and more - https://github.com/jedisct1 Michael Kerrisk - Author of "The Linux Programming Interface" and contributor to portable, multiplatform code in the Linux kernel and glibc - https://github.com/mtk-mirror Salvatore Sanfilippo - Creator of the Redis project, an open-source, in-memory data structure store, used as a database, cache, and message broker - https://github.com/antirez Niels Provos - A security researcher and developer who has contributed to various projects, including libevent, a library that provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout - https://github.com/provos Alon Zakai - Creator of Emscripten, a compiler that allows developers to run C and C++ code on the web, using WebAssembly - https://github.com/kripken Brian W. Kernighan - Co-author of "The C Programming Language" and "The Practice of Programming" - https://github.com/bwk Dennis Ritchie - Creator of the C programming language and co-author of "The C Programming Language" - https://github.com/dmr (GitHub profile not available, as he passed away in 2011) Dan Gookin - Author of "C For Dummies" and other programming books - https://github.com/DanGookin Zed A. Shaw - Author of "Learn C the Hard Way" and other programming books - https://github.com/zedshaw Mark Lutz - Author of "C Programming for the Absolute Beginner" and Python books - https://github.com/learning-python Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker William A. Rowe Jr. - A key contributor to the Apache Portable Runtime (APR) project, which includes a custom libc implementation for the Apache HTTP Server - https://github.com/wrowe Stefan Tomanek - Creator of dietlibc, a libc optimized for small size - https://github.com/stefan-tomanek (The dietlibc project itself doesn't have an official GitHub repository, but you can find it at https://www.fefe.de/dietlibc/) Szabolcs Nagy - A contributor to the musl libc project and creator of the math library used in musl - https://github.com/szabolcsnagy Alexei Starovoitov - Creator of the uClibc-ng project, a small C library for developing embedded Linux systems - https://github.com/ast-uClibc (The uClibc-ng project can be found at https://github.com/wbx-github/uClibc-ng)

ingenieroarielon Humans in Humans Out: GPT Converging Toward Common Sense in Both Success/Failure

After reading the abstract I can't help but think we only want to feed a coding AI good code. Is there an awesome-code repo that lists all the repos we consider above average?

ingenieroarielon Linux 6.4 Bringing Apple M2 Additions for 2022 MacBook and Mac Mini

On my usage the battery lasts about half the time. (Apple MacBook Air M1, 16GB). NixOS on the other side and a lot of compilation. Sleep works.

(edit) I misunderstood your question. I have charged my iPhone in a port and have also used an external audio card that drew power from it. Not sure it behaves as well as OSX but it works.

ingenieroarielon Using mmap to make LLaMA load faster

The post is a technical article that describes a very cool systems engineering approach to a problem that is usually coded away in proprietary code from nvidia.

No wonder a lot of people see it as "mmap, nothing new". But this is not the case in a lot of libraries where the norm is to just budget for a lot of time moving things to/from gpu and just relying on someone else's code.

Instead of accumulating technical debt the owner of the repo decided to merge this, have a breaking change and move on. When there was some community backlash to the breaking changes there was a pull request trying to revert all changes instead working through the issues (it was a net win for several users but not all, some configurations with slower drives were better served by the older approach). There was an ugly back and forth and the repo owner decided to ban both the person who did the pull request and the author of this post.

This article brings the conversation back to the technical merits, the roadmap, credit the the original authors and tones down the ownership tone that may have pissed off some community members.

That pull request has now been closed by the owner of the repo. They are trying to move on and be productive, let's do the same.

ingenieroarielon Using mmap to make LLaMA load faster

Should we be linking flagged items? That discussion was very ugly and unproductive.

ingenieroarielon Pairing With GPT-4

> If you give GPT-5 too broad of a problem to solve, it can get lost and forget things

For a second there I thought I needed to read more articles per day in order to not miss the release of a new model!

ingenieroarielon Fork of Facebook’s LLaMa model to run on CPU

Try switching all the .cuda() to .mps() I got a 100x speedup on a different language model on a Macbook M1 Air.

https://pytorch.org/docs/stable/notes/mps.html

ingenieroarielon Nix-Powered Development with OCaml

Seems you would need to first override ocaml and then override the ocaml used in mkOcamlPackages. [1]

  let o = ocaml.override { flambdaSupport = true; }; in
  let oP = ocaml-ng.mkOcamlPackages o (self: super: {}); in
  oP.apron

[1] https://discourse.nixos.org/t/install-ocaml-flambda-compiler...

ingenieroarielon Nix-Powered Development with OCaml

Most hardcore Nix users/developers I have met have been suspicious for Flakes for several years, so your point rings true.

That said, it feels like they are slowly coming to terms with it and just accepting it as default. Here are two examples of maintainers eventually accepting flake support on their repos after initial hesitation [1][2].

[1] https://github.com/tpwrules/nixos-apple-silicon/pull/47 [2] https://github.com/NixOS/mobile-nixos/pull/404

ingenieroarielon Nix-Powered Development with OCaml

If you are mainly using OCaml and do not mind flakes, this was a very good starting point for me: https://github.com/brendanzab/ocaml-flake-example/blob/main/...

Using Flakes you can have a devShell, a check target and a build target neatly organized.