breakds

The only thing that I carry all day besides my laptop is a flash disk with ventoy in it.

Curious on "as research tools" - do you use it to manage research papers?

I am curious which market you are talking about and how do you get or pay for the data? Thanks!

I had two successful hires who just graduate from college, with no machine learning experience (major in accounting and civil engineering). With 3 months training by working on real world projects, they become quite fine machine learning engineers.

You probably do not need AI experts if you just need good machine learning engineer to build models to solve problems.

Just in case anyone uses NixOS, the following simple piece of code just works for me, and I spent no time setting up a new NixOS machine that enables remove desktop with RDP:

https://github.com/nixvital/vital-modules/blob/main/modules/...

Not defending anyone, just saying.

- If I have $1000 I might choose to spend $2 for a gambling. - If the trader have $10M already, it might be reasonable for him/her to spend $20k gambling on some rumors.

This guy could be someone who is already rich.

It is not only about computation, but also the communication bandwidth and memory, especially in training large models. The top consumer GPUs e.g. 3090/4090 still cannot beat H100 in this area even if a lot of techniques are applied.

I bought a google nest hub on my office desk specifically for having meetings with Google Meet. Yesterday it tells me I can no longer do that after Sep 28. WTF.

You don't have to stop working. There are many interesting jobs that do not pay very well. Having a few million in your pocket give you the privilege to consider those jobs.

There are quite some wealthy people who happens to understand technology well and use and contribute to many open source tools.

NixOS and never look back.

My homelab has

1) a NAS/Server with 10700 + 12 TB HDD x 15 forming a ZFS, with 128GB of RAM.

2) A pair of machine learning servers, each of them with a threadripper 3955wx, 256GB RAM and two 4090s

3) A more powerful machine learning server, with a threadripper 5975wx, 512GB RAM and two 4090s.

All of them run NixOS. They are mainly used to serve a bunch of services: hydra/nix binary cache, shiori, grafana/prometheus, paperless, syncthing, filerun, assistant etc. And also training jobs all the time.

I would highly recommend NixOS to run on your servers for the services.

Or if you are using Nix use https://github.com/nixvital/ml-pkgs

  outputs = { self, nixpkgs, ... }@inputs: {
    overlays = {
      dev = nixpkgs.lib.composeManyExtensions [
        inputs.ml-pkgs.overlays.torch-family
        # Add some other overlays
      ];
    };
  };

in your flake.nix and you can use pytorch 2.0.1 compiled with CUDA 11.8 running on your 4090s. Downside is that the first time you will have to compile it and it can be quite long.

For flake.nix, you add that to the inputs, for example, call it `nixpkgs-old`, and then reference the package of interest with:

`nixpkgs-old.legacyPacakges.linux-x86_64.node`, or something like below

  let pkgs = import nixpkgs-old {
    system = "linux-x86_64";
  } in pkgs.mkShell {
    packages = [ pkgs.node ];
  }

I use Nix to manage my machine learning development environment: https://github.com/nixvital/ml-pkgs

Sure after the building the binary is HUGE. But I only have to build it once and cache it so that all my workstations and training servers can use it.

That is the basic version. The EDU version which opens more API/interface to the user cost much more. Also training a whole-body (quadruped + robotic arm with gripper) to do something like picking strawberries remains challenging today.

Have been using flakes in production for more than 2 years - I think it is stable enough for daily use.

This is very true. There need to be a Nix guy in the team to bootstrap and be the evangelist. My experience is that in one week everyone gets very familiar with the workflow, and in 3 months all of my team become experienced Nix users.

I have been managing all my existing and new projects with nix for a few years and never look back. Guaranteed to build and run on all my machines.

nix flake new --template "github:nixvital/flake-templates#cpp-starter-kit" my-project

will create a skeleton for my new C++ projects.

After reading the title, I was thinking "Agreed, adding type to your code definitely make it more readable".

Same here. I do not have any opinion towards systemd, but using it together with NixOS is really simple and hassle-free.

I have a bunch of 3090s in the garage for machine learning almost 24x7. The garage has no AC at all. None of them have failed yet.

Those programs are most useful in teaching you how to self-teach, which seems like a skill that you have already mastered and excel at.

The degree can definitely make your resume look better, but past work experience and projects speak much much louder.

I have been there. The progress was rather slow until I started to use NixOS. The learning curve is a bit steep but is very rewarding. It is not specific to self-hosting stuff, but as a side effect it makes self hosting super easy (declarative, readable, etc).

For most of the services that you would like, you just write a simple configuration and deploy it. For example, to run the service shiori (https://github.com/breakds/nixos-machines/blob/main/machines...), or to host a game (terraria) server (https://github.com/breakds/nixos-machines/blob/main/machines...), or tailscale (https://github.com/breakds/nixos-machines/blob/main/base/tai...). Since Nix is also a very good package manager, you also do not have to deal with installing packages and managing their dependencies.

With my NixOS server I am running all the services you mentioned.

> But I have no clue about networking,

My router is just a bunch of services running on a NixOS box (with this you have absolute control over the firewall/gateway, and it is also good experience to learn the networking stuff with NixOS). Note that before this I know nothing about the networking stuff as I skipped the class in college ...

Just want to say that filerun is a really good option for self-host NAS with WebDAV support as well as a nice web app. Would definitely recommend.

Even though you use filerun to host files, you can still point Lychee/Plex etc to the photos and videos, which might be inside a subdirectory under the root directory of your filerun instance.

But in reality, it seems that working on something that interests you, is a privilege.

Not an apple fan, but I cannot be separated from my iPad (and the pencil) for this particular use case.

Which is why I am using Nix to maintain my development for each machine learning research project.

Yes it by default build everything from source. However, most of the packages will have binary cache on https://cache.nixos.org/ so that your installation or update will download them instead of building them. Also you can setup your own binary cache (https://nixos.wiki/wiki/Binary_Cache) and building machine to make your own projects build faster.

The thesis is a master piece, highly recommended!