DanielHB

somewhat related:

I have had to do a last-minute fix on a build because a developer added emojis to success messages

a fire emoji

on software for handling industrial machinery that can explode

our QA department was not happy about that let me say that...

The reason is simple: the package manager works well. Helps if the package manager is standardized and there is a "default" registry for open source projects.

I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.

To be honest I am not even bothering learning to use these harnesses much less tweaking the bare minimum configs.

I fully expect I will be switching every couple of years to a new one. And even if I don't, they will be very different then than they are today.

It is kinda weird, because you feel so much more tired all the time. Yet at the same time you have so much more motivation to do _everything_. It is like I feel 50% more tired, yet 200% more engaged with everything I do. It really makes your life a lot more meaningful.

Before I would waste so much time engaging on video games / internet-browsing / TV-youtube / etc and just not enjoying the time I spent. It felt very meaningless, like why even bother working? I have (had) savings to go take several years of sabbatical if I wanted.

When I actually boot up a video game today I actually enjoy it. And that is all on top of normal stuff you hear about having kids.

It is funny there is no similar treatment about TBE (tick disease) which is predominantly an European disease and very dangerous and much more common than Malaria is in most (all?) South American countries...

https://en.wikipedia.org/wiki/Tick-borne_encephalitis

I have a german last name, do you know how often people outside Brazil act weird when they learn I am Brazilian?

My grandparents came to Brazil right after WW1 way before the Nazis came to power. High ranking Nazis fled to south america because there were a lot of germans living there already. Nearly all german people who moved to south america did it way before WW2.

I just run into this stuff a lot living in Europe.

Just to dispel some Brazil myths:

1) +95% of the population live on the coast very far away from the Amazon. Most of the population has not been there. Most of the coast has a very different jungle biome called Mata Atlantica and the countryside close to the coast is not that different from temperate forest of Europe. That is what most all Brazilians are used to. There is a significant population in the arid northeast though and the cold south as well (which is even more similar to europe).

2) Manaus is the biggest city in the Amazon and it is huge developed place (and has been for decades). You are not in the middle of the jungle if you land in the airport. The countryside around the city is jungle though.

3) Brazilian people do not necessarily like or are used to tacos and spicy food. Mexico is _really_ far away from Brazil.

I varies widely on piece, but even some (not all) of the cheapest wood stuff they have are pretty remarkable quality (or maybe I should say durability, the design side is still super simple for flat-packing optimizations). The non-entry-level stuff is quite good in my opinion.

But on some other stuff I tend to agree, their sofas are pretty bad in my opinion. The beds are okay, but even the non-cheap ones aren't "great".

I believe the power gains from apple stuff mostly come from CPU and memory interconnect, unrelated to consumer-grade PC GPUs. Server-grade GPUs have different memory interconnect architecture which I assume already has similar power efficiency gains.

I think raw flops per watt come mostly from fab process, not architecture. This was my original point, fab process is not getting better at a linear (much less exponential) scale anymore.

Like I said, just feels I have from the consumer-hardware space. For several generations of GPU now most improvements come from packing more transistors into a larger die than packing more transistors closer to each other.

GPUs have been getting physically bigger with huge heatsinks and fans to support those bigger dies power consumption. Just compare the TDPs:

2020 RTX 3090: 350W

2022 RTX 4090: 450W

2025 RTX 5090: 575W

Bigger dies means lower capex of course, but the similar opex (maybe slightly lower as there is less physical hardware to maintain).

I seen some specialized hardware like google's TPUs. Not sure how they compare on performance per watt with GPUs though. Regardless the manufacturing processes are still the same (EUV) which is the thing that hasn't been improving. A fully optimized specialized hardware can at most deliver a single-time linear improvement (that could be very significant, for example 30% is still huge of course) and then little compared to normal GPUs.

I don't think renewable power generation is going to massively reduce costs for data centers, especially considering power transmission hasn't meaningfully reduced in cost. If anything the only thing that I think will have significant impact for data centers would be dedicated nuclear power plants physically located right next to the data center.

In fact I expect power generation to get more expensive as demand can increase faster than supply can be established. I imagine setting up new solar farms and transmission lines to be significantly harder (as in, takes longer time due to approvals and so on) than new data centers (which requires a single large location and I assume less approvals).

I run qwen3.5-9B on an RTX 3080 with 10GB of vram. It runs at ~77tk/s with around 50k context size.

As soon as I switch to a model that doesn't fully fit into vram it tanks to <10tk/s which makes it unusable for me for most tasks.

From my experience if it doesn't fit on vram it is rarely worth to bother except for a few narrow tasks.

For example make an essay about something where you don't actively engage with the LLM after the initial prompt. So mostly one-shot prompts.

Is the hardware really getting better? It feels performance per watt is not getting better at all which is the metric that will matter eventually when supply-demand stabilizes.

As it is, it seems the improvements are about making the hardware cheaper (as in capex, not opex).

This is just feels from me from what I hear on the news and see on the products though.

I am very much a beginner to local LLM stuff and I find it incredibly hard to figure out how to run models optimally with the correct settings for my hardware. The number of different variations of the same model and how each quant work is super confusing as well.

When I tried to run llama.cpp directly I was getting max 9tk/s on qwen3.5-9B, then I tried LM Studio with the same model and got 77tk/s. I haven't figured out yet how to get MTP working properly in either.

I live in Sweden now, so yes I have seen some IKEA furniture around.

IKEA sells spare parts and for small items (like bolts and fittings) will send them for free.

I disagree with this sentiment, IKEA furniture is much more sustainable than artisanal furniture. Most of the "back in the day" furniture came from tearing down virgin woods. Sorting + shredding-recycling + making new "cheap particle board crap" furniture takes a lot of energy, but not a lot of non-renewable resources. With increased electrification this problem is being drastically reduced and the wood is sourced from re-grown tree farms (which also have their environment problems, but it is nowhere near as bad)

I am from Brazil and before the recent soy-growing deforestation of the amazon the #1 cause of deforestation was furniture-making. My parents have a really old dinner table made from high grade wood, is it is a pain in the ass to move and it has a ton of scuffs we can't be bothered to fix. Once my parents move we will probably throw it away. It lasted 40 years (with a big varnish rework around 25 years in) but the trees used to make it will never come back, my ikea table has been with us for around 8 years and still works fine.

Particle board tech has come a long way and it is much better than it used to be, IKEA particle board is consistently "okay" to "great", but other manufacturers it is not guaranteed. The only thing it lacks (by design) is the weight, particle board will be much lighter than "normal wood" furniture, which is both good and bad depending on the situation (some types of furniture you want to be heavy).

Yeah, the analogy is more applicable to reusable software that used to be hand-crafted. For example many enterprise systems like ERPs, CRMs, CMSs, etc.

Even in that space there is still plenty of hand-crafted solutions or customization to the enterprise-system that might as well be hand-crafted solutions.

English Wikipedia is around 25GB compressed

I wonder how much more effective LLMs would be at general knowledge if you just download wikipedia and set up an MCP for it.

What kind of things you only get with a larger model?

wait what? I know nothing about this.

Oh god my company just switched to .io and that was a huge PITA. I hope we don't go .ai...

I used briefly some 8 years ago, it is very good for getting full-native (including native UI toolkit) cross-platform desktop apps. It can build self contained binaries that just work.

My use-case was building a simple USB-stick-portable application for windows and it was great for that.

I don't think so, I think most LLM content is bot generated and amending bots to remove watermarkers would be trivial if the bypass is trivial.

You are just pointing out the very visible single cases. But the mass of low-visible content is much higher and more dangerous (like propaganda bot-farms). If a social media platform adds watermarker checks the bots would implement bypasses ASAP.

It seems like it would be so low effort to bypass, especially when you can just train a system (maybe even another LLM) using the watermarker validation from Anthropic themselves.

It seems it would get as simple as:

  outputText = promptLLM(prompt)
  scrubbedText = scrubWatermark(outputText)

Might help with students and low-technical people passing off work as their own, but any industrial scale slop-generator should be able to bypass it trivially.

But what prevents someone from using Anthropic own detection system to train a watermark-scrubber?

Seems like this would only catch the most unsophisticated cases.

I have tried running llamma.cpp on my PC and I found it hard getting it to run at decent speed. On Qwen 3.5-9b I get at most 10tk/s.

I eventually switched to LM studio and the same model runs much better, like 70tk/s.

Not sure if it was because I was running llama.cpp inside podman or badly tuned LLM arguments. But LM studio is unfortunately much more practical.

Although I agree with you. I do not really know what kind of telemetry LM studio is running and I would rather not be using it.

One thing I realized is just how much offline local models can hurt mass data collection.

For example, I needed to write an invitation letter for immigration control for a relative visiting me. Previously I would have used a search engine for a template. Today I fire up my local qwen 3.5-9b for this kind of stuff and feed it all the private data I need.

Unfortunately it is unlikely the average user will known how to avoid this data collection. Even if the LLM is local you are likely feeding the prompts to remote servers if you harness/chat-interface is not properly vetted.

I have been running Claude Code using its built-in /sandbox feature with this config:

  "sandbox": {
    "enabled": true,
    "failIfUnavailable": true,
    "autoAllowBashIfSandboxed": true,
    "allowUnsandboxedCommands": false,
    "filesystem": {
      "allowWrite": [
        "."
      ],
      "denyRead": [
        "~/*"
      ],
      "allowRead": [
        ".",
        // a few more dirs
      ]
    }
  },
  "defaultMode": "auto"

But that doesn't feel nearly safe enough. What is the most pragmatic way to run agentic AI properly isolated?

I am guessing:

1) Run the harness inside a docker container

2) Volume-mount my project folder (and depending on the dev stack the dependencies folder) into the container

3) Install dependencies from outside the container (no private registry keys)?

4) Run git commands from outside the container (no git ssh key)

Anything I am missing?