dandinu

Correction - "Obviously, this is probably a bad idea."

That's a good question. It started off as pure technical curiosity (in the realm of: will this even work?). but as a side note, I always think of my screen time and the amount we spend chatting away on social media.

Obviously, this is probably and idea. I could imagine I connect my calendar to Hermes, and automate myself into 12 dinner plans and a trip to Disneyland on a Thursday afternoon just because I once mentioned I'm a Mickey fan to my nephew.

The method I used in the article is real and is pretty standard. Also, I do a decent amount of distillation and tinkering with weights for work, so I can assure you I did try that before resorting to good 'ol RAG.

Overall, even with a finetuning-as-a-serice like Tinker (the one from Thinking Machines) which is pretty cheap, the economics didn't work out that well.

Also, you probably one-shot this with Claude, I agree. But, you need to have an expensive Max subscription, which not everyone is willing to shell out 200 bucks for, just to have some weekend fun.

Author here. This is a personal weekend project that grew into a working WhatsApp bot. It replies as me to two allowlisted contacts (myself + one friend who knew about the experiment). The interesting part is not the agent framework but the retrieval I eventually got the best results with: every reply gets generated by Claude after pulling 8 of my real past replies to that specific contact, filtered by recency.

Built on Hermes Agent + Baileys + Chroma + nomic-embed-text-v2-moe + Claude Sonnet 4.6 via Azure AI Foundry. About 2 hours of work plus an hour debugging a WhatsApp multi-device LID issue. Total runtime cost: ~$0.005 per reply.

The bot is not running on a dedicated number. It is hooked to my primary WhatsApp, which is a ban risk I accepted in exchange for being able to test with real contacts. The killswitch (Telegram command that empties the allowlist and restarts the gateway) takes about 10 seconds. There is also a hard kill: unlink the device from WhatsApp on the phone, ~5 seconds, severs the bridge session entirely.

Happy to answer questions.

So the deal with AWE (Address Windowing Extensions) is that it lets 32-bit apps access memory above 4GB by essentially doing manual page mapping. You allocate physical pages, then map/unmap them into your 32-bit address space as needed. It's like having a tiny window you keep sliding around over a bigger picture.

The problemis that llama.cpp would need to be substantially rewritten to use it. We're talking:

  cAllocateUserPhysicalPages()
  MapUserPhysicalPages()
  // do your tensor ops on this chunk
  UnmapUserPhysicalPages()
  // slide the window, repeat

You'd basically be implementing your own memory manager that swaps chunks of the model weights in and out of your addressable space. It's not impossible, but it's a pretty gnarly undertaking for what amounts to "running AI on a museum piece."

Oh darn, you're absolutely right (pun intended) about the 32-bit situation. SSE2 is really the "floor" there if you want any kind of reasonable compatibility. I was being a bit optimistic with SSE4.2 even for 64-bit - technically safe for most chips from that era but definitely not all.

The Pentium Pro challenge though... pure x87 FPU inference? That would be gloriously cursed. You'd basically be doing matrix math like it's 1995. `-mfpmath=387` and pray.

I'm genuinely tempted to try this now. The build flags would be something like:

  -DGGML_AVX=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF \
  -DGGML_F16C=OFF -DGGML_SSE42=OFF -DGGML_SSSE3=OFF \
  -DGGML_SSE3=OFF -DGGML_SSE2=OFF  # pain begins here

And then adding `-ffast-math` to `CMAKE_C_FLAGS` because at that point, who cares about IEEE 754 compliance, we're running a transformer on hardware that predates Google.

If someone actually has a Pentium Pro lying around and wants to see Qwen-0.5B running on it... that would be the ultimate read for me as well.

Thanks for the kind words. Always fun to find fellow retro computing degenerates in the wild.

That's pretty accurate. I'm always amazed how much we move forward with technology, just to later realize we already had it 15 years ago.

regarding your question:

I have a 32bit XP version as well, and I actually started with that one.

The problem I was facing was that it's naturally limited to 4GB RAM, out of which only 3.1GB are usable (I wanted to run some beefier models and 64bit does not have the RAM limit).

Also, the 32bit OS kept freezing at random times, which was a very authentic Windows XP experience, now that I think about it. :)

there is a full technical write-up in the Github repo in "WINDOWS_XP.md": https://github.com/dandinu/llama.cpp/blob/master/WINDOWS_XP....

Sorry for failing to mention that.

Link to vcredist thread: https://github.com/LegacyUpdate/LegacyUpdate/issues/352

I just finished testing the Tinker API launched by Mira Murati's Thinking Machines and I can honestly say, even though it feels unfinished, it's pretty great. The setup is really smooth and with some minimal coding (and using their examples in the Tinker Cookbook) I was able to fine tune a Llama 3.1 8B Base on the Romanian language in under 20 min. The result was pretty decent and eventually I got it to write better poetry than the base model, in the target language.

I mostly did it because I got some free credits from them and I was curious what a company valued at 50 billion has to offer. The product overall feels like it's half-baked since there is no real interface, but the API does a lot of the heavy lifting in the backend while maintaining this local development feeling, which I personally find pretty cool.

I put the code up on Github, if anyone is interested, but I am curious what y'all think about their approach top fine tuning.

Hey everyone, stumbled upon this cool info about AI 'Mega-Personas' I thought I'd write a post on.

Everyone knows in AI, 'Personas' are like simulated fictional characters. But 'Mega-Personas' takes the concept up a notch. Consider this: You instruct an AI to simulate 5 lawyers but adding unique details to each one to create diversity in responses. Now imagine the information flow if you simulated 10.000 lawyers all with unique attributes - that’s the main concept of 'Mega-Personas'.

Can't wait to hear your thoughts.

They removed the Azure templates used to deploy as well, so I created an up to date tutorial on how to deploy the whole thing manually: https://tensorthinker.hashnode.dev/privategpt-a-guide-to-usi...

Apparently, they removed the Azure templates used to deploy as well, so in frustration, I created an up to date tutorial on how to deploy it manually: https://tensorthinker.hashnode.dev/privategpt-a-guide-to-usi...

or this: https://github.com/dandinu/azurechatgpt

I managed to fork it here: https://github.com/dandinu/azurechatgpt

Well, I guess the myth that Facebook is listening in on your conversations and showing you relevant ads are coming true after all. It feels like a self-fulfilling prophecy.

Not at all, I like to believe I wanted to convey the exact opposite. I agree that is not something altruistic, exactly my reason for saying this:

> I knew I made the right decision, not because the candidate was a specific sex, but because one was a better fit for the team’s work style.

Looks interesting and from the video tour I was rather convinced to use it. I saw that it does not have a grid system to snap the columns and text at proportional sizes/positions. That's important for an infographic creator, otherwise I can stick to Gimp or Illustrator.

This is Europe's way of trying to make users read the Terms & Conditions of every website.

The intentions of the law are good, but I do not believe the approach is that smart. They could have made the companies who own the websites place cookies on the users machines only after they have actually created an account... that seems like consent to me.

I believe you are dismissing Chrome+Gmail too fast. With Google Gears, i think it´s as competitive.

Sweet!! I love it.

One question: Can this library be extended to more specific areas of the body? Like detecting if somebody is naked?

I´m serios, i´m working on an adblocker especially for this.

You will need a lot more than a website and a banner to stop flame wars. This disscussion will probably become a confirmation of that rule, really fast.

I don´t find how it would actually help me... i need more insight.