omneity

Also dishonest since the reason Qwen 2.5 got so popular is not so much paper performance.

Crazy timing!

I shipped a similar abstraction for llms a bit over a week ago:

https://github.com/omarkamali/borgllm

pip install borgllm

I focused on making it Langchain compatible so you could drop it in as a replacement. And it offers virtual providers for automatic fallback when you reach rate limits and so on.

Eventually through experience and self-play with the technologies in question.

Is it actually growing, or is it swarm-like assembly, such as a coral colony?

I just posted incidentally about Wikipedia Monthly[0], a monthly dump of wikipedia broken down by language and cleaned MediaWiki markup into plain text, so perfect for a local search index or other scenarios.

There are 341 languages in there and 205GB of data, with English alone making up 24GB! My perspective on Simple English Wikipedia (from the OP), it's decent but the content tends to be shallow and imprecise.

0: https://omarkama.li/blog/wikipedia-monthly-fresh-clean-dumps...

They did and then spent $5B purchasing Instagram and Whatsapp to recover.

Asking the fox to guard the hen house..

Hello HN!

I've been developing a lot of LLM-based apps recently, and I got tired of writing the same glue code each time to switch providers, find the base URL again, or other strategies to deal with provider idiosyncracies, so I extracted out that code into an MIT licensed package named BorgLLM.

It's a zero-config Langchain client to 100s of providers & models, it has automatic API key rotation, rate-limit fallback strategies and more concerns you won't have to put in your application code...

One liner to rule them all (you can pick any model or any provider):

llm = create_llm("anthropic:claude-sonnet-4")

And my application code is finally clean, focusing on the domain. It's not a very complex package yet it might be one of the biggest time savers I added to my development workflow.

I hope you'll find it useful. Questions and feedback points are most welcome!

My thoughts exactly. Or at least curate a sample of true positives and use zero shot classification with semantic embeddings + keywords.

Looking at the post, the methodology (checking the presence of a few keywords, not even including "LLM") is very simple with a lot of potential false negatives. 20% is closer to the lower bound.

It could also explain the lower votes, "AI" or "GPT" being more generic terms is correlated in my personal experience with lower quality.

Hello HN!

I've been developing a lot of LLM-based apps recently, and I got tired of writing the same glue code each time to switch providers, find the base URL again, or other strategies to deal with provider idiosyncracies, so I extracted out that code into a package named BorgLLM.

It's a zero-config Langchain client to 100s of providers & models, it has automatic API key rotation, rate-limit fallback strategies and more concerns you won't have to put in your application code...

One liner to rule them all (you can pick any model or any provider):

llm = create_llm("anthropic:claude-sonnet-4")

And my application code is finally clean, focusing on the domain. It's not a very complex package yet it might be one of the biggest time savers I added to my development workflow.

I hope you'll find it useful. Questions and feedback points are most welcome!

I've been quite obsessed about ramping up (technically complex, not basic crud/wrappers) SaaS development with Gen AI tools, speeding things from months to weeks to days. But then I hit a snag: operations are the new bottleneck. How can I support all of these products, let alone promote them or find customers? My focus shifted to agents, and I realized that access for these AI bots was a major hurdle, despite all the MCPs available.

The thing is, we’ve been retrofitting software made for humans for machines, which creates unnecessary complications. It’s not about model capability, which is already there for most processes I have tested, it’s because systems designed for people are confusing to AI, do not fit their mental model, and making the proposition of relying on agents operating them a pipe dream from a reliability or success-rate perspective.

This led me to a realization: as agentic AI improves, companies need to be fully AI-native or lose to their more innovative competitors. Their edge will be granting AI agents access to their systems, or rather, leveraging systems that make life easy for their agents. So, focusing on greenfield SaaS projects/companies, I've been spending the last few weeks crafting building blocks for small to medium-sized businesses who want to be AI-native from the get-go. What began as an API-friendly ERP evolved into something much bigger, for example, cursor-like capabilities over multiple types of data (think semantic search on your codebase, but for any business data), or custom deep-search into the documentation of a product to answer a user question.

Now, an early version is powering my products, slashing implementation time by over 90%. I can launch a new product in hours supported by several internal agents, and my next focus is to possibly ship the first user-facing batch of agents this month to support these SaaS operations. A bit early to share something more concrete, but I hope by the next HN thread I will!

Happy to jam about these topics and the future of the agentic-driven economy, so feel free to hit me up!

This is an incredible achievement for a solo developer. The dev is from the Deepseek team by the way.

It will work, but at the scale needed for pretraining you are bound to have many quality issues that will destroy your student model, so your data cleaning process better be very capable.

One way to think of it is that any little bias or undesirable path in your teacher model will be amplified in the resulting data and is likely to become over represented in the student model.

kinda? omarkama.li

I am not entirely surprised by the relative equivalence for the sparse model. The combined bandwidth of 2x 5060 Ti ≃ 1x 3090. There are inefficiencies in multi-gpus that are more negligible at smaller dimensions, hence why the dense 32B model performs significantly worse on the dual 5060 setup.

For reference I am getting ~40 output tok/s on a 4090 (450W) with Qwen3 32B and a context window of 4096.

> Ultimately, as the user note aptly put it, the decision largely boils down to how much context you anticipate using regularly.

Hah. (emphasis mine)

That seems to be the gist of it. You cannot rely on serverless alone and you need one or many pre-warmed instances at all times. This distinction is rarely mentioned in serverless GPU spaces yet has been my experience in general.

> Time-to-First-Token of approximately 19 seconds for a gemma3:4b model (this includes startup time, model loading time, and running the inference)

This is my biggest pet-peeve with serverless GPU. 19 seconds is a horrible latency from the user’s perspective and that’s a best case scenario.

If this is the best one of the most experienced teams in the world can do, with a small 4B model, then it feels like serverless is really restricted to non-interactive use cases.

Minutes or hours at most depending on the model size and the training hardware.

Very rougly you can consider the Bs of a model as GBs of memory then it depends on the quantization level. Say for an 8B model:

- FP16: 2x 8GB = 16GB

- Q8: 1x 8GB

- Q4: 0.5x 8GB = 4GB

It doesn't 100% neatly map like this but this gives you a rough measure. In top of this you need some more memory depending on the context length and some other stuff.

Rationale for the calculation above: A model is basically a billions of variables with a floating number value. So the size of a model roughly maps to number of variables (weights) x word-precision of each variable (4, 8, 16bits..)

You don't have to quantize all layers to the same precision this is why sometimes you see fractional quantizations like 1.58bits.

Worth mentioning that a single H100 (80-96GB) is not enough to run R1. You're looking at 6-8 GPUs on the lower end, and factor in the setup and download time.

An alternative is to use serverless GPU or LLM providers which abstract some of this for you, albeit at a higher cost and slow starts when you first use your model for some time.

With a lot of motivation, grit, coffee and reinventing the wheel every couple of weeks.

Have you tried open weights vision models such as Qwen VL, MiniCPM, PaliGemma...?

I'm also curious how usable are simpler vision models such as Florence in case you explored this direction.

The article initially appears to suggest that all AI in science (or at least the author’s field) is hype. But their gripe seems to be specific to an architecture named PINN that seems to be overhyped, as they mention in the end how they end up using other DL models to successfully compute PDEs faster than traditional numerical methods.

Yeah it was right in front of me and I missed it. Thanks a bunch!

This looks cool and has the opportunity to replace my old and trusted todo.txt, but I couldn’t find how to create or resolve bugs. The CLI has features related to syncing but nothing about this, or did I miss something?

The section “Backed by leaders from Google, Microsoft, Meta..” is not clear. Do you have angel investors working at these companies, or what’s the relationship with your company?

Is it the same Sakana from the cheating AI coder tribulations? There were some fundamental mistakes in that work that made me question the team.

https://www.hackster.io/news/sakana-ai-claims-its-ai-cuda-en...

https://techcrunch.com/2025/02/21/sakana-walks-back-claims-t...

It's not just you. It feels like there's a missing sentence or paragraph somewhere connecting the two. Maybe there's a compounding effect making the impact of turning the satellite off worse?

For me it’s the double question marks that showed me how serious they are:

> Q: What is the founding principle of Mills GUT-CP??