kennethwolters

First commit was 2 weeks ago. A clean-room rewrite of only the most essential APIs of litellm.

Was annoyed that DSPy was about 250 MB big. Mostly due to litellm being bloated and numpy. In order to create dspy-lite [1], I had to create a lighter litellm and make numpy optional (only essential for embeddings).

Made the lighter litellm litelm by selecting the parts of litellm's test suite that mattered and build it back to match litellms API.

Wrote a piece arguing that AI 1.0 is feudal because GPU scarcity concentrated power in a few cloud providers, but the coming GPU glut will invert this dynamic and make open weights + commodity compute the foundation of AI 2.0. The thesis is that sovereignty requires ownership of your stack (weights, compute, data) rather than renting intelligence from centralized APIs.

LLM-driven narrative game. Main technical issue is how go do compaction. I’ve devised a memory hierarchy that compacts the story to a constant amount of tokens per layer. Arc -> Scene -> Moment -> Line. Not sure if that’s the right dimensions to decompose into. Also tinkering how to get the right amount of “divergence” for story progression option generation. A lot of unanswered questions…

for me it feels like: Everything is stateful by default/convenience. Building robust systems is in part about confining statefulness to as few parts as possible. To contain statefulness. It’s to buy you some time and capacity. Yet the toughest problems often arise in the stateful parts of the system as well as quasi-stateless parts which sometimes develop hidden statefulness (think of syncing webclient and server state). So being good at handling stateful systems is valuable. Maybe one should even embrace statefulness. However, the AWS Solution Architect will tell you otherwise.

LLMs are pretty slow. Started a blog post series sharing what I've learned in search of faster LLMs.

Wrote an essay on what LLMs reaching their technological limits could mean to "AI Engineering".

not an attempt at rebutting or so, just adding my thoughts:

It doesn't matter if the product is great because of those values if there is a lack product (bc politics and bureaucracy make it infeasible to produce). Makes you think if those values might contribute to the bad politics and bureaucracy. And if so, how.

Why don't you think that calling the app "Llama-OCR" is good enough?

can someone tell me what the difference is to the webclient? Seems like a webview to me. No local caching/store.

Yeah that's a problem. I agree.

You are responsible for curating your "For You" tab by following/liking-posts-of/replying-to-posts-of/muting/blocking/unfollowing the right accounts.

Once I accepted that my "For You" tab got much better.

It is very good at "non-human subjects in photos with shallow focus".

Really curious to see what other low-hanging fruits people are finding.

It is very impressive :)

txtai is not (just) a vector store, it's a full-fledged RAG system. Apples and oranges if you ask me.

Don't have anything to add to the others. Just sharing a way of thinking for deciding between RAG and fine-tuning:

(A) RAG is for changing content

(B) fine-tuning is for changing behaviour

(C) see if few shot-learning or prompt engineering is enough before going to (A) or (B)

It's a bit simplistic but I found it helpful so far.

I am a big fan of local-first design principles. For most enthusiasts, it's the performance benefits. However, for me, it's the potential it has for privacy in web clients.

If you have a web client that only does on-client data storage, you're not dependent on centralized server-side storage for persistence (which is the main privacy-loss hazard).

The problem is that the app host still has all the technological freedom to not honor their privacy agreement, and there don't seem to be backstops to that behavior in browsers.

tuition-free for how long? current class? Forever?

fair point!

Thank you, I will have a look :) This YT channel is gem I found: https://www.youtube.com/@SailingTipsCa Doesn't post a lot, not very high production quality but very high quality information.

I am fascinated by off-shore sailing, competitive and otherwise.

How racers manage to do the vendee globe (solo, from france around-the-world back to france) in just 80 days with just sails and ropes as engines is beyond me.

FYI the fastest solo sail around the world was set in 2017 at 40 days in a Trimaran

Don't forget Kary Mullis as another famous scientist thinking HIV does not cause AIDS. I personally don't know what to make of Mullis' and Duesberg's claims. But I found an interesting Mullis article recently in which he presents a hypothesis about how AIDS could develop independently of HIV. Quite interesting, but I am guessing nothing really more than that. https://link.springer.com/article/10.1007/BF01435010 The article is unfortunately paywalled.

This actually looks great. More choices in terms of GitLab hosting pricing models.

I am looking into renting Hetzner GEX44 dedicated server to run a couple models on with Ollama. I haven't done the arithmetics yet but I wouldn't be surprised to see a 100x cost-decrease compared to OpenAI APIs (granted the models I'll run on the GEX44 machines will be less powerful)

IMO petroleum scattering around earth as CO2 damages earth less than petroleum scattering around earth as a petroleum-based solid. Both is bad tho - can't mix layers that are not meant to meet.

This is a Great project I am really looking forward to using for internal apps. IMO a central requirement for internal apps is auth, especially OIDC/SAML-type auth. How do you recommend implementing this in mesop?

Looking through the docs I am guessing a nice place for auth to hook into is the on_load parameter of the pages API? https://google.github.io/mesop/api/page/#mesop.features.page...

I anticipated this type of reply that states that for HOAs there is not enough freedom of movement and diversity of "HOA flavours" to reap the benefits of this localist/federalist system. And you are right. The situation seems dire and the question is how do we solve this problem?

Do we do so by smashing HOAs (and giving the responsibilities to someone else)? I believe this is the wrong approach because it will destroy any potential for this localist (and thus inherently good, imo) system to heal. You're basically passing the baton onto another entity that might do a better job off the start but will inevitably fail because it is even less capable of evolving.

I'd rather try to discourage rigidity/complacency and incentivise innovation in the HOA-space which is much more complex to implement but better than smashing HOAs.

Regarding your point about "not buying the whole argument": I think it would be foolish to organise a military or pandemic-prevention-force on a municipal level. The levels need to exist and some higher levels have responsibilities that the lower levels don't have. I absolutely believe that this is good.

In conclusion, somehow communities of a certain region need to be organised if they want to be organised, especially when the region is densely populated. I'd rather organise these communities through self-emergent associations than top-down mandated government. And yes, the self-emergent association-type HOA is broken. I'd rather fix it than replace it with a inherently worse alternative.

Switzerland is an "ultra-localist" country according to Nassim Nicholas Taleb and I'd agree. The Kantons have more power than the federal government. So in essence, Switzerland and the USA are much alike. It's just that the USA has a flavor of this localism so extreme and chaotic that the resident in a way is supposed to relocate to a better place if they don't like it (see residents fleeing California for better places). Switzerland seems to have a more orderly form of localism where instead of the resident, the representative moves.

The more a person is dependent on "associated or local governments" [1] relative to "national or supernational governments", the more freedom they enjoy.

This is somewhat counter-intuitive because one might think that if e.g. every resident of a nation had to obey a singular and simple set of national rules, that would promote freedom more than having governments and rules differ between regions.

I wont go deeply into the underlying mechanisms of why the more chaotic choice promotes freedom while the more orderly choice reduces freedom but it has something to do with organisations competing for members and members being willing/free to move between those organisations.

[1] HOAs, school districts, unions, chapters, guilds, bars, insurances

I feel like I am making better life choices when well-rested. Think about it. To be well-rested 95% of the time has a huge impact on a knowledge workers 25-year retro.

The US deindustrialised more intensely than Germany (think of rust belt), yet it seems energy production almost doubled since then [0]. I think energy production is more a function of the economy than of industrialisation.

And in the special case of Germany, the relatively small amount of industry that remained domestic kept Germany relevant globally. If Germany looses that, it becomes a fully service-based economy which the anglo-states are much better at (think finance, IT).

[0] https://www.eia.gov/energyexplained/us-energy-facts/