eggbrain

Right now you are right -- even if I ran my local LLM all day, the quality is not nearly as great, and it runs slowly -- so I use the tier one AI subscription services as they are faster and smarter. But that might only be true for a limited amount of time, and a limited number of circumstances.

To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.

Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.

So right now you are right, but who knows how long that will last.

Performance optimizations don't just help the western labs, they also help with running more powerful/useful LLMs locally.

If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.

I can imagine the attention economy sinking, but I struggle to believe that we as a species won't replace it with something potentially worse.

E.g. before the internet there was TV, and advertisers have filled up that space for decades -- before that magazines and newspapers did the same. At this point, can humans believe that they are not being farmed for their eyes?

That being said, I do think the internet potentially becomes basically plumbing -- something that we don't really touch (as our AI agents or whatever comes next are the ones that absorb the content there).

This is the biggest opportunity in open source I've seen in my lifetime. I bet you in three years we'll have more and better open source libraries than ever before.

For example, a few months ago I was looking to build a license detection tool, and within a few weeks with the help of Claude built a tool that was 20x better than the best open source one I found (I also open sourced mine as well: https://github.com/licensedetector/cli).

The biggest problem will be navigating through the glut of libraries that have been written in my mind -- 100 tools for the same use case, all with < 100 stars will make it hard to decide between. Also, if the user didn't write them thinking about anything other than their own use cases when releasing their library, the standard response will probably be for the agent or programmer to recreate the wheel yet again, only for their specific use case, making it now 101 tools for the same use case.

In the paper (section 5.1), they actually tried to abstract high level directional insights into the prompt in order to see if that helped, and they basically found it underperformed a prompt that didn't have those insights at all, implying that directional guidance perhaps over-constrains things.

Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing.

Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlimited steps of course, because you have a finite amount of compute.

So each agent goes off, and by the end, Agent 1 got to 90% accuracy, Agent 2 got to 80% accuracy, and Agent 3 got to 89% accuracy. Agent 1 wins, of course.

But then you look at the refinement steps, and after 2 steps, Agent 1 was _already at_ 90% accuracy. So the agent spent the next 8 steps basically not moving at all. Agent 3 on the other hand, perhaps was continuously climbing in accuracy at every refinement step, but hit step 10 and had to stop.

Now because you recorded every step from every agent, you know what you'd do differently next time -- you'd not allocate as many steps to Agent 1, and give Agent 3 more steps, because perhaps that might result in Agent 3 coming up with a better answer.

From my understanding, that's what they built in the form of a "search" controller -- a way to evaluate automatically and reapply how you could allocate resources more effectively, when applied to a new problem.

But I guess my misunderstanding is how applicable the search controller is when applied to new problems -- just because one pathway stalled early for one problem, doesn't mean it would work for another?

I've stopped waiting for sites to do what I want in terms of features and functionality -- their incentives and mine very rarely align.

I think the solution here eventually will be a search engine that lives on your personal computer (it's also what I'm building currently). Ingest content from the web -> sort/filter/view it locally as you choose.

Note -- if you do this, watch out for requesting access to "all tlds". They send you two emails per TLD -- one for your pending state, and one for your approved/rejected state. I suddenly had 1k+ emails flooding into my inbox, until I found the setting on their website to disable emails.

This is actually where I see software going in the short term -- cloud moving to local.

A few years ago, if you wanted translation, you'd use Google Translate. If you wanted to search the web, you'd use Google search.

But for a few gigabytes, you can now install nllb-200-distilled-600M, and get translations for almost any language locally. You can have your computer crawl the web, create abstracts and categorizations for websites, and build search exactly as you want it.

The main limiter now is hard drive space (and to an extent, local compute) -- but right now it feels like the 70s again where the terminal into a remote server turned into building applications locally.

You may need to select a smaller model, but an M4 with 16GB ram should work!

DHH was not the point of my comment (in fact, I said if the author had focused on him they'd have more of a point), so this unfortunately feels needlessly attackative.

This post does not feel like it was made in good faith. The general premise ends up being:

- Matz is nice -> DHH / Tobias are "not nice" -> Ruby is doomed (because of two not nice people, also perhaps because of AI)

I think if they had focused perhaps on DHH (DHH's writings have been very polarizing), and also dropped the AI bit at the end, they might have had a stronger point, but Tobias ends up being judged guilty by the author almost purely through association:

> [...] Lütke doesn’t say anything nearly as incendiary as DHH, but the actions of the company he controls tell their own story, and his posts on X, the Everything App, document his turn to the right, along with all the crypto and AI stuff you’d expect to go along with it.

The author ends with a hope that a fork of rails will happen, quoting a user on mastodon:

> [...] we're building a community to execute a bigotry-free #Rails fork [...]

Here's my issue: let's imagine the author gets exactly what they want. Rails is forked, the fork becomes more used than rails, the community is saved!

The problem is that this forked code now becomes two things: a product of software (what it was before), and a product of ideology.

When a piece of software now also represents an ideology, people that want to contribute (but are not coders) end up contributing to the ideology instead. E.g. in this case, they'll try to help define what constitutes a bigot, find potential bigots in the community to root them out, things of that nature. When the "easy" bigots are found and removed, the work does not stop -- the definition of what is a bigot changes, and new bigots are found.

To be clear, the above happens regardless of ideology -- it's just human nature (See: Levari et al., Science, 2018). For example, many people on Reddit who claimed it had gotten too left-wing/right wing tried to create their own social media platform, only to realize that targeting an ideology as their main differentiator ended up eventually only attracting the people who are so deep in that ideology that it pushed away anyone casual.

About the same thing I saw. I kept having to increase my bid every day, and even at $4 CPC basically saw nothing converting.

As you said, it's also really hard to get visibility: metrics are poor, and targeting seems more like a whim/suggestion than anything else (their label for context hints basically tells you they only _might_ take your guidance on who they will show your ad to).

Audiences consists of hashed emails or phone numbers -- no targeting demographics, job titles, etc.

Overall it's just not there yet -- I paused my ads today.

> Can you imagine a venue refusing entry to former clients 2% of the time just because they’ve “improved their experience”?

On the flipside, if a client enters enough venues that refuse entry to them because of something the client can fix on their end, eventually the client will probably change themselves -- "If you meet one asshole during your day" and all that.

To bring the analogy back to browsers, if a website works fine for a client, they'll have no pressure to change anything on their end -- why upgrade from Windows XP when the site looks fine in IE6? Eventually the client is forced to upgrade -- normally by their operating system. That works, but what if the operating system adds another 2 years to their end of life -- do you just hang on and hope the shim / hacks you added hold?

Gauntlet AI I believe is correlated originally with Lambda School (YC S17). YC founders I believe are able to post job postings on Hacker News, although this might stretch the definition a bit...

I feel we need a "proof of work by human" for emails. Something that could be signed that attests that someone took the time to write the email, not just sent a template / used AI to auto-generate a personal looking email, etc. Sure that could be gamed as well (have an AI write characters one by one to look more human-like), but taking more time usually is a fairly good blocker for spammers / salespersons / etc.

For those of us on subscription plans:

* From today through June 22, Fable 5 is included on Pro, Max, Team, and seat-based Enterprise plans at no extra cost.

* On June 23, we’ll remove Fable 5 from those plans. Using it after that will require usage credits. If capacity allows, we’ll extend the included window.

* After this point—when sufficient capacity allows us to do so—we aim to restore Fable 5 as a standard part of subscription plans. We intend to do this as quickly as we can.

The "offer, then remove" aspect is a bit eyebrow-raising -- it feels like they are trying to get subscribers to switch to usage-based billing, which makes me wonder if we'll ever get it after that June 22nd window.

> [...] People do not realise how much of a toll it takes on you if you actually care about the environment, exploited workers, theft from the people who can least afford it, the impact on people's cognitive skills, the centralisation of power, the spread of disinformation, the ruination of the web and/or the destruction of entire career paths (not billionaire of course, that's always a safe one), and not endorsing (either distinctly or tacitly by using) AI.

I believe people do understand the toll caring about something deeply takes -- but caring about all these things at once, many which you personally can't control, feels more like atlas syndrome or compassion fatigue by the author.

I also find the author a bit all-or-nothing in general. Losing friends because they use AI? Why does the dichotomy have to be so black and white? Can people have moral quandaries about AI while still using it, or does the moral stance always have to be absolute?

> [...] With organized criminals, you can't actually see what the abuse is 'worth' to them.

Even without collecting events, you can calculate what the abuse is worth to you, even if the math ends up being fuzzier.

At the small platform operator level (one guy running a platform, as this article), the cost can be as simple as "this pisses me off and I have weekends." They can burn forty hours bolting on JA4 fingerprinting and a disposable-email blocklist to stop an abuser whose dollar-EV to them was roughly zero. Looks irrational, and that's exactly the deterrent — abuse pricing assumes a rational counterpart, and a guy who'll overspend his own life-hours out of stubbornness is unpriceable.

At any scale larger than a small operator, you also do get real numbers -- you can't perfectly price reputation, but you can price traffic and ad conversions, operational costs, LTV of customers (and conversion funnel metrics) etc, all of which don't stay still while abuse increases.

> [...] That's why it's worth collecting events before acting: what the account is about, which IP network they use, whether they fake devices, whether there's any warmup prior to registration. Because that's what helps estimate whether your mitigation will actually work, and lets you respond in a balanced manner instead of under- or over-reacting.

Isn't this just a way to estimate exactly how much the 'abuse' is worth to the abusers?

To the end platform, what's the difference? Mitigation techniques largely remain the same, in that you make it more time / energy / money than what the end result of their abuse is worth. The platform cares about stopping the abuse -- not neccesarily correctly identifying whether the people abusing their platform are small shop "bot farms" vs organized crime.

There will always be a subset of users whose goal is to not use your service, but to arbitrage your service into the maximum value for themselves.

For example -- let's say you offer $100 in free AWS credits by signing up to your platform. Expect a malicious user to eventually come to your platform, realize they can resell those $100 in credits for $50, and start using your platform for their own gain. Unless the mechanisms you add in place to reduce fraud / second sign ups / etc is greater than the value that they are receiving ($50), they will continue.

With sites where the platform is free, the math almost always makes sense for these malicious users to eventually abuse. In this case it was leveraging the email reputation of another domain at no cost to their own (along with the added value of anyone getting phished), but on other sites it's public profiles being used for backlinks / spam, etc.

The value of human interaction cannot be overstated -- and the writer did a beautiful job outlining how AI isolates us. But there are also hidden difficulties in human interaction that AI helps ameliorate.

- My doctor friend does not wanting me pinging them asking for free medical advice every time I get health anxiety

- My chef friend does not want me calling them every time I'm struggling with a recipe

- My author friend does not want to read the 20th draft of my book, in which I've changed perhaps 10% of the content from the last draft

In these, the cost is a tax on the relationship -- relying on someone else too much to the point where it could potentially be impacting _their_ life.

Similarly, there are enough communities out there that are not accommodating -- even if I wanted to get a human answer and/or connect with someone, the interactions themselves can be painful. Do we remember what it was like posting on Stack Overflow? Do we believe Stack Overflow was a one-off outlier?

I also believe human imagination and knowledge shouldn't be bound to the relationships you have around you. What if my social group is small, or diversity of knowledge that my social group has is small? Should I not be able to think and explore an idea because my best alternative would be to contact a professor at a university that 99% of the time will not answer me?

I do believe that many people use AI now instead of learning and connecting -- I know my own programmatic knowledge has weakened now that AI has acted as a superhuman autocorrect. But on the other hand, with the help of AI I've also learned about a ton of things that would have otherwise been unavailable to me -- and I believe has improved me on the whole.

Normally, I separate the download filename (what the server / person chose to call the file) from my own organization system file name.

So if I download or get sent "Book.pdf", I'll rename it to how I'll remember it -- "Book Title - Author.pdf", etc.

That being said, I don't think there's any right answer here, it's usually just a matter of time and energy. If I had to enrich every single file I download with a great title / detailed metadata / etc that I'd need to find that file later, that's all I'd do all day.

> Stripe (their payment process) will handle adult content payments. It puts the account into the high risk category due to the high rate of fraud in those categories.

Stripe _says_ they will handle these type of payments, but more often than not, within roughly a year of implementation you'll get an email from them kicking you off their platform, no matter how vigilant you were, or even if the things you were selling were more rated R than rated X. Source: my own insider knowledge along with colleagues in the space.

Lately I've been building Aho (https://aho.com) -- an API for verifying age, credentials, and identity using cryptographic proof from digital wallets instead of document inspection.

For context, I built out Playboy's age verification system, and watched as it hurt conversion (nobody wants to upload an ID to an adult website, who would have thought!). Cryptographic signatures from issuing authorities (DMVs, universities, employers, etc) with selective disclosure (e.g. you don't need to upload your full ID, just the fields that matter) is how verification _has_ to work going forward -- AI can fake documents, but not private keys.

I've been working on this 6 months full time, and implemented all the W3C VC, OpenID4VCI/VP, SD-JWT specifications myself.

Would love to get people's thoughts on it!

There are two opposite answers here, and I feel like I could argue either one:

1) Humans were never held accountable, really

Outside of a few regulated industries, the worst that happens to an engineer who pushes negligent code is that they get fired. But after that happens, what actually changes? The organizational structure of the company that allowed the employee to push bad code still exists.

2) Humans will still be held accountable

If a human (managing a fleet of AI agents, let's say) ends up deploying bad code to production, they won't be able to point to the AI agent and say "it was them that did it!" -- it will still be the human at the end of the line that is held responsible.

Your comment seems to imply AI is currently at a junior developer's level -- 12 months ago I would have agreed (like I mentioned in my parent comment, both near the end and about the "latter" team I was a part of), but it's gotten quite good over the past few months.

When even Linus Torvalds compliments AI code (ref: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fa...) I think we can say he wouldn't have said that about any junior engineer.

That's not to say it won't ship bugs, but so does any engineer (junior or senior). It's up to you as to what level of tooling you surround the AI with (automated testing / linting / etc), but at the very least it doesn't also hurt to have that set up anyways (automated tests have helped prevent senior devs from shipping bad code too).

I find people over-rotate on whether we should be reviewing AI-produced code. "What if bad code gets into production!" some programmers gasp, as if they themselves have never pushed bad code, or had coworkers do the same.

I've worked at places where I've trusted everyone on my team to the extent that most PRs got only a quick glance before getting a "LGTM". On the flipside, I've also worked on teams where every person was a different kind of liability with the code that they pushed, and for those teams I implemented every linting / pre-commit / testing tool possible that all needed to pass inspection (including human review) before any code arrived on production.

A year ago, AI was like that latter team I mentioned -- something I had to check, double check, and correct until I was happy with what it produced. Over the past 6 months, it's gotten closer (but still fairly far away) from the former team I mentioned -- I have to correct it about 10% of the time, whereas for most things it gets it right.

The fact that AI produces a much _larger_ volume of code than the average engineer is perhaps slightly concerning, but I don't see it much differently than code at large companies. Does every Facebook engineer review every junior engineer's pull request to make sure bad code doesn't slip in?

That isn't to say I'm for letting AI go wild with code -- but I think if at worse we consider AI to be a junior engineer we need to reign in with static analysis tools / linters / testers etc, we will probably be able to mitigate a lot of the downside.

Many of today's news websites (tech or otherwise) cashed in their goodwill / reputation / page rank to sell ads.

The first shoe dropped when news websites realized they weren't generating content fast enough. Hard, in depth journalism takes time, but when people want to know something that happened _today_, they don't want to wait a week for all the facts to come out, and so the major websites started losing traffic to websites that churned out articles fast.

The additional benefit of churning out articles was that you could match against more and more long tail keywords, which lead to more traffic and more ability to sell ads. To keep up, many websites dropped quality for speed, and consumers noticed.

The second shoe then to drop was with affiliate marketing -- articles on CNET / Wirecutter etc were already ranking and rating products, so they figured "[...] why shouldn't we get a cut if someone ends up buying a product we recommend"? The challenge then became that consumers couldn't tell the difference between a product that was recommended because it was good, or because the product gave the biggest "kickback" to the website for using the affiliate link. Thus, people that gave "honest" opinions on products (e.g. people asking on Reddit, at least for a while, as the article suggests) became the new source of truth.

The result of this means that these days, if you read a lot of articles on the major tech websites, they feel more like they've been optimized for speed (e.g. churning out an article fast), SEO, and not much else. Many people have talked about how recipie websites are now short story generators more than food instructions, but it's been common for a while where I go to a tech website to read about something I specifically Googled, only for it to feel more like it was written _specifically_ to capture traffic for a keyword, rather than actually solve the issue or question I came into the website with.

The cherry on top is that AI has none of these problems (so far) -- yes, there's some movement on trying to do SEO for AI, and of course ads will eventually come to AI like it has everything else, but currently, you can get the answers you want, described to you exactly how you'd like to hear it -- who wouldn't want that?

I'm curious -- are there any stories of projects that launched on Hacker News, Hacker News loved it, and it ended up _also_ being a big success?

E.g. we have stories like Dropbox where HN seemed to be dismissive only to be proven wrong, and there are numerous launches where HN was dismissive and they were proven right, but I'd be more curious when the HN crowd got it right in a positive way.