Hacker News Reader

jamienkon Ember-1

This is so cool - I'm aware of this in a vague way. Can you write a little tutorial or give some good links. I want this to be the next new things I do :)

jamienkon Ember-1

Diff people have diff motives to experiment, then new work is done on top of stuff that "hits" in a way no one anticipated. Then work gets piled on top in a way that might make it hard to port

jamienkon Ember-1

Ignoring for the moment issues of what "counts" as open, won't open models rapidly advance due to stuff like this in ways that it's less possible for the proprietary ones to do? This is exactly how Linux & Wikipedia, for example, overtook their "frontiers", right?

jamienkon Understanding the Impact of LLM Watermarking on AI Agent Behavior

I think it is related to AI watermarking, though, in that all forms of trying to channel words in certain directions, or to add a meta-layer to language, share a family resemblance.

There might be an inevitability to this - there is always a form PC policing, or euphemism, or re-brands, or rhetoric. But there is always the danger that what is considered a well-meaning tweak to language masks or evolves into repression.

From TFA "Text watermarking is meant to help identify whether content was generated by AI, but inside an agent it also becomes part of the generation process that produces decisions."

jamienkon Understanding the Impact of LLM Watermarking on AI Agent Behavior

With Political Correctness, you analyze a word, reveal connotations that expose an unwanted origin or association, try to come up with a new word that does not have those connections, and then try to enforce the usage of the new word. All with the idea that the force of the old associations will no longer have a hold (manners and mores, which I agree with you about, don't have this goal). But the old forces and powers are perfectly capable of attaching themselves to the new words.

The struggle in repression is real, even if it's unconscious. The psychoanalyst asks to speak with FREE ASSOCIATION and the inability to do this in SPEECH is what reveals unconscious problems. Repression, though, is a STRATEGY that we most definitely sign-on to. It FEELS unjust to be asked to take responsibility for something we didn't INTEND to do, but that, unfortunately, is our task in life.

jamienkon Understanding the Impact of LLM Watermarking on AI Agent Behavior

In _1984_ the Big Brother regime has the idea that by controlling language you can influence what is possible to think, and thus becomes a key tool of political repression.

Political Correctness has a similar idea that by adjusting the terminology we use, we can purge biases and historical implications and speak in a purer way.

Psychoanalysis has its own idea of repression - where a person struggles to BLOCK our associations between ideas, memories, and words in order to try to stop one thought from being contaminated by another, intolerable thought.

All of these attempts to control language are fundamentally misguided at best, often have severe unintended consequences, and are genuinely immoral at worse.

jamienkon ArXiv receives multiyear commitments to support it as an independent nonprofit

Did we completely write off the "web of trust" as a workable system? Where you exchange keys with people and co-sign each others keys and then trust keys signed by them. This would be an alt to "algorithms" and also a way to "subscribe" to things that got vetted by trusted people or orgs. I kind of do this with uBlock Origin, where they curate a (very complex) list of ads (but without the key signing). Orgs could vet and publish lists of arXiv papers and I'd be subscribed to that. This could also be the source of my social media feed (friends of friends, etc).

I remember in the early days of PGP this was put forward as a vision. I remember the various key-holding systems (that were hard to use). I did a key-party with my brother and his friend in the '90s.

Why did this fail?

jamienkon ArXiv receives multiyear commitments to support it as an independent nonprofit

I think we underestimate the psychology of this kind of use of LLMs. I don't think these people (students, academics, lawyers, etc etc etc) are all just lazy morons. I think that using an LLM gives a strong knee-jerk feeling of "OMG this is exactly what I have to say. This is MY idea, these are my thoughts, this is - in a real sense - MY writing!" The feeling floods you when you see the output being churned out, way before you you read the thing (if you ever do) - it's the initial *seeing* it. Then when you hand the generated text in, unread by you, you do not have the feeling of cheating, you have the feeling of having exercised your powers, pushed right to the edge of your expertise and thoughtfulness, you successfully overcame obstacles because of your experience and unique capabilities.

This is not always the psychological situation, but I think it might be a lot of the time.

I also think that until we recognize this, we won't be able to help people to not do it. Calling them lazy or being bewildered by them or feeling rage or contempt towards them or threatening them is not going to be practically helpful.

jamienkon Xiaomi MiMo v2.6

I’m still not at all sure about the “billions” invested claim. How much of that is cloud running the models? How much is pre and post training (which may or may not be part of what we’d want to include in accounting). Etc. Does anyone have links to good reporting about this: not blind recitations of numbers, but analysis and thought mixes with investigation?

jamienkon Nobody pays for FOSS, we can force them to

This is very interesting: it is a lot like the GLP (pun intended) solution: it doesn't try to "cure" the symptom, it just piggybacks onto the existing facts-on-the-ground and tries to make things better from there.

I'm not totally sure I understand the "distribution/registries" thing though? This also related to GitHub and HF, right? And Netflix. (You mentioned Spotify.) How they host stuff and then leverage that position to build add-ons and lock-in, lobby for laws. Companies PAY for this "bandwidth."

I always thought that BitTorrent would eventually take off and make these kinds of sites irrelevant, that it would democratize bandwidth, the last leg of the battle for universal accessibility.

I like your idea. I wish you luck.

jamienkon ChatGPT now knows what you do on other websites via ad collector

But politics is when people who feel differently try to do something about it

jamienkon ZK-JPEG: Zero-Knowledge Image Editing and Compression

Or what about: the photographer tries to hide so that they don't contaminate the real-life that they are watching, but then the person sort of hears a small sound from the photographer's direction -- they don't actually SEE the photographer, but they get interrupted.

Or: the CCTV is well hidden, but the people who installed it left behind some mud that got on someone's shoes and that made them late...

jamienkon ZK-JPEG: Zero-Knowledge Image Editing and Compression

This is so good. I think this is what Hegel meant by the "absolute" - not some final God's-eye view that he's often caricatured for, but the fact that every objective "substance" is and will always be wrought by "subject", by a POV, a mediation, a representation.

jamienkon ChatGPT now knows what you do on other websites via ad collector

We. Are. FUCKED.

THIS is where the regulation needs to start.

jamienkon Pirate Face Rescues LLM Models from Deletion

Let's imagine that a model is pulled from HF by order of the new overlords or because of some other kind of censorship. Wouldn't the question of it having a Free license or not potentially become a complex legal issue?

But if the point is to be "censorship-free" then why respect licenses at all? They are among main choke points today. If authoritarians use licenses to censor political, artistic, scientific, etc., speech that they want to block, does that make the censorship more respectable?

When Anthropic sues a Chinese lab for IP infringement and get a court to put a bar on that software, does it THEN get pulled from Pirate Face?

I know that an awful lot of international negotiations have become focused more and more on questions of "IP" - licensing battles are already intensely politicized and it's hard to imagine a future where it doesn't get much much worse. Imagine N Korea coming after you for violating a license that they worked hard to control and leverage.

jamienkon Pirate Face Rescues LLM Models from Deletion

Can you explain this a bit to a non-expert?

I haven't wrapped my mind around this

jamienkon Microsoft director: AI scraping 'the largest theft of labor in human history'

When we sing, draw, speak, play, we are not attacking the soul of someone else!

Culture has always had a kind of violence to it, but not FROM the THIEF: you HAVE to speak the language that was already made; you have to understand and participate in their song-structures, their plot devices, their ways of attesting.

The "IP owners" should be paying ME!!!

jamienkon ZK-JPEG: Zero-Knowledge Image Editing and Compression

Hasn't this crossed over into being a philosophical question? You want to attest to reality or provenance. But then you start making lists of "acceptable" changes: file format; compression; color-correction; size; taking "medium" into account... It then slowly slides into "average human perceptibility"; "irrelevant details"; keeping the "spirit" of the image intact; judging the intention or motive of the user or of the viewer; aligning the image with "values"; appropriate/legal use... etc. Not sure what the end-game is?

We are trying to make images in the AI era be as reliable as they were pre-AI? But they were not reliable pre-AI, it was just more difficult to intentionally doctor them.

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

"can't be used as evidence" << the whole field is only a few yrs old and is based on academic maths from decades ago.

A lot of progress is in harnesses and distillation and weird research that requires creativity. The "Frontier" always tries to frame things in terms of "the frontier" in order to limit how we look at things.

We don't know the true costs of pre- and post- training, running data-centers, hiring people, making business deals, etc., so even the (maybe) quantifiable question of cost vs capabilities is not at all clear. The efficiency of businesses ultimately gets measured by their ability to maximize a commercial return; but this is NOT a good metric for other important things, including maybe "how good is this"? (Where "good" is open to debate but maybe not quantifiable.)

It had been true that the "best" operating systems required a lot of business money. And encyclopedias. And ... (list some other good historical examples here!) ... Are Linux and Wikipedia the "efficient frontier"? In a sense they are both infinite performance at zero cost. Hard cash outlays sometimes stop being the determinant for high-effort endeavors.

I remember in the 1990s Encyclopedia Britannica's CDs and MSN (Microsoft Network(tm)) among many others argued that the web would never be able to match the curated, bespoke, expensive products they offered. It was IMPOSSIBLE for that crap to be matched!

It is a POLITICAL question.

jamienkon Tin: full-text search for Postgres

"...are both considered Open Source" << Free Software (GPL) is the O.G. - it's like saying "George Washington is considered to be an American President" or "The Beatles are considered to be a pop/rock band" or "water is considered to be..."

I remember so well when "Open Source" branding started with Bruce Perens - all the business arguments. When you need a database, someone ELSE'S business decisions (how THEY are going to make money) never end up helping YOU. If you use proprietary extensions and become dependent on them, you inevitably will get BURNED when their business needs diverge from your needs. - They close shop - They refuse to interop with something you need - They demand that you obey their arcane rules - They rug-pull - They get hacked as only they can - They lie to you - They stab you in the back

jamienkon I built non-autoregressive decision models with RL a year ago

Why do we ("society") need the "frontier" companies at all? Their business goal has settled on trying to CONFUSE the shit out of us so that we don't understand the big pictures about various aspects of AI.

THANK YOU, Nandakishor Mukkunnoth, for putting in the work to help to clarify this stuff!

You are like a firefighter compared to their fire-insurance racket.

jamienkon Hister: A private search engine for the pages you visit and the files you keep

Would like: * Local web page interface or even browser UI element (since extension needed anyway) * Ability to add notes to history * Flag if bookmarked, allow filtering "bookmarks only" * Keep old versions of pages * Human-readable text diff vs current live page

jamienkon Hister: A private search engine for the pages you visit and the files you keep

https://bugzilla.mozilla.org/show_bug.cgi?id=342916

Can I add NOTES about pages? This might be a good spot to do that...? Maybe the interface can be in a web page instead of terminal?

Before Google took off there was a vibrant ecosystem of FOSS dev around search, all different little aspects of it. Then after Google people stopped fiddling with search, search became "solved" or maybe "must be coded by the big boys". Shame.

Thank you for this, looooong time coming

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

1) I don't think "frontier model" is a term of art, it is a marketing term. I have an M1 MacBook Pro. What do you have? Always the "frontier model" laptop? Do you always order the most expensive "frontier menu item" at the bar and grill?

2) I do not believe the "billions" number is OpenAI/Anthropic's training costs. I suspect it includes business expenses (including the big $$ to the guy who came up with "frontier model") and infrastructure, etc. That includes the data-center costs for running the cloud. And the "R&D" expenses which includes who-knows-what. And the settlement payment for data access. Etc. Why are the cost breakdowns not available to the public? Not because of thoughtfulness, altruism, care for the human race, but because of business plans.

3) "Piggybacking off OpenAI/Anthropic" - Scraping the web is "piggybacking" too, and so is buying existing data or even paying for new data. The "L" in LLM stands for "language" which is our common heritage.

But what does this have to do with anything anyway? People argue that truly Free (FOSS) LLMs couldn't be be developed because of costs, but I do not think that that is obvious. This used to be the argument against Linux and Wikipedia.

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

I think it is not just liability, it is dawning on us that we can run powerful models locally and this is the start of a Cloud-backlash.

The AI gods have tried to make it seem like they are not just running their programs in the cloud, but that the AI is so "powerful" that it requires a super computer to run.

But my 5 year old Mac is running pretty good AI, shocking fast, surprisingly powerful! The "natural" course of dev (Moore's law, etc) makes it pretty clear where this is going...

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

https://hugovergnes.github.io/little-lm-3-8b/ << less than $1k for a 4B model

Remember how DeepSeek v.whatever cost ~$5m

Stable Diffusion 1.5 was reportedly $70k in compute.

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

Is this "compute is a moat" true though? Why are the economics so obfuscated? Where can we see how much money it costs to TRAIN various models.

https://hugovergnes.github.io/little-lm-3-8b/

The training and cloud costs are often conflated. R&D costs too (which are hard to compare to open systems).

A lot of compute is clearly "wasted" where they are not focusing on optimizations, etc.

jamienkon David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

There ARE other options: network effects and lock-in. These tried and true techniques (mastered by FB and M$) have not yet been taken up by the AIers. I do not think they will rely solely on the push for regulations, so they will soon be doing things like slurping our address books, having "teams" settings, and any other crap they can think of.

jamienkon More questions about whether researchers can trust OpenAI with unpublished math

ChatGPT accesses my IP address and geo-locates me. Claude code now asks if it can have my browser cookies. Next they will take my address book. They might scan my whole computer. Etc etc. These are pretty low-tech, normal techniques.

We can't trust any of their denials. FB denied everything year after year.

AI regulation needs to start here. Forced interop, forced source code licensing, harsh penalties for privacy violations or conspiracy to access private data. Block lobbying. Etc. These are the kinds of old-fashioned solutions we need for this kind of old-fashioned evil!

jamienkon More questions about whether researchers can trust OpenAI with unpublished math

I think OpenAI and Anthropic are slowly feeling the pressure to GET SOME $$ or a plan for some $$ — they need to somehow generate some NETWORK EFFECTS and LOCK-IN. Without that there's no stability: selling ad hoc one-offs is much much too quaint! This is dawning on them like it dawned on Google when they stopped not being evil. Need... to... "MONETIZE"...!

Model: FB. FB scraped other websites on a massive scale, then spent big on legal lobbying to block others from scraping. FB slurped our address books and spied on our friends. FB bought other companies and mixed the databases. FB made an art & science out of generating "sticky engagement" (they literally acted like trying to addict kids was a worthy "academic" goal, suitable for "serious" investigation thet they consider legitimate "science"). They mastered the cookie and have researched web fingerprinting techniques running 24/7/365.25. Recall that FB recently backdoor-installed a webserver onto every iPhone they could in order to circumvent tracker-blocking.

We aren't just disclosing by chatting. The AI companies now run binaries on all of our computers. They are 1000% non-transparent about everything. They make up new econ-jargon (like "run-rate") to make it seem like they are disclosing. They are constantly doing complex international lobbying and mucking in international relations. They have powerful propaganda/spin centers generating stories, ,manipulative warnings, and misleading info.

This is NOT a comment on AI tech. I like AI, and I support the right of people (programmers) to scrape the open web.

But in short: these are good, old-fashioned tech companies that we have seen over and over ... and over. They are positioned to be the next M$, the next FB (IBM, AOL, lol). Did you follow the latest Steve Balmer news? Do you read Pro Publica?

I get on my knees and PRAY...