Hacker News Reader

Comment by rvnx | Hacker News Reader

As the weights have leaked anyway;

one great move that FB could have done would be to ride the wave of positive PR + get all investors hyped:

---> "Meta is a credible alternative to OpenAI, the company is switching from "Meta"-bullshit to an "IA"-first company",

and get the investors to pump the Meta stock,

and then dilute some of the shares to raise some cash (or issue new shares to newly specialized IA hires).

But no, FB is still going after the VR gimmicks and NFTs.

4 billion USD per quarter wasted on Oculus (!), while they could use this money to fund and support a whole ecosystem around LLaMA.

View on HN

Is IA something different from AI? Or maybe just the French version ("intelligence artificielle" I imagine...)

My feeling is that a lot of Meta's AI/ML work actually ties into the AR/VR long-term dream. How do you make the so-called metaverse alive? By having people design it themselves. They're not going to do that in Maya, that's for sure. But if they could create virtual spaces and virtual people with Holodeck-style natural language instructions...

Could just be a small typo too

Maybe they mean "information architecture." Facebook does produce Docusaurus, after all, one of the best documentation platforms out there today :)

Intelligence Augmentation?

Latin languages use IA

I apologize, my French accent broke into the wild.

Yes, it's totally AI, and IA is the French version :)

To complete the joke, Information Architure (IA) = Architecture de l'Information (AI).

I've been known to mix both when switching context between Spanish and English

Programmers in particular tend to overuse anglicisms, and often I end up mentally doing spanglish in my head,

"Necesito este value" "Este command deberia hacer esto" "Este iterator va a hacer $something a este object"

Haha, I noticed my coworkers do that when they're speaking Hindi or Chinese too. All the key words are in English, so you step in and say "Oh, you guys are talking about that iterator bug?" and they're shocked: "Wow, you speak Chinese?"

Chinese, unlike Japanese or Korean, lacks a consistent phonetic mechanism for "importing" foreign words. Phonetic approximations are, with few exceptions, limited to foreign names. As such, practically every bit of jargon is semantically translated (examples include 电脑 "electronic brain" for computer, 端口 "end opening" for port), but since most programming is done in English-based languages, it's not uncommon for Chinese programmers to be more familiar with the English jargon.

I don't know how you'd better express 'port' than end-opening. After all it refers to the place where boats are loaded and unloaded. Honestly the semantic translation seems a far more apt term. I guess there's something like 港 but that's probably just more confusing.

> Honestly the semantic translation seems a far more apt term.

That's interesting. For me, the port metaphor makes me think data port, which seems like a pretty apt parallel to the flow of goods at a shipping port.

Different metaphors for different folks, I suppose.

Fair enough! It's interesting to think about the literal meaning of the keywords and industry jargon in English too.

Interestingly, many glosses using Chinese characters (not all, but many) were from Japan-- sometimes used phonetically (e.g. https://doi.org/10.1163/9789004438200_008), and sometimes used to translate concepts. Chinese imported many such loan words for technology from glosses that were made in Japan with kanji for words which had no existing equivalent in Chinese (https://www.jstor.org/stable/23754127). But then Japan increasingly gave up new words in favor of transliteration, while Chinese continued to have this need. Chinese words that are phonetic approximations of foreign words are perhaps less common, but by no means rare. They're not all from English (e.g., 'grapes' is a transliteration from Indo-Iranian), and sometimes you have to look at the Cantonese pronunciation to hear the resemblance to English, and sometimes they're neatly hidden by a choice of word that fits well phonetically _and_ semantically.

This has been true for Finnish also, more because of a stuffy language office that wanted to avoid unnatural influences.

Computer = tietokone = knowledge machine

Swap file = heittovaihtotiedosto = throw exchange record set (or some such).

So yeah, Finnish programmers rarely used the official words.

IMO it highlights just what a commodity they weights are. If all one needs is the weights to reproduce the work, then where is the value? I mean there is very little moat here. Further, what does it say about consciousness and individuality if we all are simply the values of the weights in our wet neural networks? Or whatever the biological equivalent is?

Nothing. It says nothing about consciousness and individuality. Numerical weights and physical neurons aren’t even remotely comparable.

Even if they were, any data is also only bytes - numbers - and therefore any knowledge, books, any kind of sound or picture would likewise be trivial. But isn‘t.

There's nothing "simply" here. The weights in question are a particular configuration of several gigabytes worth of data. They're not random. Getting anything comparable by randomly generating a number this long is a "total atoms in the universe to the power of total atoms in the universe" kind of a deal.

In abstract terms, those weights are by far the most dense form of meaning we've ever dealt with.

I think the point is more that the primary edge that they (Meta, OpenAI, etc.) have is compute capacity, since the models are easy to implement (and the most of the training data is out there). Once you have the weights, you can do with it as you please.

This is probably why they hate/fight these leaks. Once the model weights are available, they lose much of their competitive advantage.

If you look at their paper they especially encouraged reproduction. That's why they only used publicly available data.

could ai, just basically brute force better weights? say instead of doing training to get their own weights, maybe have 10 training inputs, test a new set of weights, have gpt4 rate the quality of the output from each set of weights and then try another and maybe infer a pattern to how weights could maybe potentially be created without all the work?

I feel like soon, ai will be it's own researcher and go far beyond what we ever could do on our own.

You just reinvented reinforcement learning

LLMs are very good imitations of what we expect from intelligent people. They are not intelligent themselves.

Not only do they have no street smarts whatsoever, but their book smarts start to disappear when you deviate from the training data.

Speak for yourself. What I've seen of all these language models so far is equal to a 10 year old with a good vocabulary and a tenuous grip on reality and no understanding of truth vs lie or right vs wrong.

I agree. GPT4 is impressive when asked to play chess, but the incapacity for understanding makes it really limiting

Unless you believe that you think with your soul or something, what else could you be other than your quantum state, or some close-enough compressed equivalent?

That’s like saying that creating a song or a movie has no value because after they are created anyone can download a file with it.

However it does open up a point which is: should we allow people to make huge amounts of money by infinitely copying and distributing their own work? Should we be protecting this model?

Uh the structure of the neural network matters just as much as the weights. Everyone has a different structure to their neurological arrangement.

They could do what Microsoft used to do in the 90s-00s, make pirated Windows/Office available (by turning blind eye on private users) so that they capture and keep the mindshare.

Facebook, which ran over the law in order to be successful now uses the law in the exact opposite sense. Ultimately, it's greed, "What is good for me to do to you is not good for you to do to me."

Actually I think nfts are wound down https://www.theverge.com/2023/3/13/23638572/instagram-nft-me...

Zuck really thinks he can build the Ready Player One reality I guess.

Apple, Steam, Epic Games, and VRChat will kill any chance Meta has at attaining VR monopoly.

Oculus is between a rock and a hard place. The rock being consoles that want their own native VR solutions and aren't interested in sharing their cut of games etc and the hard place being Steam.

You just can't compete with Steam. It's been tried. All competitors are mercilessly drowned in the giant fucking moat surrounding GabeN's castle.

PC gamers are fiercely loyal, "generally" more informed then other consumers and absolutely despise any forms of dark patterns. Because Steam has always done right by PC gamers Steam has benefited from that loyalty for a good 2 decades now.

Everyone is welcome in GabeN's castle but y'all gotta pay the toll to cross the drawbridge.

>Because Steam has always done right by PC gamers

It took a lawsuit to get refunds for Steam.

>absolutely despise any forms of dark patterns

Steam has been doing lootboxes and NFTs for more than 10 years

Sure, but thanks to bad behavior by the rest of the industry, Steam is still looked at very favorably by consumers in contrast.

These kids don't remember how much gamers hated Steam when Valve first crammed it down our throats

The new batch of kids don't seem to mind using Game Pass or picking up Fortnite of the epic games store, either

For people interested in this topic, the "People Make Games" channel on YouTube have a couple of related videos with original research/interviews:

* "How Valve is Profiting from Steam's Back-Door Casinos": <https://www.youtube.com/watch?v=eMmNy11Mn7g>

* "Working at Valve: 'A Fearless Adventure' or 'Lord of the Flies'?": <https://www.youtube.com/watch?v=s9aCwCKgkLo>

Oculus has thrashed SteamVR, because the hardware side of things fell apart and the Quest 2 is a lot more like a console. It was actually pretty successful in 2021, Zuckerberg is doing his best to kill it with random rebrands and missing why people buy it (stuff like Beat Saber instead of weird replacements for the real world), but Vive/Reverb/whatever is nowhere

Oculus is not even being sold in Germany, the largest EU market.

Someone should tell him (and all the other metaverse people) that VR is almost always dystopian. It's what everyone gets sucked into when civilization stagnates and there is no opportunity, no culture, and nowhere to go. It belongs in worlds of Malthusian collapse, after a nuclear war, or where inequality is so high the majority of people have reverted to high-tech medieval peasants.

VR has a legitimate niche in gaming but outside that it's just not appealing. It's dystopian and depressing. Nobody wants to spend time in a social network with a helmet on their head being served ads.

The obvious retort is that many people, especially Gen Z, do believe civilization has stagnated, and there is no opportunity, culture, or appealing destinations. I personally think they’re crazy, though I do appreciate them leaving the nice areas empty for me. :)

>where inequality is so high the majority of people have reverted to high-tech medieval peasants.

In what way does this NOT describe the situation in America?

That 4B # encompasses AI and Metaverse. Hard to tell where the $ is actually going (plus, metaverse requires AI advancement).

That’s one big reason the stock is up: $4B in the metaverse is nonsense. $4B in AI, though? Transformational

It seems like VR is less than half of the investment by RL. In Meta's 2022 annual report, they say "Many of our metaverse investments are directed toward long-term, cutting edge research and development for products that are not on the market today and may only be fully realized in the next decade. This includes exploring new technologies such as neural interfaces using electromyography, which lets people control their devices using neuromuscular signals, as well as innovations in artificial intelligence (AI) and hardware to help build next- generation interfaces. ... *in 2023, we expect to spend approximately 50% of our Reality Labs operating expenses on our augmented reality initiatives, approximately 40% on our virtual reality initiatives, and approximately 10% on social platforms and other initiatives.*"

I'm not sure if Horizon falls into "virtual reality" or "social platforms" but it seems to be the latter: "For example, we have launched Horizon Worlds, a social platform where people can interact with friends, ..."

there is a saying in Sanskrit that goes.. "the laziness of a Lion is the security for the rest of jungle".

Let 'em burn it all.

Exactly. Going after the folks that are basically helping improve your credibility is plain stupid.

This could have been their “contribution to the society” - yielding much better PR than they could have ever hoped for.

Reply on news.ycombinator.com