ppaattrriicckk

My read is that they're criticizing moving the reasoning (more) into the latent space, such that not even researchers/safety-evaluators can actually see what's going on.

Of course this still then has a 'trust me, bro' feel to it, but making it outright impossible to reasonably inspect the CoT is a significantly larger and riskier step, IMO.

> OpenAI switching to Sol/Terra/Luna renaming is just a way to rip off people and charge more money for the same sized model.

Excuse me, but what are you on about?

Unless I'm mistaken, they have literally(1) stated that it will cost $5 per 1M tokens in, and $30 for 1M output tokens. The same as GPT-5.5.

[1] https://openai.com/index/previewing-gpt-5-6-sol/

Apart from getting 16 sq. km of solar arrays and radiators into orbit - and without jumping to conclusions about whether this is a borderline scam - I can imagine 2 obvious showstoppers:

1) Space debris. This is proposal is several orders of magnitude larger than the biggest things in near-Earth orbits. Thus equally many orders more likely to be hit by, and create, space debris

2) Heat transport - this isn't my home turf, but I can't imagine building something lightweight enough to be launched, yet also capable of transferring enough heat away from the 5 GW core, without it melting/breaking

It's been a while since I read their whitepaper, but I don't recall either of those points being addressed.

... what tf?

On top of studying engineering at uni, his "side-gig" is being creative, empathetic, and fantastic at communication - and your prime recommendation is to "hire" him to be a specialist hidden away in the back office? Which interwebz forum are you on?

EDIT: My last question is clearly an echo-chamber statement. But that doesn't subtract from the fact that, yeah, should he found a business, yes, he'll deal with certain "BS". That is the weight we'll all carry. But he's quite likely capable of moving civilization forwards, so... :shrug:

That's quite the statement. What makes you so certain that we won't have AGI within 30-60 years?

Because "AI" is a well-established academic field, and the definition by your quoted comment is in line with this.

It's a very broad umbrella term, machine (and deep) learning are only subsets - although the main drivers these days.

Glad you found it useful.

A small caveat, though: The correlation is linear with the logarithm of compute. So here's hoping Moore's law & friends live on a tad longer!

And a somewhat unrelated fun fact: The authors surprisingly found the lowest correlation between compute and the performance in the domain of Go (and not the real world). Although the data is very sparse, I suspect that it's due to algorithmic advances.

It might not come as much of a shocker, but a couple of researchers last year claimed a strong correlation between raw compute and predictive power in various fields/domains, including weather forecasting: https://arxiv.org/pdf/2206.14007.pdf#page=10 (page 10 specifically has graphs on Weather Forecasting vs. Compute).

In their study they claimed a strong correlation in these fields (vs. compute):

* Weather Forecasting

* Protein Folding

* Oil Exploration (at BP)

* Chess

* Go

... The latter 2 being games, which I personally do not find surprising. But I do find it inspiring that we can "just" calculate our way out of some important issues. That hopefully translates well to other fields.

Small correction:

Net income: 0.524 bb

So 19.6% profit margin. Still nice.

Taking GP's point to less of an extreme: Are all humans always right in everything that they state?

Yeah, nah, I beg to differ: Just because the broader population might not (yet!) have an intuitive understanding of the field, and its shortcomings, I don't think we should rename decades of academic consensus. I have more faith in entire fields of research, at least.

I can't make up a good counterexample, so here's a so-so one: Fusion vs. fission power.

I've extremely rarely encountered the term "fission" outside of talks about fission power. Yet, I don't think we should call Fusion Power "Star-like energy generation" and Fission Power "never practically encountered power generation, but somewhat like star-like, except the opposite underlying mechanism"

I don't think anybody serious in the machine learning community has any delusion as to when a model says "A" with "99% confidence", that actually truly means that the model is fundamentally outputting "A" correctly 99% of the time.

Robustness and interpretation of machine- & deep learning models is a quite active field of research.

I don't know if I'm asking for too much, but maybe it's a question of educating people on these tools? I mean, even though most people aren't electricians, nor probably have a good intuition about how electricity works, we wouldn't touch a live wire

Sure, or you know... Rather than ranting on twitter, one could look up "Large Language Model" (LLM), "Deep Learning", "Machine Learning" as well as "Artificial Intelligence" on Wikipedia, and realize that historically the following has been the norm in terminology for decades:

Artificial Intelligence > Machine Learning > Deep Learning > LLMs

I.e. LLMs are indeed a subset of what is classically considered Artificial Intelligence. Just as well as route planning in Google Maps or getting beat by a non-machine-learning chess algorithm.

EDIT: Regarding "for decades" doesn't apply to LLMs, of course :-) and barely to deep learning. But for the time they've been around, this inequality holds

Except you don't have astronauts going to the vast majority of the 1000s of satellites orbiting the planet.

So apart from the 1-2 active space stations, where people go without having to do a full $100 million launch, I'd say laser transmission is a pretty good option for transferring data

I love bashing Intel as much as the other person, but looking at the number of tokens LLaMA[1] was trained on (1,400B tokens), the number of parameters of GPT-3[2] vs. the number of tokens it was trained on (175B params vs. 300B), it's not like the announced 1,000B param model is unreasonable. Taking 175/300*1400 yields 816B parameters - which is fairly close to 1,000B.

Not outright the most efficient utilization of data, but as others have mentioned, there might still be something left to gain (from not solely optimizing for compute). E.g. look at Figure 9 in [3]. Although the model trained on the largest model obviously utilized the data most efficiently, it's not perfectly clear - to me at least - whether some over-parameterization will necessarily lead to a decrease of test/out-of-sample performance.

Of course, LLaMA was trained on way fewer parameters relative to data set size. I've just mentioned LLaMA as a point of reference to the largest dataset known to me.

[1] - https://arxiv.org/pdf/2302.13971.pdf

[2] - https://arxiv.org/pdf/2005.14165.pdf

[3] - https://arxiv.org/pdf/2001.08361.pdf

EDIT: Formatting of references

I have created a small, open-source framework for generating 1000s of variations of images based on sampling prompts.

The purpose of the project is 3-fold: 1. To efficiently test out ideas for images 2. Subsequently to easily hone in on the best prompts for your given goal 3. Reproducibility and keeping track of experiments (and

To give an example, I wanted to create images of "a dog wearing sunglasses", but I didn't really have any particular styles nor compositions in mind.

So then I've set up a list of possible scenes like ["walking on a tropical beach", "floating in space", "black background", "grey background"] as well as a list of styles ["vector graphics", "art deco"].

Now I'm randomly generating prompts combining the above. And tomorrow I can scroll through 10k pictures of very cool dogs :-)

If anybody is interested, maybe this should be considered along with the core inflation rates - which is high in the US (peaked at 7% within the last 12 months and currently at 5.5%) and possibly still rising in the EU (12-month peak [now] at 5.7%).

If the central banks are going to stick to their declared purpose of keeping this around 2% we're still in for further rate hikes, I would imagine.

* https://tradingeconomics.com/united-states/core-inflation-ra...

* https://tradingeconomics.com/euro-area/core-inflation-rate

I know it's not within the desired behaviour on HN to simply write "thank you!", and I hope I did provide a bit more than just that. That being said: Thank you! Also to ed_balls and madcaptenor for collecting data in the other thread.

In case I'm not the only one a tad confused about this (awkward) sentence:

> JWST looked at TRAPPIST-1 in mid-infrared wavelengths of light — 20 times redder than the human eye can see

What's referred to here, in terms of wavelengths, is the range 4.9 to 27.9 μm according to https://jwst-docs.stsci.edu/jwst-mid-infrared-instrument. The longest human eyes can see is around 0.75 μm, so I guess that's where the notion of "20 times redder" comes from

I know this doesn't add much, but a small addition: The whole concept of the lost Dwemer civilization is an entire layer of depth to the fantastic world. A world which IMO is unparalleled

You capture what I failed to articulate. I feel exactly that way about Oblivion and Skyrim

I remember having one of those free websites together with a mate from school (webbyen.dk, was the thing in Denmark). We'd write our own review of games, half-conscious that nobody would ever read our reviews. But for Morrowind, I was so... crazed out from pure anticipation, that I wrote a review before having the game in my hands.

The game more than delivered on my expectations. The music still soothes me to a large degree, and the world (glass and daedric armour or houses made from giant mushrooms) was so absolutely fantastic to me.

I've always wanted to experience the exact same thing, but with present (or future) graphics. I've tried installing the various mods and rebuilds, but it never was that great an experience.

Recently I bought Elder Scrolls Online (ESO) while it was on sale, and while I think the game itself is quite boring and constantly nudging you to subscribe, I was blown away by exploring Vvardenfell. In my opinion it really captures the same mood and it's... Just. So. Beautiful!

Not quite RDR2 graphics, but it really hit a soft spot for me, seeing Seyda Neen "remastered". I can highly recommend spending the dollars/euros, simply for that dose of nostalgia :-) or simply an image search to see some of the good old scenes with modern day graphics

Yeah, you're right. That was a typo and now I can't correct it.

I'm not sure what "high room temperature" would even entail :)

I had no idea, thanks for sharing.

Again, I'm very much a layman to this subject, but how does miniaturization necessarily affect that particular aspect ratio? Since it's literally a ratio of two dimensions of the torus, shouldn't this be invariant to the overall size? (Assuming all things being equal, which I have no idea whether holds.)

In extension to this news and the press conference, one thing I am super excited about, is the private SPARC project and the MIT-spinoff Commonwealth Fusion Systems (CFS). If you don't know about it already, I would highly recommend checking it out (e.g. by searching YouTube for "MIT Sparc Fusion Reactor" for some fairly accessible videos on the theory behind why they should achieve fusion way faster than the current roadmap with ITER and DEMO).

In the press conference just ended, they repeated how exactly the JET reactor worked as predicted by theory. In my layman's understanding, for the exact same reason (seemingly very sound theoretical groundwork), the SPARC reactor should exceed breakeven within the next few years.

From Wiki on CFS:

* Back in September 2021, they built the strongest high-room-temperature superconducting magnet (20 Tesla) suitable for a fusion reactor

* Theory dictates that with stronger magnets, the reactor can be scaled down (with the square/cube, can't remember exactly), and thus cost and time to develop

* Back in November 2021, they raised $1.8 billion from the likes of Bill Gates

https://en.wikipedia.org/wiki/Commonwealth_Fusion_Systems

Boy, do I think it would be crazy cool if they succeed, even taking twice as long as they've planned! :)

Exactly that - apologies for not elaborating myself. At least I couldn't have put it better myself. Thanks for being good sports about it, both :-)

That was the joke

That's a very fair point. Hard to do anything about right now, since they unfortunately didn't include those.

However, while very hand-wavy on my part, I believe there's plenty of evidence that Covid on its own wreaks havoc on our cardiovascular system, so I'm definitely leaning towards it being the biggest sole risk here.

This probably sounds harsh, but are all vaccine-bashing readers deliberately selectively-illiterate? In the abstract (academic for "TL;DR") it literally states that Covid is between 7 and 40 times more likely to cause Myocarditis.

And a lot of you write about palpitations, which can also be caused by stress and anxiety. I'm 30, in good physical shape, and used to have weekly episodes with irregular heartbeats due to severe stress. Now I'm vaccinated (which is for 99.999% of us obviously completely irrelevant to experiencing this, but a lot of commenters draw that conclusion) and the palpitations are gone. Let's not base conclusion on anecdotal evidence, please. For the same reason that my grandpa lived until 85 while smoking since he was 13.

According to the Danish Companies House they haven't been very Danish since 2015. They've moved ownership 100% to Singapore (and from there, who knows).

Still 300 FTEs in Denmark, though.

https://datacvr.virk.dk/data/visenhed?enhedstype=virksomhed&... (the original company with David Helgason etc)

https://datacvr.virk.dk/data/visenhed?enhedstype=virksomhed&... (IP company)

https://datacvr.virk.dk/data/visenhed?enhedstype=person&id=4... (reference to the Singaporean company)

I second this.

Try to have a look at the statistics of Taiwan, South Korea, Germany, Norway, Thailand, Denmark: https://www.worldometers.info/coronavirus/

Seems that some countries manage to _consistently_ keep the spread at a comfortable (for a lack of better word) level.