I can archive my data just fine by using an access based storage tiering like S3 - have your cake and eat it too. Gets around the headache of actually querying all of the 1B+ records because realistically there’s very few use cases for that. If you ever really need it, spin up Kafka and run through the entire archive.
> The entire field of behavioral psychology might be a scam
I studied cognition and came to the very same, sad conclusion.
There’s a reason there was a reproduction crisis and it’s far from over. I remember vividly how one of my professors, during a paper debate, would end criticism with: “but it’s been published” as if that handwavy gesture could explain away serious research gaps that even graduate students saw through right away.
Shoddy statistics in a system that rewards quantity can do a lot of damage.
Not really. The Pareto frontier is itself a distribution over all optimisation paths and dominance is a real and useful property we test when researching evolutionary methods. It’s not chosen to be insufferable but to try to be precise.
> Messages that students did send were mostly bare answers
This has been my observation in university as well. LLMs can be valuable in digesting proofs into smaller, workable pieces (e.g researching the origin of a particular linear algebra property).
This however requires solid fundamentals and the ability to dissect proofs.
In spite of this people with neither usually reach straight to the solution out of sheer desperation (myself included).
> There are many interesting open questions. The first is whether, and how, Dust can find better directions than backprop’s first-order gradient
Both algorithms are bound by the same Pareto frontier based on the Empirical Risk Minimisation Principle, so they’re already on the same trajectory.
Interestingly backprop is limited by conditioning of the Hessian matrix in order to converge (differentiate correctly). So removing this limitation is actually a great step.
I’m excited to see a comeback of evolutionary methods because they’re much more general, albeit costly and naive.
We’re now very close to what can be described best as brute forcing the Pareto frontier out of our datasets. Not sure that’s what we want but I have no better ideas either.
That’s not true. I can ask ChatGPT for a verbatim copy of a licensed book. It’s been documented numerous times. I fail to see how, in the real world, a verbatim slightly changed version is different than a copy. But they’re selling ChatGPT, so effectively they’re selling access to libgen. So long as they can’t guarantee your unlicensed copyright won’t be reproduced verbatim, it is essentially reselling copyrighted material.
Your assumption is wrong. Unlike artificial neurons, real neurons process massively and concurrently, let’s say 10-20 thousand inputs at once in less than 1 millisecond.
In Kahnemans theory system 2 always involves the anterior cingulate cortex and arises when there are multiple conflicting streams of information. That’s the defining characteristic. Both system 1 and system two process in the same way, including looping back to previous areas. So really, no, stopping an LLM early vs letting it run has nothing to do with Kahnemans theory.
In general LLMs are so far removed from what we consider natural cognition that it’s very hard to apply neuroscience concepts to LLMs simply because they share no real world similarity. At best, you’re simulating something in a very inefficient way.
In terms of making an LLM faster but not in terms of meta-cognition. System 1 thinking as defined by Kahneman doesn’t have 100000x more compute than System 2, it is actually the opposite. That completely contradicts your claim
A fine is a slap on and wrist. Imagine you individually copied and sold libgen. Not only that, you’d say you did it for profit, you knew it was illegal, and conspired with others to cover it up. I don’t understand why no criminal charges are filed? Why isn’t anyone going to prison for a crime committed?
Honestly I don’t give a damn about copyright. I’m baffled by the double standards. Aaron Swartz was put away for so, so so much less than the massive amount of criminal activity we’re seeing here. They’re conspiring to break the law, openly, even jokingly. They’re hacking people, they’re stealing from people, openly. If no one is going to put a stop to it we might as well declare the legal system failed in its entirety.
If this doesn’t put anyone in prison then we might as well declare copyright dead. They knew they broke the law and then they deliberately covered it up. How much more evidence is required here?
> Brains. After you reach adulthood, neurons don’t divide. If some of your neurons die—which happens every day—then they’re gone. If you get a brain injury, the other neurons will try to “learn around” the injury, but the neurons themselves are never replaced.
This is a very outdated view. Neuroscience has shown that neurogenesis continues throughout the entire life span of humans, albeit at a much slower rate. You might have a hard time recovering from a serious brain injury or stroke but losing a few brain cells won’t change a thing.
> but somehow the Trump administration has convinced itself the public concern was manufactured in China.
Wait a minute here. Critical journalism could dissect how opinions are formed. It’s far from speculative that the White House opinion about AI opposition is informed by AI companies. They’ve been warning about China for a long time. So there’s really no somehow. Vested interests with massive amounts of money are lobbying and succeeding to convince the government that any opposition to AI is outside influence. It’s not somehow, it’s deliberate. Do better.
We’ve established how many empty houses there are and how much further demand there will be, and guesstimated that with the current empty apartments, we can cover almost 6 years of the cities growth.
That is, if you do absolutely nothing but to force people to rent their homes.
Now in reality, we’ll do more than that. Both the government and public sector are and will continue to build houses. But those are just projections. The existing houses, well, exist. So let’s make sure we use them.
Whenever this argument comes up people get extremely defensive for no good reason whatever. It will make a significant impact to the housing market if we force stalled housing to become available. Sure, we can and must do more than just that, but why not start with the cheapest and easiest thing we can today? Because it’s someone’s private property and we can’t even demand they rent it out in a housing crisis????
Come on now. Stop the madness.
Thanks for actually proving my point! 2% of 2.07 million residential buildings is 54 thousand empty buildings. Let’s say an average number of 2 people per building. Hey, we could house 100 thousand people today. I think that would solve the annual 17 thousand growth for the coming years. And you’ve done nothing, spent nearly no money, no planning involved. Just a single law to force people to rent, and everyone will have housing.
Hmm, crazy isn’t it?
We’ll wait a minute here! Why not call it LLM-archive.today???
Obviously we’re not trying to hurt someone’s copyright!!! We’re making a derivative work from it!!! An LLM! So Europe can be cool too!!
Oh but you’re saying downloading the entire internet in the process violates someone’s copyright??
Well now, don’t be dramatic, what if we fall behind China?? Surely a little copyright can be violated for that.
/rant that I have in my head whenever someone complains to the government about copyright. Either charge LLM providers for copyright violations, or be done with it, and declare copyright dead. Can’t have it both ways, or sooner or later people will figure out a loophole to it anyways.
But it does. There’s immense amount of houses that are unused because they’re part of speculation, and it’s easier to leave them empty than to deal with tenants and their rights. This is an extremely well documented, systemic issue in large European metropolis and especially in Berlin.
Forcing existing property to be used immediately adds supply, and it also lowers prices. Exactly the things large property developers do not want. But I don’t give a shit what they want. I want affordable rent.
> Cryopreservation
+
Demonstrate the ability to cryopreserve and recover live wild-type mice with high viability.
Specifically, demonstrate the reversible cryopreservation of live, intact, wild-type adult mice in a whole-body frozen or vitrified state. The mice must remain frozen or vitrified for at least 24 hours, must be recovered with >99% viability, and must not suffer any permanent organ damage or bodily harm. Somatic genetic engineering is discouraged but permitted. All experiments must be conducted with ethics approval.
This has been done in the 50ies, freezing and thawing mice with microwaves. IIRC the recovery rate was about 74% with no observed side effects.
The research was given up on because in this specific case a mouse is a bad biological model. Thawing agents must scale cubically with size. The author says that at approximately the size of a cat you can’t quickly enough evaporate all the agent because the energy required will simply burn the tissue.
TLDR: it’s possible, it’s been done, it could be perfected if necessary, it does NOT scale beyond mice
Imagine after the suicide scandal OpenAI would have had to pull up with hundreds of thousands of GPUs in front of the Holy See to betroth ChatGPT as a deodand to the church. Fucking hilarious
Interestingly enough SQLites FTS supports Lucene queries out of the box with great performance characteristics. IIRC only writes become pretty slow after a while.
I’ve always wondered what exactly would prevent PostgreSQL from strapping that implementation into its own database. My experience with ts_query hasn’t been particularly rosy. It can be better than LIKE but only marginally so and at the cost of insane index sizes…
If this extension becomes open source and we can test it out in the real world I’m sure there’s a sweet spot
I worked in realtime trading. No. Not at all outrageous. Quite reasonable actually. If that’s what we agreed and I need you to be reliable I will charge you back for being unreliable. I’m happy to pay top and extra dollar for the SLA but that means it needs to be acted on.
Then again if you’re a business relying on GitHub enterprise you have an SLA and you can and WILL charge GitHub for failing their SLA. Usually there’s a real measurable dollar value tied to that SLA per dollar and it’s not cheap.
What surprises me in particular is that the global API and the GitHub EE API are the same which is a big no no. This is even more surprising given the fact that paying GitHub customers are clearly the minority both in numbers and code velocity.
My assumption would be that GitHub is keeping EE up and the rest of free or pro users just have to suck it up. If that’s not even the case then it’s only a matter of very short time until GitHub will see businesses leave to more reliable competitors
> The EU needs to be truly Federal to succeed economically. That might create uniform regulations with more uniform regulations and actually make Europe competitive.
But also:
> The European Union has a nominal GDP of roughly $20 trillion, placing it closely on par with or slightly ahead of China ($19 trillion), but smaller than the United States ($30–$31 trillion)
I don't know about you but the same GDP as China sounds economically successful... or not?
> Without alignment, further improvements in capability turn LLMs into wanton felony generators
Honestly, I don’t think that’s bad at all. I hope OpenAI and Antrophic keep RL training runs up that randomly fuck with a lot of people.
Until the day the DOJ comes knocking, locks those idiots up in jail and closes them both down for the insane lack of responsibility and carelessness they’ve shown.
Sounds like the IDEAL outcome. Finally some jail time for all the fraud, negligence, outright scamming, hype inflation etc. if anything can accelerate this, oi, be my guest. Amodei might be afraid because he knows if he keeps pulling the stunts for investment theatre, at some point they’ll actually face consequences. AWESOME. That’s what we want right there
Honestly, mount a front and back dash camera, and allow people to submit tickets for speeding and tailgating themselves.
I’m not kidding excessive speed and not keeping a safe distance equates for some 50% of all deadly traffic incidents.
It’s because a small and vocal minority drives like absolute fucking assholes and they will rally lawmakers not to introduce speed limits and to loosen grip on enforcement. The ADAC (German motor association) is a perfect example. When they announced after decades that they support a 130km max speed limit members jumped by the doves.
The only vastly meaningful improvement I can see in FSD is faster reaction times when it comes to assholes. Coincidentally those are the people who insist on driving petrol and will never upgrade to FSD because they like to drive like assholes.
IMHO it’s time for strict enforcement. Let’s ban them from the road finally. Set fines in percent from income. Allow self reporting via dash cams. Add stricter penalties for recurring offences, if you’re caught speeding 5 times in 2 days your licence is suspended.
That’ll have a much bigger impact than FSD. Maybe we can rally our representatives making them all giddy with all the extra cash they can earn from these offenders?