OpenAI – transformer debugger release
github.com
github.com
Because me and everyone else here really hates Elon Musk.
> You say it raised legitimate questions about their non-profit status and lack of non-profit related activity.
As soon as OpenAI accepted that investment from Microsoft it has only been used as a for-profit proxy to have their own ‘DeepMind’ to bring down Google which they have tried for years with Bing and the Scroogled campaigns.
OpenAI is now no better than Google DeepMind and both are as closed and secretive and OpenAI ditched from their charter and non-profit status and continues to make excuses for not giving back their research.
Speak for yourself.
I get Elon is not popular right now. But jeez this is playground emotional level logic.
If I understand correctly, the OpenAI board asked Musk to remove himself and sell his stake as his for-profit pursuits conflicted with the mission of open, non-profit research. But after he left, they started a for-profit subsidiary and did exactly what they didn't want him doing. I could see how a judge might side with him.
They are also very upset with Elon's very public shift away from the current Left.
It's Elon derangement syndrome.
These legitimate questions about their non-profit status ARE his core arguments. His complaints are the same as the popular notion on hackernews that OpenAI has failed to be open.
Hence his argument boils down to "You made me sell my shares because my 'closed' AI research conflicted with your 'open' non-profit ideals. However, you have since stoped to adhere to your mission statement, effectively seizing to publish your research findings and pursuing for-profit goals. As a consequence of this I have lost a bunch of money."
And as nuts as Musk is, I kind of see his point here.
Imagine being the part-owner of a bread focused bakery. You tell the other owners that you think the business should focus on pastries – even if everybody agrees that bread products are more important – as pastries make more money (that money can in turn be invested into the bread business).
The other owners hard disagree and ask you to leave the company, because a) that does not fit into their bread-centric mission and b) you own a pastry-supplier in town and would obviously be profiting from this move. So you sell your shares and move on, no harm no foul.
But what if they then turn around and start producing pastries, claiming it's only doing so to financially support their bread business? What if they start entering lucrative deals with other pastry-suppliers and effectively stop making bread?
In this case I would argue that you would be entitled to some portion of your foregone gains. You sold your stock while being under the impression that the company would never go into the direction which you deem is the *only viable one*, only to find out that they did exactly what you suggested after you left.
> ... because it isn't working on tech I agree with, ...
Please notice that this is not what happened. They all agreed on the tech to be researched. The only disagreement was the business model and company structure required to fund the research.
Even your bread/pastry scenario doesn’t quite make this part clear. And this is what will be difficult to prove - did OpenAI know and discuss a rug-pull of “we willingly plan to say we’re going to be doing non-profit work, but actually our plan is to be a for-profit company”.
If their pivot was pre-meditated then I could see Musk having a case, but if they pivoted purely from market factors and realizing they wouldn’t be able to cut it as a non-profit, I’d think he’s SOL
But please reread the comment which is the root of this discussion: > The (imo) shakier part of the argument is that he is entitled to damages even though he doesn't own shares in the company.
I was never arguing that Musk is entitled to damages. I am merely arguing that it is possible to not own the shares of a company and be entitled to damages. Whether Musk specifically is owed damages is something a judge has to decide.
However, please remember your comment which is the root of this discussion: > The (imo) shakier part of the argument is that he is entitled to damages even though he doesn't own shares in the company.
I was never arguing that Musk is entitled to damages. I am merely arguing that it is possible to not own the shares of a company and be entitled to damages.
Except for the portion where he advocated for a capped-profit arm which he would run under Tesla. You know that part that is shown in the OpenAI emails. It seemed he was ok with it when HE would stand to benefit.
You can't believe that Elon is "a great businessman" while also believing that he is totally altruistic and ignores all of the self serving impulses that have gotten him to where he is.
But I don't see how your dichotomy makes any sense. There are selfish poor people. There are altruistic poor people. There are selfish wealthy people. And there are altruistic wealthy people (disregarding blanket denials by far left wackos with emotionally derived selective criticisms of capitalism).
Eye-rolling conspiracy theories aside, Bill Gates is absolute proof that someone can be an objectively great businessman and objectively highly altruistic.
Seems to me that the impusles which got Musk to where he is an obsession with extreme risk taking, combined with sufficient education and clarity of vision to sometimes bend extreme odds in his favour. History is littered with examples of wealth incinerated in the pursuit of building cars and rockets and it says a lot that he's been associated with a multitude of ultra-unlikely successes.
Whereas many other billionaires are more directly driven by wealth accumulation, as evidenced by the trappings of extreme wealth which surround them.
All of this goes without saying.
> Eye-rolling conspiracy theories aside, Bill Gates is absolute proof that someone can be an objectively great businessman and objectively highly altruistic.
The biggest and most glaring difference here is that Bill Gates is not on record attempting to take board control of a for profit company which is attempting to make malaria vaccines (of the many public health efforts he is pushing).
> Seems to me that the impusles which got Musk to where he is an obsession with extreme risk taking, combined with sufficient education and clarity of vision to sometimes bend extreme odds in his favour. History is littered with examples of wealth incinerated in the pursuit of building cars and rockets and it says a lot that he's been associated with a multitude of ultra-unlikely successes.
> Whereas many other billionaires are more directly driven by wealth accumulation, as evidenced by the trappings of extreme wealth which surround them.
None of this has anything to do with the merits of the case and really just comes of a Musk-coddling, a favorite pastime here it seems.
If that happened, I'd be surprised if Bill Gates didn't sue them.
(As for my latter paragraphs, I was replying to something you said which didn't have anything to do with the merits of the case either. I'd like to think you could extend me the same courtesy you extended yourself.)
"BUT THEY ARE DEMOCRAITIZING IT"
Its good to see that I was always right, even last year. I just didn't have a dad who owned an emerald mine.
Was there ever an emerald mine? Maybe. Though nobody has been able to find a scrap of evidence for it. Literally the only "evidence" is Errol's own boasts, which is that he bought a half-share of a Zambian mine for 100k ZAR ($50k USD). Hardly the stuff generational empires are made of.
> [Errol and a Zambian busienessman] agreed on a price [for Errol's plane], and instead of taking a payment in cash, Errol was given a portion of the emeralds produced at three small mines that the entrepreneur owned in Zambia. [...]
> Errol, who never had an ownership stake in the mine, expanded his trade by importing raw emeralds and having them cut in Johannesburg. "Many people came to me with stolen parcels," he says. "On trips overseas I would sell emeralds to jewelers. It was a cloak-and-dagger thing, because none of it was legal." After producing profits of roughly $210,000, his emerald business collapsed in the 1980s when the Russians created an artificial emerald in the lab. He lost all of his emerald earnings.
$210,000 in 1986 would be about $600,000 today.
[1]:https://www.rrauction.com/auctions/lot-detail/34638440643009...
Wonder if it would be possible to somehow trace the origin of that emerald to where it came from. Anyway its probably the Musk Family office that was the winning bidder so we may never find out now.
I don't know why this guy keeps having emeralds come up in his life. This auction is hilarious given that he repeatedly denies having anything to do with Emeralds. In fact the origin of the Emerald story is Elon himself, in a Forbes article that has since been scrubbed from the internet.
The best I can find is this Snopes article that references the Forbes article [2]:https://www.snopes.com/news/2022/11/17/elon-musk-emerald-min...
Here is the quote:
JC: How do you handle fear?
EM: Company death – not succeeding with the company – causes me a lot more stress than physical danger. But I've been in physical danger before. The funny thing is I've not actually been that nervous. In South Africa, my father had a private plane we'd fly in incredibly dangerous weather and barely make it back. This is going to sound slightly crazy, but my father also had a share in an Emerald mine in Zambia. I was 15 and really wanted to go with him but didn't realize how dangerous it was. I couldn't find my passport so I ended up grabbing my brother's – which turned out to be six months overdue! So we had this planeload of contraband and an overdue passport from another person. There were AK-47s all over the place and I'm thinking, "Man, this could really go bad."
Meanwhile, there's still no trace of this supposed mine. You'd think with all of the obsession people have with this mythical origin story, someone would have found it by now.
If some other entity (Government agency? Another company?) Comes around to pull this string, its going to set a bad precedent.
What is "non-profit related activity"? IANAL but the only justification needed for non-profit designation is not being for-profit.
And maybe their structure with a for-profit subsidiary is up for debate because of this. But you don't get to be non-profit by doing "good stuff" like open sourcing things. You get it by investing all profits back into the business.
Evil Corp for Doing Evil could be a non-profit if it invested all its profits back into the business.
Good thing you are NAL, because this is flat out wrong. "Non-profit" is a term used to refer to specific tax-exempt institutions under US tax law 501(c), and has nothing to do with investing profits "back into the business" (otherwise, you could have called Amazon or Uber "non-profits").
OpenAI[1] is specifically a 501(c)(3)[2] which explicitly requires the organization's purpose be one of the following "exempt" purposes:
>charitable, religious, educational, scientific, literary, testing for public safety, fostering national or international amateur sports competition, and preventing cruelty to children or animals. [3]
If it does not continue to fill one of those purposes, it can lose its tax-exempt nonprofit status.
[1] https://projects.propublica.org/nonprofits/organizations/810...
[2] https://www.irs.gov/charities-non-profits/charitable-organiz...
[3] https://www.irs.gov/charities-non-profits/charitable-organiz...
I'm offended. Presumably if I was AL I would have gotten it right.
Maybe scaling transformers is the best way forward. I'm hopeful. But it's a big assumption that it will produce AGI.
"Better than the average human at most profitable tasks" is a much lower bar than most people on HN might think.
I have vendors who instead of filling out a web form which remembers their inputs and eventually even fills everything out for them instead print it out and fax it back in.
We're probably only about 2-3 years away from transformers being self-optimizing enough in prompts and evaluations to outpace the average worker in most tasks in most roles. (It won't necessarily be that much cheaper after the multiple passes and context windows required, and crucially probably won't be better at all tasks in most roles.)
If you define AGI as "better than any human at profitable tasks" or "better than average at all tasks" then yes, we're a long ways off and transformers alone probably won't get us there.
This is not the definition of AGI. You can't just make up a random definition to fit your argument, lol.
https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
And I particularly like this one, which is much more clearly measurable. If you feel AGI is taken, maybe we should coin this one as APGI or something like that
On a timeframe for achieving AGI: https://youtu.be/cEg8cOx7UZk?t=1658
The poster here created his own definition, but what is wrong with that? He set a very specific bar to achieve, something that most "high-level" thinkers in the space have not really done. Isn't the point of discourse to bring your ideas to the table?
https://www.businessinsider.com/sam-altman-thinks-agi-replac...
I think the HN crowd forgets that what really runs the world are min wage workers running around and doing real world things, not code monkeys and glorified type writers filling excel sheets. So yes, replacing the bullshit jobs we invented to keep people busy will be relatively easy, that's if you don't account for the fact you'll now have to create bullshit+ jobs to keep them busy
And even then we're far away, sure it can shit out code for a todo webapp and create semi realistic images of a monkey eating a burrito but that's about it. More than a year ago someone betted against me here that chatgpt would revolutionise the world in the next year, nothing happened really, geeks are excited, execs are buying the hype, tons of money is transferred, yet there was no 4th industrial revolution.
What happened though is that the web is flooded with absolutely useless content, amazon is full of ai generated books, students rely more and more on chatgpt to generate homeworks, thesis, "find" solutions, &c. it might very well end up being a net negative for the average joe in the long run
This is not true at all. How many products do you use that come primarily from minimum wage workers?
If a few people responsible for Google maps running stopped working the GDP loss would be much bigger than if magnitudes more minimum wage workers did the same.
I doubt agricultural wages or truck drivers are minimum wage jobs where you live. Assuming US https://www.nass.usda.gov/Charts_and_Maps/Farm_Labor/fl_allw...
20% of them live in poverty, half of them make less than 22k euros a year
Truck drivers earn between min wage and 150% of min wage, while being on the road every day and not having a social life, they drive 8 hours per day and sleep in their fucking truck while some code monkey makes 300k+/year coding memeojis at apple. Guess which ones will be automated first by openai lmao
Where are you getting this information? It's absolutely wrong. Long haul truckers (the one's you're saying don't have social lives because they drive 8 hours per day) make $71,196 on average in the US[1].
[1] https://www.ziprecruiter.com/Salaries/LONG-HAUL-Truck-Driver...
Just imagine what would happen to a trucker's salary in the US if it were to create a unified market with Mexico and all of Central America.
https://en.wikipedia.org/wiki/2004_enlargement_of_the_Europe...
It's not necessarily a bad thing. Economies of Eastern European countries have been growing after all and Western Europe does not have enough workers because of its demographics anyway. My take is, that everybody is winning, there is less poverty than before, but some sideffects look ugly for a while.
I'd be extremely happy making this amount. Some people are just accustomed to easier lives.
>while some code monkey makes 300k+/year coding memeojis at apple.
Meme position for a privileged caste statistically irrespective of skill in an institution that can piss money on anything and still succeed.
Besides the farm work, there is food processing workers (cutting, cleaning, basically assembly lines), packaging, workers at warehouses, people who work at the counters of the store, and all the support roles for those positions. If you go outside and eat, you have all the restaurant personnel to take into account as well.
There is a lot of low skilled labor that goes into making what we eat today. I'm not sure how you could possibly claim that none of those people are on minimum wage.
Minimum wage in Spain is significantly more money than anything I've made in my life. It's a very comfortable position for the vast majority of the world.
>There is a lot of low skilled labor that goes into making what we eat today. I'm not sure how you could possibly claim that none of those people are on minimum wage.
People doing essential work that isn't trivially replaceable have the bargaining power to charge more than the minimum wage in a moderately free market of human work, usually they do.
Did I miss the part where the other comment mentioned retail, or where you respond to the half dozen other examples of essential work?
> Minimum wage in Spain is significantly more money than anything I've made in my life. It's a very comfortable position for the vast majority of the world.
Instead of moving the bar some more, could you just define what minimum wage would be an acceptable bar for you in this conversation?
https://uk.indeed.com/career/warehouse-worker/salaries
Do you really need me to Google every single essential position known before conceding that society is not maintained by minimum wage workers?
It was a good thing.
If trucks stopped deliveries every city would die in a week
If construction workers stopped building / maintaining we'd be in deep shit in 6 months or less
If the people in warehouses stopped working for a week the economy would tank like it rarely does
Nurses, doctors, bus/tram/train drivers, police, firefighters, ambulances, janitors, trash pickers, plumbers, sewer workers, electricians, people taking care of water treatment plants, power plants, teachers, social workers, ...
You could delete facebook, openai, instagram, twitter, netflix, tesla and 90% of startups from the face of the earth right now and I'd have the exact same life as yesterday. Remove any of the people I mentioned above and society would crumble in no time
And none of these are even remotely close to being automated at all, nobody cares about most of these jobs. But hey, here is a dancing kangaroo: https://www.youtube.com/watch?v=Zuivg5rz_aA
https://money.cnn.com/2016/02/24/news/economy/trash-workers-...
https://money.usnews.com/careers/best-jobs/garbage-collector...
>You could delete facebook, openai, instagram, twitter, netflix, tesla and 90% of startups from the face of the earth right now and I'd have the exact same life as yesterday. Remove any of the people I mentioned above and society would crumble in no time
Yes because you picked non-essential work. (?)
Now that there are great inventions like TikTok, teenagers are depressed as hell, and they don't go to meet each other to play soccer together, because the "social" networks are giving the illusion of having that connection.
Then again tell me who we're automating out of the work force right now ? Trash pickers or code monkeys ? Truck drivers or artists ?
Now my trash is serviced by one person who mostly never leaves the cab and who would be better described as a skilled machine operator than as a menial labor role. The work isn't completely automated but technology has reduced the job market for trash truck crews by two-thirds. I'm guessing the barrier is higher now too, requiring training and certifications to run the robotics on the truck instead of physical fitness being the primary prior qualification.
Yet. The industrial revolution didn't happen in a year either.
Sarcastically asking me "Can you meet girls on that?" and then laughed.
He wasn't wrong in the short term but laughably wrong in the long term.
Is this really true? It's certainly a nice soundbyte when you're making class arguments or trying to dunk on the "HN crowd", but I think it falls apart under any level of scrutiny.
Who keeps your lights on? Who drives your bus? Who manages your sewage? Who teaches your kids? Who builds your roads? None of them make minimum wage and would probably be a little insulted to be characterized as such.
It's pretty reductionist to call anyone outside our realm of tech a "min wage worker", they're just workers like or I. I think it's a pretty stupid and pointless exercise to subdivide people into useful or non-useful workers, serving no purpose but to further pet the smugness of HN AI skeptics.
This sort of thing usually takes longer than you expect it to, and then it usually happens faster than you expect it to.
Somewhere along the way we built computer that are so intuitive people find printing and faxing easier than our web apps. This isn't completely the fault of any single web app, users have a lot of learned avoidance because of bad experiences with many apps.
In the end completely automating the job ended up being easier than building a good interface for a human to do the job.
If you're asking for understanding the actual internal world model they develop, it's basically equivalent to trying to understand a human brain's internal world model by analysing its neurons and how they fire.
There is absolutely no proof for this statement.
https://ahtiahde.medium.com/limits-of-turing-machines-and-al...
If you don't have a CS background, I would suggest reading the wikipedia entries referenced in the medium article as well.
A major challenge here is that it's very hard/expensive to analyze these neurons, especially at any kind of scale within one human.
Not so with LLMs.
That has to be some extremely narrowed version of all research that has happened (or will happen?)
Is any sane person actually trying this?
I can't imagine an LLM ever going AGI.
Probably not.
> assuming that AGI can be achieved by just scaling up current LLM models
Lmao.
Lmao.
Note that there can be other types of LLMs too that are not necessarily transformer based (SSM, RNN, etc)