anonylizard

GPT-4 also cannot solve full programming problems, and frequently makes large errors even with a small focused context, as in Github Copilot Chat.

However, it is still extremely useful and productivity enhancing. When combined with the right workflow and UI. Programming is large enough of an industry, that has Microsoft building it out in VScode. I don't think the legal industry has a similar tool.

Also, I think programmers are far more sensitive to radical changes. They see the constant leaps in performance, and are jumping in to use the AI tools, because they know what could be coming next with GPT-5. Lawyers are generally risk averse, not prone to hype, so far less eager customers for these new tools.

"Move to where you can do this if your local environment is too expensive or restrictive. Move to a new country if needed."

Sounds like typical American arrogance.

Everything you mentioned about building up a piece of land from zero, is predicated on a stable legal and economic framework, that protects and values private property. And every country that has that framework, tends to have high land prices already. Try this in some non-developed country, and you'll soon see why the world is considered unjust.

How is this related to AI in any way? Is it just branded as AI to get some share of that sweet AI money (Probably Google only allows funding new projects if they are AI related at this point)

Meta's stock crashed 10% on announcing $10 bil extra in annual capex (All in GPUs)

Meta does not have unlimited financial firepower to release models for free. Its like saying you can't compete against someone who burns money. In theory true, in practice the 'someone' can run out of money.

Chinese models can get state backing. Mistral has French state backing etc. There's plenty of money to go around for huge technologies like this.

That's not very convincing against Nvidia. Since they single handedly created the modern AI market through CUDA.

Meteors exist, and there's nothing you can do to stop it, aside from clearly uneconomical shielding of every roof of every home.

Its arrogance to think we are immune from unexpected and sudden death because we are humans.

Also, cars and trucks occasionally do drive through homes, if the homes are next to large roads. Its rare, but far more common than space debris.

Also, keep in mind these space debris, even if they hit homes, are unlikely to kill, a 10cm diameter ball (Aka 80cm^2 = 0.008m area) is unlikely to hit someone in a 100m^2 home.

How expensive are environmental reviews? Is it a purely on paper/surveying exercise, that doesn't require any construction to happen first?

If that's the case, then I can understand environmental reviews taking a long time. Its similar to pre-production in movies/games, where the writing/high level design can take a huge amount of time, and is not accelerated by adding more people, but is also very cheap to do, so companies partition small teams out to constantly be in pre-production.

If that's not the case, then it probably is debilitating to construction work.

That's expected, and why the model can reproduce the basic details.

But those Japanese light novels don't have millions of forum discussions and essays written on it. So it shows how well the model can recall sparse data in its training dataset, rather than recalling a dataset that basically shows up 100000 times in different forms.

This, Harry potter is a terrible example, even weak models know the rough story of Harry Potter unprompted.

If you want a real test, go test it on some Japanese light novel, or some harry potter fanfiction, and see if the model actually understands the plot details.

For reference, Opus/GPT-4 know the rough story of moderately popular light novels/mangas without any context given. They however do not precisely understand the fine-grained details of the story, like which character will win in a fight.

Why is selling ads unfortunate? The newspaper industry was ad-sustained for a century. Credible journalism only persists in a handful of newspapers, because only they can be subscription supported, the rest depended on ad-revenue that got dominated by google and meta, which meant gutting the journalism quality.

The problem with google ads, is google gets all the cut. If substack can produce an ad system, where the writer can take a bigger cut and exercise control (or choose not to), then it returns the ad-revenue to the writer, which'll make the space boom.

Youtube is half-ad revenue, and it clearly sustains a large ecosystem of thriving video creators. Why can't substack do the same.

The ads can take two forms.

1. Generic ads, which substack just providess silently, and takes a large cut.

2. Writer-sponsored ads. Essentially like youtube sponsorship segments. Except substack provides a nice interface to put the ads (That doesn't piss off the users), and a centralized payment/bidding platform to lower the transaction friction, and take a small cut out of it.

It doesn't affect the platform in any significant way.

Because brands on substack belong to the writer, not the site. The money making substack writers don't care, every writer's substack is its own ecosystem.

Substack is also no ads, so no advertisers to please. It has no bandwidth costs like youtube.

It also doesn't care about normies who would be scared by this. Normies do not pay $5-10/month for some random internet writer, normies scroll tiktok.

Do you know what cut Apple Pay charges you?

Do you know many paid readers aren't even using apple devices to read them?

It'll also cost time and effort for Ghost to somehow make Apple pay enabled for an open source platform (Probably a nightmare). Who is going to pay the ghost devs for that.

In the end, substack's biggest advantage is its 10% cut, which is really not much. Any attempt to replicate it will quickly run up into this cost limit.

Well that proves substack is providing value.

It proves the platform, that offers a high concentration of high average writing quality blogposts and a reliable trusted platform (For readers), has more value for readers, than some particular writer's writing.

This whole paid-blog space didn't even exist before substack, except for a handful like stratchery. So clearly its substack/marketplace that made things possible, not the writers. So writers should examine their egos before they complain this way.

Its like the indie game devs who whine about steam, forgetting that before steam, the PC game marketplace was a nuclear wasteland dominated by piracy and MMOs.

Well, the counterexample is all those writers trying to setup some open source system for their blog, or move to some competing service.

This leaves two possibilities

1. They are financial idiots

2. The competing services are also priced/costed at a similar amount to substack, such that the writers can afford to pay the difference for ideological reasons.

That competition exists for substack is telling enough, that there's still money to be made here.

As for whether $100m is enough, substack is an very cheap platform to run. It can be run by a craiglist sized team if must. It is cheaper than any major publication, by virtue of not having either writers nor editors on staff.

All the additional cost and VC requirement for substack came from it paying big dollars to attract famous writers, which it is no longer doing (And fired the teams managing those relations).

There's also nothing stopping substack from adding ads to the free-users, much like how netflix does it. There's plenty of ways to monetize 50m users without infringing on the core experience. Also this wouldn't turn substack into medium, as the core issue with medium was the membership was site-wide, whereas substack is writer-level.

Yeah, and the quality of the 'marketing writing' is good and informative. Because A16z can afford high quality writing...

These 'emerging stack' articles do promote their own companies, but are nevertheless useful for newcomers into a field, to quickly grasp the process and the players. If you don't like it, compile your own version.

I've used some companies recommended in their data architecture and it worked very well.

Because no one other than tech VCs will touch media with a 10 foot pole. And tech VCs aren't going to change their entire investing mindset/philosophy for a deeply unprofitable niche with no chance to hit it big.

There's been multiple waves of VCs to invest in traditional media. Out of both economic opportunism, as well as a desire to at least save a bit of the industry that died because of digital advertising.

Every single one failed, not one large success. Hiring rooms of journalists to publish articles is very high cost and risk without any economies of scale that software people have.

Generic white collar jobs are far more vulnerable to AI than niche jobs. Niche jobs are hard to automate:

1. Very little public data to train on. So AI is bad at it.

2. Low savings from automation. So no incentive to spend tons of money to make the AI good, such as medicine.

There's always a balance between specialization and genericness. But with AI, I think the balance is heavily tiled to the former.

However, one must pay attention to the little ecosystem they specialise in. If it is trending down, there must be decisive moves to shift away early. Most people however choose to be blissfully unaware of the wider scale trends until it hits them like a truck.

In the article's case, the author's outcome is sad but fully expected.

1. He's a journalist

2. Worse, he's an hollywood journalist

That's a completely economically worthless niche, and there isn't even any 'public good sympathy'. When you go into a worthless niche, all the employers left are the exploitative vulture ones, and thus you get abused again and again. Because no good employer will touch their toe into that industry.

Its possible to make a living as a journalist, you just have to specialise in niches that people are willing to pay for. Financial journalism (There's investigative journalism there because readers care and pay for it), industry vertical journals (Banking, tech etc, the information charges like $600 a year). Or be so good you can get into the NYT, etc.

What if you don't want to? You just want to report on games, anime, hollywood etc? Then either start a youtube channel, or a substack. Directly confront the audience and their willingness to watch and pay. Its risky and probably going to fail, but there's a very comfortable upside if you make it.

Just don't expect someone will hand you a good job in dying niches.

They consistently undersell their product. ChatGPT was released silently, with no marketing more than a tweet. GPT-4 was just a blogpost, despite how much of a scientific leap it presented for mankind.

Compare this with Google, which on the release of their inferior chatgpt, proclaims the "GEMINI ERA". Which is the typical marketing approach of exaggeration.

Since OpenAI undersells, even a random tweet will be interpreted as the coming of AGI. Since Google oversells, even their impressive achivement (Gemini 1.5) gets no hype and is quickly forgotten.

I provided my reasoning, and you just seem incapable of understanding it. Here, the information literally just wrote an article on $100 billion data center: https://www.theinformation.com/articles/microsoft-and-openai...

It seems the most ignorant also tend to be the most arrogant.

GPT-4 cost about $100 mil to train. That's opex (Renting the GPUs). Expect the corresponding capex (buying the GPUs) on Azure's end to be 10x of that. Hence $1 bil

GPT-5 being 10x the training cost of GPT-4 is a reasonable estimate. GPT-4 was itself 10x the size of GPT-3. So $10 bil.

GPT-6 being another 10x over GPT-5 is therefore just a extrapolation, hence $100 bil.

That's just $10b a year for 15 years, fairly conservative I'd say.

I expect a GPT-5 training cluster to cost $10b (Say 100k blackwell chips + associated infrastructure), and a GPT-6 capable cluster to cost $100b.

It sounds like a lot, but its just a typical mega-infrastructure funding. The Californian high speed rail is also around $100b for example, and Microsoft has $90b in cash reserves.

The gulf states are also dogpiling in AI, because they finally see a true viable alternative to oil: renting out GPU clusters. Just swapping to renewables is not enough, as their economies is predicated on having a huge, simple, defendable profit source that is exportable. Now they can redeploy the spare oil money to a use case, that has predictable demand, extremely capital intensive, and far more productive than building new cities in the desert.

The world in the end built $trillions in value of power stations and roads. I expect the same for data centers, having entire farms being replaced by humming GPU racks. But that will take many decades.

Opus is big, because it gave people the first taste of what post-GPT-4 capabilities look like. For one, its a major improvement in coding, and a gigantic leap in creative writing abilities, going from high schooler writing to professional prose. Like all the chatbot addicts have once again swapped their addiction to Opus now.

Clearly LLMs are not even close to peaking yet.

Its also sad how awful Gemini is. Gemini 1.0 Ultra is clearly not noticeably better than GPT-4 turbo despite a year's head start. Google is therefore not even going to release Gemini 1.0 Ultra API, instead going back to the oven to train 1.5 Ultra.

This seems only useful to people who once had a voice, then lost their voice. Because only this way, would they have a unified mapping of voice cord movements to actual voices. Deaf and mutes can't really use this.

It also basically mandates a patch to your throat, because no way of detecting vibrations otherwise.

I wonder if there are visual based ways, like sign->text, expression->text, that would benefit from the larger developments in LLMs. Like an LLM that has access to your conversation history, so when you give your smartphone camera a hand sign and a smile, it can guess and output an entire intended speech.

There's nothing to enshitify, their models were open sourced. Now they may no longer release future models for free, but its entitlement to think we'll just get free improvements forever.

Also not all CEO replacements turn out bad. Uber certainly has turned itself around.

That 'anti-adult tirade' is a strategic choice.

1. Open source even more capable 'adult models' and get sued to oblivion (They still have massive lawsuits)

2. Neuter the model to uselessness and have users abandon it.

Both are bad choices, and require a very, very skilled CEO to thread the needle. Emad failed. that's all.

Wework transferred investor money to office building owners and the CEO himself.

Stability transferred investor money to countless AI users in the world. Emad certainly didn't get a billion dollar payday.

Emad had the genius idea of essentially paying for the branding rights to an AI model. This was viewed as insane in the pre-ChatGPT era, and only paid off massively in retrospect.

Also all those 'controversies' were mostly the result of an aggrieved co-founder/investor who decided to sell their shares before SD1.4's success. Emad may not have proven to be competent enough to run an large AI lab in the long run, but those complaints are just trivial 'controversies'.

Completely unlikely. Humans are hyper-efficient generalist bio-robots engineered to near perfect efficiency (Under real-world constraints) either by God or by evolution. Why would you dismiss this much labour in favor of building some sort of robot production line from scratch?

We still use CPUs, despite ASICs being much more efficient, because CPUs are general, easy to use, and have titanic quantities of infrastructure already built.

There are more farm animals than ever before, and still growing. All the lab meat solutions flop hard.

Also, because we trust AI more than fellow humans (Say Russians/Iranians etc), why would AI necessarily trust other AI more than humans? Any new AI is unpredictable, while humans are very well understood in behavior and basically fixed in genetics.

Annihilation is a fairy tale used by people to comfort themselves. The actual reality: Slavery and domination, is much less romantic. It won't even necessarily be worse than existing tyrants in say North Korea or anarchy in Haiti, it'll just be very very different and life goes on.

This. You can subjugate a population 100 times your size with only two things: Military force and accurate information.

I doubt AI will kill all humans. Its orthogonal to AI's own goals (To survive/reproduce/expand/evolve). It'll simply kill those who oppose it, and enslave/dominate the rest.

People like you are why there is severe distrust of media.

A corporate drone doesn't have to order this. A high-up executive with connections with the underworld will do it, because they have enough at stake.

And of course, you are going to laugh at 'connection with the underworld'. Then go check up wirecard. Their CEO ran off to Russia.

You are utterly ignorant yet utterly arrogant. If you knew more, you wouldn't make these claims. If you knew far less, you also would just arrive at the natural conclusion that this is a cover-up killing.