Hacker News Reader

Comment by mark_l_watson | Hacker News Reader

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially!

I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models.

I was walking this morning with a non-tech friend and I was trying to impart to him my feeling that all knowledge work, and a lot of mundane work is getting disrupted. Even though I have many tech-friends who are "Large Language Model doubters", my friend got it this morning. Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through.

For myself, I am spending a very large amount of time experimenting with the OpenAI APIs, LangChain, and Llama-Index - and I am enjoying myself tremendously.

View on HN

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through.

This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career.

I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to day work and opportunities for employment.

25 years ago in university, I did roughly the same stuff as I do today. Languages are higher on the abstraction ladder, networks and hardware are faster, etc, but stuff is still roughly the same. I can see a real discontinuity in the work on the horizon.

I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown.

You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually.

Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.

Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?

Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me.

But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.

>Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me.

Does it? Slowing our breakneck anti-human business pace and stopping mindless and environmentally and culturally harmful consumption and production of trinkets is surely better!

Huh? Have you attended a local city council meeting recently? I can guarantee you they're running on shoestring budgets with an endless backlog of requests that they are only treading water on because of anemic tax revenue and low prestige for government jobs. "Productivity" isn't just some high tech, consumer bauble producing phenomenon. It helps everyone out. Highly profitable, headcount-bloated big companies are the outlier.

I can guarantee you that the reason they do so, is an explosion of BS requests, extreme bureucracy, and needless requirements, with a simultaneous drop in actual useful basic work - on city instrastructure and such.

> anemic tax revenue

You mean, spending >1/2 their budget on police?

Every BigCo I’ve worked at (and I’ve worked at places that put the B in Big) people aren’t sitting on their hands because there isn’t way more to do than they can feasibly do. It’s usually because they’re fundamentally illsuited for the job and sitting on their hands is better than being homeless, or the people who are illsuited for their jobs feel anxious about being homeless so force everyone into meetings and team building and off sites and retrospectives and sprint planning and burn down chart analysis and whatever else they can conceive of that seems productive but doesn’t change production in any way. To that extent at least an AI would be left alone to write actual code, and those folks can have their endless meetings and still justify their job the same way they ever did. No AI will ever host a breakout session at an offsite to discuss strategy for getting people to work in the horrific hotel seating dehumanized office, they’re too capable.

> No AI will ever host a breakout session at an offsite to discuss strategy for getting people to work in the horrific hotel seating dehumanized office

Well, let’s hope not.

I can already feel it dramatically increasing my personal productivity. I expect it's going to eventually lead to a 2x or 3x time productivity increase for software engineering generally (if not more).

Will that result in less demand for software engineers? I doubt it. I think it will lead to companies and organizations doing way more software engineering.

I was going to use the old "ATMs resulted in more jobs for bank tellers example", but apparently that particular example is flawed: https://www.vox.com/2017/5/8/15584268/eric-schmidt-alphabet-...

> apparently that particular example is flawed

Um, yes. Been in a bank branch lately? Had to visit a bank branch for any reason?

Increased supply doesn't change the demand curve but there are basically no scenarios that don't lead to an increase in volume and a decrease in price.

I mean, people aren't going to be able to solve 100x as many problems because of this.

I don't think anyone is claiming that. But 2 or 3X? Possibly. And that would be enough to be a game changer.

Yeah maybe, but will it lead to less jobs? I am not convinced.

I have started using it for a few days. Basically asking to write and modify pieces of code, for now small ones where I can easily spot a bullshit. I am very much looking forward for it to improve to the point where I can spend more time thinking about business domain rather then how to translate it to efficient code.

Somewhat similar boat; I just hope it winds up helping me do my job better, instead of me trying to teach it how to do my job. Setting aside professional anxieties, after using chatGPT for a bit I quickly realized I am completely uninterested in prompt engineering. But as a tool, like with an IDE, it could be a big leap forward.

I literally just finished writing some documentation and had Copilot running, sometimes it'd autocomplete something with garbage (and I would just type what I wanted anyway), more than once it autocompleted with several lines of exactly what I wanted to write. This is way better than the phone-type autocomplete I've seen in gmail where it just guesses the next word.

I ducking agree

I've been calling the mockery and dismissiveness, "nervous laughter". I think we have a sharp crowd here but they either lack imagination or are in denial.

I do not think that is quite true. Most of what I am seeing is hesitance and wariness.

Every innovation can be used both for good and for evil, and it often has unintended side-effects. Just ask Thomas Midgley Jr., who played a major role in both leaded gasoline and CFCs. Or Alfred Nobel, who tried to make mining safer, only to be appalled at an erroneous obituary to describe him as the "merchant of death" due to its military use.

Over the last two decades we've seen tech giants go from a bunch of nerds in garages to parasites invading our personal lives and consuming every bit of data about us they can get their hands on - and social media went from a cute gimmick to a society-destroying tool powering genocides.

AI is undeniably extremely promising, and it is not hard to see that it has the potential to change the world forever. However, we are inherently unable to understand how it works, or to alter it in a meaningful way. It's a Pandora's Black Box: will it give us a better and easier life, or will it destroy us? Nobody knows, least of all the people who are opening it!

We're at a point in time where almost any novel technology is just going to take jobs away, increase wealth inequality, and down the line make the world a worse place. It's difficult to be happy about a new powerful tool finding its way into the hands of tech giants.

Indeed, it's fascinating to witness a sizable segment of the HN community and distinguished intellectuals like Chomsky displaying absolute denial. I've started bookmarking the comments so I can look back at them in 5 years and have a good laugh. Some from a few months back are already aging badly[0].

[0] https://news.ycombinator.com/item?id=34197033

Has Chomsky been right about anything?

My first introduction to him was when one uncontacted tribe or anothers' language refuted something he thought was fundamental to humans.

In the decade(s) since I kinda think "Chomsky was wrong" is pretty much the only reason people bring him up.

I see a lot of irony in this.

When Chomsky proposed generative grammars, his theory of universal language acquisition, and so on, they were radical ideas that upturned the central cannon of linguistics.

Time has been on his side - entire schools and subfields of linguistics went extinct as more evidence emerged that Chomsky was fundamentally right. Basically every computer language and data format in existence is parsed/lexed in ways inspired by his models of language.

But now Chomsky is considered the stodgy old establishment, and whenever one of his theories is contradicted somewhere on the margins people shout "Aha! He was wrong the whole time!" and ignore the 99% of cases where his models are still the best ones we have.

What's even more bizarre is the ChatGPT is proving him right, that a neural net can build logical grammar, and he is denying it!

What did he say about logical grammars and neural nets in the past? Sorry, not very familiar with him.

He wrote an op ed recently with the oft-repeated argument that stochastic models are fundamentally distinct from structured symbolic/logical models. In simple terms you can never really “trust” chatgpt’s answers because it’s just guessing the answer that looks right, not applying structured reasoning like humans do.

https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...

But might be missing something more specific, would love to read that.

> In simple terms you can never really “trust” chatgpt’s answers because it’s just guessing the answer that looks right, not applying structured reasoning like humans do.

Here's an argument otherwise, that they really are developing some sort of model: https://the-decoder.com/stochastic-parrot-or-world-model-how...

This is a hot take, would love a link/explanation on why you think neural nets are human-like. For example, from the op Ed you’re likely referencing:

“ Their deepest flaw is the absence of the most critical capacity of any intelligence: to say not only what is the case, what was the case and what will be the case — that’s description and prediction — but also what is not the case and what could and could not be the case. Those are the ingredients of explanation, the mark of true intelligence.”

Isn’t that just plainly true of LLMs? Sure they can produce text that looks like an explanation, and sure putting in a header telling it to spell out its reasoning will get it to output text that looks like reasoning, but I feel the nature of its hallucinations make it clear that it’s not actually performing those steps anywhere in the net.

But maybe I’m just a Luddite? These technologies are amazing and transformative, I’m just shocked to see hate on HN of CHOMSKY of all people, the father of modern linguistics and cognitive science…

Why do you think this is impossible? Have you tried making an account on chat.openai.com? Because it can in fact do that right now; and even people who are otherwise skeptical of LLMs have panned Chomsky's article.

That example was highly controversial, and I think Chomsky was actually right in that instance (all languages have recursion). Generally my understanding is that he's considered to have held the field of linguistics back quite a bit with a non-disprovable hypothesis.

About foreign policy? No.

Was he wrong to oppose the Vietnam war?

He supported the communist though. He also supported Pol Pot. You can predict his opinions of foreign policy if you understand that he bases them on the "US is evil" axiom and therefore anyone who opposes it is good, no matter how actually evil they may be.

The Pol Pot thing seems debunked: https://www.abc.net.au/news/2011-07-01/brull---the-boring-tr...

The communism support doesn't seem to quite stick either if WP is to be believed: https://en.wikipedia.org/wiki/Political_positions_of_Noam_Ch...

In Britannica and WP he's labeled an anarcho-syndicalist, so seems to be against both capitalism and noncapitalist authoritarian systems.

Chomsky absolutely was a Cambodian Genocide denailist at the time. He very arrogantly dismissed actual eyewitness testimony of the genocide from refugees purely for ideological reasons.

https://en.wikipedia.org/wiki/Cambodian_genocide_denial#Chom...

(Disclaimer: i'm just reading up about this now)

Did you skip my first link or are you disagreeing with it? It's about the same subject. And in your WP link has a second quote writing about the genocide as fact.

Another I think credible seeming article about it that seems to judge against denialism in an academic journal called "Genocide Studies and Prevention": https://digitalcommons.usf.edu/cgi/viewcontent.cgi?article=1...

Chomsky's skepticism in the media criticism article talking (among other things) about a book about the then-contemporary events of course looks quite unfortunate in retrospect, but given the volume of work and constant writing about world events over a long career it seems to be a error in judgement in the moment, rather than denying history, especially as he evidently updated his views later as more evidence came about.

"looks quite unfortunate in retrospect"

It was completely unjustified at the time. There was absolutely no reason to doubt the testimony of refugees coming from Cambodia, other than they didn't fit into Chomsky's strange worldview that the US is the cause of all problems in the world.

Even today his views on Russia's invasion of Ukraine are completely moronic.

https://foreignpolicy.com/2022/12/22/russia-ukraine-war-left...

For ‘Peace Activists,’ War Is About America, Never Russia Their own hard-left worldview is so absorbing that they will take the side of any aggressor in the anti-Western camp.

https://www.e-flux.com/notes/470005/open-letter-to-noam-chom...

Given how emphatic he is about everything, the answer is clearly no.

I didn't bookmark them, but I recall comments from ten years ago predicting that 50% of the workforce was going to be replaced by AI in the next ten years. Fast-forward to "the future" and the reality is AI has left without a job a grand total of 0 people. The grandiose promises we're hearing now concerning AI are nothing new. They were laughable then, as they're now.

Those comments about employment are indeed laughable and I bookmark them as well. We're at near full employment despite the invention of mechanized farming, electricity, the printing press, cars, the computer, the Internet, etc. I was referring to people who are in denial about the rapid progress of AI, and the impact it is going to have in coming years.

[0] Until we reach AGI, but I don't dare attempt to predict when this is going to happen.

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through.

Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. Part of me wonders if at least some of the layoffs we've seen in the last couple of months are because companies such as Google, Microsoft, Amazon, etc. (the ones that have pretty clear visibility into what's happening in the AI space) are realizing that they aren't going to need as many knowledge workers in the not-so-distant future.

I think there was always this idea in the back of our minds that this was going to happen someday. But someday was always like 15 to 20 years out. Looks like someday is knocking on our door now.

> Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost.

OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at it. You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses.

Yes, it's going to be different but I don't think we know exactly what is going to happen yet.

Kind of like a Jevon’s paradox for software engineering!

> You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses.

And in a lot of cases they're not going to need to hire a software developer to do develop it.

Yeah definitely, but we already saw this coming before LLMs right? Low code solutions, site builders, I mean MS Frontpage is as old as the Web itself. Just like you needed top MechEs to build heat exchangers in the old days, nowadays a small team makes a design and technicians handle most of the actual work of assembly and maintenance. Likewise you'll probably just have low code engineers who glue existing parts together to make things work at your local cafe or restaurant.

One of the reasons the work at Big Tech is so fun is because they actually need to engineer a lot of things. Smaller tech companies or businesses that utilize tech can just glue a few libraries together to make it work.

What do you call the person they are going to hire to use the tool or are they going to switch gears from selling painted dodads and chicken sandwiches to fire up an IDE even a web based one.

The average human is only slightly smarter than the average rabbit. They are going to pay someone who specializes in this task.

Thanks for the future you helped create for us young people. Hopefully the next generation is more merciful to 'useless' people than the previous one.

This is shortsighted. AI slave labor is the only route to a true overproduction and marxism, a welfare state. Or every body become slaves.

Who needs slaves? What use will they be?

Why overproduce?

The ultimate use of slaves (or servants, or employees) is about having power other other people (them) - this goes beyond production needs...

The average person might not care for this, but the psychopaths who crave (and are in) power do...

Overproduce so that the plebs have something after the elites seize the lion's share

Who needs plebs? I mean, a couple of thousand. Maybe, But beyond that?

Who needs people? Elites or not, what use will they bring? To do what? And elites do need plebs without which they’d have no point of reference to set themselves apart

Hi Mark,

One question out of academic curiosity: I'm exploring ways to use these tools for research projects in econ and am struggling to see a good angle.

For instance, suppose I have lots of PDF reports on how firms have evolved on each quarter (10,000 reports or any other number beyond what I can read).

Can LLMs be used to spit out variables based on these reports? EG: indicator variables (optimistic-vs-pesimistic), categories, etc. that I can then use as data to test economic models?

I tried simple cases with ChatGPT3.5 and got the feeling it's great for outputting narrative text, but felt mediocre for when the output was more narrowly defined into categories (which was surprising).

Look at the LangChain and Llama-Index (used to be called GPT-Index) projects that make smaller projects that need to use a large amount of text data do-able. There is also support for reading PDF files (and many other data sources), and pre-computing embeddings. If you spend a short while looking at example code in the documentation, find something that is similar to your requirements (e.g., semantic search, conversational chat about a set of documents, etc.), and build on that.

Amazing; thanks a lot!

Is that what it is? I've been struggling to figure out why people have had such difficulty seeing the clear impact is going to have this

are you finding langchain and llama-index similar?

Reply on news.ycombinator.com