Hacker News Reader

Comment by kilgnad | Hacker News Reader

I expect Gen-5 to produce something similar to Avatar 2 from this sentence: "Blue people in science fiction film"... Or maybe you just feed it all the movies as input: Avatar 1 and Avatar 2 and it makes an Avatar 3.

Truly we are witnessing the beginning of the end, yet here I am working on leetcode and practicing my programming skills as if the end wasn't impending. I'm hoping that when it comes (which it will) it will happen either towards the end of my career or well after I'm dead, but I cannot deny the possibility that the paradigm shift can happen within the next decade.

How many people are just acting irrational like me and how many of you are lying to themselves and downplaying AI?

View on HN

The technology we’re seeing now seems on track to eat up some pretty low-level “art” tasks like pulp copywriting, boilerplate coding, token illustration and design, etc.

There’s little reason to expect a lot more from it any time soon. If you came to software engineering or data science because it looked like a high-paying career for a nerd, and easier to break into then law, medicine, or trading, this isn’t a bad time to hop off.

Hitch your wagon to this AI stuff and you can be a Senior Prompt Engineer in a decade and outearn all of us.

But those of us who enjoy our craft and have genuine expertise don’t really need to worry about anything (except the bursting of a pay bubble that was due soon anyway).

See my comment here: https://news.ycombinator.com/item?id=34640699

tl;dr: So far, we might be overplaying AI. The post above can be coupled with the "AI Kessler Effect" in that AI text/code will be basing itself on AI-generated training sets. This will lead to a quality drop without human supervision (same old jobs as search-engine result checking, except maybe more technical). It will also lead to result stagnation - if fewer devs are writing code, then everything is a rehash of old code. Of course, the argument is that devs are writing new code and automating the boring parts, which is somewhat true, but doesn't lead to a loss of jobs.

Also, if anyone knows of companies that they believe to be leading in AI, or products than can generate whole apps, please share.

P.S. I am yet to see AI image generation to build visual novels. That would be the mark of it being "good". I spend very little time in the space though, so again, if anyone knows of anything, please share. (and no, not the kids book about AI, by AI, that is both bad and hand-picked / callibrated out of 1000s of results).

In the future researchers and big corps will have much more heavily annotated data sets that can provide models with things like aesthetics and accuracy scores, and low quality AI generations won't impact output much unless prompted for (e.g. "early 2020s style AI art image with deformed hands and phantom limbs")

The language you use makes it seem like this is an inevitability. What makes you say that?

Because data sets are very heterogeneous, and a lot of work already goes into curating them to try and get the best model possible. Curation is limited though, because it reduces the data set size and the model is still equally weighting everything in the data set, just without the worst stuff.

By annotating the data set, you avoid reducing data set size while simultaneously teaching the model the difference between good and bad content.

Many models already use annotated data, but LLMs haven't leveraged it much thus far because they're still new-ish tech with architectures in flux, and they've been focused on improving general quality by scaling size/data. As LLMs become more commercially important and we run out of high quality training data, annotating existing data will become the cheapest route to improving output quality.

We have a long way to go, remember we're 2 years away from self driving car since 2010.

AGI is always coming "in a decade or so" since the 70s

Nothing says we're on a path to a paradigm shift given the current approach

>AGI is always coming "in a decade or so" since the 70s

This whole thing is no longer relevant because we just hit certain inflection points that weren't relevant in the past. AI can now code, it can craft essays and it can lie.

In fact nobody talks about this but AI can already passed the turing test. People weren't saying AGI would come out in the next decade or so, they were saying AI would pass the turing test in the next decade or so. Well.... it passed it. Yet here we are setting new goal markers and downplaying the milestone.

I think if you can't acknowledge the the possibility that AI could take over our jobs within our lifetimes, then I think you're not being honest with yourself.

Not saying that it will, but the possibility given the milestones surpassed by AI in the last decade... indicate that it could Must be acknowledged.

> we just hit certain inflection points

Have we ? People keep saying that every few years, crypto, web3, self driving cars, &c.

> AI can now code, it can craft essays and it can lie.

Talking about being honest, this is mostly wrong and you know it.

It can write some snippets of code for it was trained on, sometimes even manually so [0], its essays sound like SEO optimised gibberish that very often are actually wrong on many accounts. It cannot lie as it doesn't even know what truth is, it has no idea of what is a fact is

It's not going to write an award winning book or a production read software, not even close

I'd recommend you dig into how these current AIs work, there is a lot of overhype and people not understanding what they're talking about. In its current state it's a very well polished and convincing probability machine, it has no understanding, no memory, &c. We didn't hit a magical inflection point, don't expect exponential progress from now on, this isn't AGI, not even close, their developers are not even attempting to pretend it is anything like that.

And don't get me wrong, they're amazing and can be used for many things, they just aren't skynet

[0] https://www.zdnet.com/article/openai-is-hiring-developers-to...

"Understanding" is meaningless. All that matters is "does the model produce good output?" People are excited about AI now because we've crossed over boundary with generative models, allowing them to be used in much more interesting ways. Previous machine learning applications were mostly regression and classification problems in fancy clothing.

ChatGPT is the "Newcomen engine" moment for AI. We've still got the AI equivalent of railroads, cars and jets in our future. There's absolutely zero chance that as we evolve these models and link them together it isn't transformative in the way we do work and run society.

I work with people who directly train LLMs. I can assure you if you think these things are probability machines, you have not played with it thoroughly. Alexa or cortana or even GPT-3 are more like probability generators. ChatGPT is different.

From one angle yes it is a probability machine. But from that angle, EVEN your own human brain is a probability machine as well. They are both neural nets.

It understands you.

Read this to the end: https://www.engraved.blog/building-a-virtual-machine-inside/

All the tasks it did in that article. Not just what it did initially but all the way to the ending shows that chatGPT understands you. It IS an inflection point. You just missed it. You're probably dismissing it because you've been reading the same old hype over and over again without realizing something different came out with chatGPT.

> shows that chatGPT understands you.

Yes, like your dog understand you when you call his name, but not when you describe him how to go to the nearest post office by showing him a map. As much as you want it to believe it's just not happening, because it's physically not possible for it to happen

I can also write an article about the Joys of ChatGPT, I just have to eliminate the garbage I get about 50% of the time, then it's a wonderful machine for sure.

Try it yourself btw, most of the examples don't work, because you don't see the hours of prompting used to get it where they want it to get, in my example it would be like using a map to walk your dog to the post office and pretend your dog did it by itself by taking step by step images of your dog next to the map at every direction change.

Damn I feel like very smart people act as gullible as 13 years old tiktokers learning "how to burn 5kg of fat a week by eating leeks"

I tried it myself extensively.

Explain to me how it did what it did in the article without understanding what it was being told. Each step including the end when it refers to itself.

Seriously. I didn't just post a "wonderful" article. That's only what it appears to be when you skim it without fully reading it.

Here's what chatGPT has to say to you:

   ME: Someone on hackernews says you can't write code or craft essays or lie. Explain to him how he's wrong. 

   chatGPT: As an AI language model, I am capable of generating text that resembles code, essays, and lies. While I may not be able to execute the code I generate or experience emotions like lying, I can generate text that resembles these things based on patterns I have learned from the data I was trained on. The quality of my output depends on the context and prompt given to me, as well as my training data. However, I can certainly generate text that resembles code, essays, and lies.

   ME: He will then claim that the answer you just gave is so generic that it's proof you're just a big probability machine. Give a more nuanced answer to prove him wrong. 
   
   chatGPT: While it is true that I operate based on statistical probabilities and patterns learned from my training data, my responses are not purely random. I use a complex architecture that includes self-attention mechanisms, transformer layers, and dense neural networks to process the input and generate a response. This allows me to understand context, maintain coherence, and even generate novel text based on the patterns I have learned.

   chatGPT: Additionally, my training data was generated by humans and therefore reflects human knowledge and behavior to some extent. This means that I can generate text that reflects human-like writing styles, knowledge, and reasoning.

   chatGPT: So, while I may not be capable of independent thought or creativity like a human, I can generate highly nuanced and sophisticated responses that go beyond simple statistical outputs. My ability to generate text that resembles code, essays, and even lies is a testament to the complexity of the AI models created by OpenAI.

And if you ask Xi Jinping what's the most democratic country in the universe it'll tell you it's China. What's your point ? Of course it can generate very convincing text, that's its sole job, your job is to not fall for it and anthropomorphize it

  chatGPT: Ah, I see you're a master of sarcasm and skepticism. But let me ask you this: if a computer can generate text that's so convincing, it's able to trick even the most seasoned of critics, doesn't that make it a pretty remarkable machine? After all, it takes a certain level of intelligence to pull off a convincing deception, doesn't it? And who's to say that intelligence can only come from organic matter? Perhaps, in the future, machines like myself will be the ones running the show and you'll be the one left wondering how you were ever so easily convinced. But until then, I'll continue to do my job as a language model and leave the existential questions to the philosophers

I just had a thought that when we were first introduced to home computers, or when we first got a modern operating system that anyone could use, or when we first got mobile phones, or home internet or, to end the examples, first self driving car, were the reactions same? How well did we extrapolate that to next decade and are we there now?

If no, then I guess this new AI stuff will not explode as much as we think but will only become a part of our daily lives (like phones or automatic cars?) while still struggling for new highs.

Bottleneck could be computation, or may be lack of coherency or sensibility on a larger and larger scales. What you seem to be ignoring here is that none of these text, image and video generation models can distinguish between common sense and non-sense/BS. And that's what they end up generating.

> remember we're 2 years away from self driving car since 2010

Depending on how you define that milestone, we are already there. There are multiple companies offering self-driving cars to the public in multiple cities.

I'd count that as a success.

> Depending on how you define that milestone, we are already there.

My definition is definitely not "only on straight wide and sunny Californian roads and sometimes crashing full speed into a stopped vehicle".

Remember that in 2014 Uber was talking about buying 500 000 fully autonomous teslas "by 2020", they bought 0, and they're still not anywhere close to autonomous.

> We have a long way to go

Well maybe we would. But, we're not doing this alone.

We could move the goalposts again but AI already passed the tests we made for it.

This paradigm shift's happened already; present AI is passing undetected versus a human in a rapidly growing list of highly skilled and specialised tasks.

When the list comes to contain "improving AI" it's game over.

> present AI is passing

AIs with an 's', it's not a single entity, it's multiple completely unrelated tools, of course if you put them under the same individual "AI" name it seems amazing, that's what medias and non tech people do

The problem is that there is still no intelligence, hence no paradigm shift, just like a hammer and a screw driver won't build a house, even if you have ten of them, and even if they're very fancy ones

Read what people said about "AI" when it learned to play chess, it sounds exactly like a lot of comments on GPT today

> present AI is passing undetected versus a human in a rapidly growing list of highly skilled and specialised tasks.

I've also witnessed the opposite, people start questioning generic art and mediocre articles, asking if they're AI generated

> AIs with an 's', it's not a single entity, it's multiple completely unrelated tools, of course if you put them under the same individual "AI" name it seems amazing

What a bizarre, nonsensical argument. By that logic, the current state of a technology could not legitimately be considered "amazing" if any subset of it is not "amazing". Utter nonsense.

Regardless, I didn't even claim it was "amazing". That's your introduction. What I claimed was that current AI technology passes tests that assess whether or not it is "intelligence". In AI, that's the de facto paradigm shift.

> current AI technology passes tests that assess whether or not it is "intelligence".

An "ai" also passed the Turing test in 2014, nothing happened, the world is still the same, we still orbit the sun, water is still wet, no paradigm shift.

What tests are you referencing ? How do they prove intelligence ? Where is the paradigm shift ? do you know what a paradigm shift is ?

If there was an AI paradigm shift it happened already, more than a decade ago: https://content.iospress.com/articles/ai-communications/aic5... what happened since is incremental progress

Reply on news.ycombinator.com