AI Hyperopia | Hacker News Reader

AI Hyperopia

mvcalder-01701.medium.com

91 pointsmvcalder82 comments

I liked the main theme of concentrating on practical AI that adds value right now, but disagree with his opinions on CoPilot.

I use CoPilot in all or my development modes except for the LispWorks IDE. I find it very useful, especially when I toggle over multiple code completions, picking the best one. It is not as much about writing code for me as it is saving time looking up documentation. I usually work in Emacs with a REPL so checking generated code is quick.

> The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not real intelligence.

> Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'"

https://en.wikipedia.org/wiki/AI_effect

I think that we're going to see that again and again.

At some point, we should learn and try to manage expectations, by using less emotionally charged words.

Because it’s all narrow AI that’s often hyped as it’s science fiction counterpart. Augmented intelligence would have been a better term, because that’s what AI had been to this point. Leave the term Artificial Intelligence for AGI. If and when the day comes that we have Data, HAL, Skynet, then I don’t think the critics will be making the same complaint. They’ll have bigger things to worry about.

I hate this meme. Essentially, an arrogant and condescending straw man argument.

Here is what really happens. People who are not familiar with programming assume that some mental faculty is necessary to solve a particular problem. Indeed, it is required for a person to succeed at the problem. That mental faculty is often general and can be applied to other tasks. It's some aspect of what we consider general intelligence.

Instead, software engineers look at a problem and solve it by changing its representation to fit the capabilities of a pre-existing algorithm. The solution has nothing to do with mental faculties in the general population. The solutions ends up being specific. If you want to apply it to another problem, you need to transform that problem into some clever representation as well.

People notice the lack of generality and say "no, this is not what we had in mind". It's an entirely reasonable sentiment.

+1. You will find such positions often championed by people who don't understand how machine learning works - but maybe have tried out a fancy tool that used such a model (GPT-2, Stable Diffusion, etc). Classic example of "any sufficiently advanced technology is indistinguishable from magic" combined with Dunning–Kruger effect.

thanks your opinion, I usually see this but can't find a good reply.

> An Intellisense AI that detected when an arrow key press was an intended interaction with Intellisense and when it was a continuation of the normal typing flow could smooth out these awkward discontinuities.

I think this is bad UX actually. It should be obvious to the programmer what the arrow key will do next, the programmer shouldn't have to guess what the AI is thinking. Navigation should be predictable

As I often say, gravity should always be down. Tools like IDEs, autocomplete, search, etc. should be completely deterministic. The same action should always end in the same result.

The reason for this is that the IDE is an extension of the arm. The brain literally treats it as an extension of its own capabilities though processes like transactive memory. It can only reliably integrate and achieve a state of deep flow if it can form a reliable model of your external brain. If actions don't always give the same result, there's no model to integrate.

>Navigation should be predictable?

That is the whole point of the AI. It should match the user's prediction on what will happen.

There is rarely a single answer for what is "intuitive" UX, everybody's opinions may differ. So either you're going to disagree with the AI sometimes (making it unpredictable) or the AI needs to read your mind (we don't have this technology yet)

The problem is the naming.

Even weak/incomplete AI results tends to fuel the imagination, we can easily fill the gaps with anything from fiction or our own dreams.

On the contrary, when AI works and is strong (like with chess) we stop calling it AI and start seeing it for what it is.

Stockfish is called a chess engine. Not an AI.

And the algorithm behind current chess engines is not super smart, it is mostly brute force.

The trick is to avoid doing brute force at every move, but instead doing the grunt work once a huge computer and store everything in a compact form.

I agree on the general thrust of your comment. Though it's worth pointing out that the just the neural net behind AlphaZero is still pro-level on its own, without any search algorithm attached (the "engine", so to speak). It's not super-human without the engine, but still "super smart" for some definition of the words.

That said, Stockfish NNUE took the heart of AlphaZero (the evaluation neural net), ripped it out, refined and shrunk it, and stuck it back into Stockfish, and now Stockfish is clearly back on top and is still clearly an "engine" and not an "AI" as you say.

It seems like this will be the case for AI vs. Engine in a lot of fields going forward. A bio-inspired "AI" approach will do well. Then that thing will get stripped down and refined to the bare minimum to do some particular task, and it will then be more of an "engine".

It will be very interesting to see what this looks like for "general intelligence". That is, how many components of intelligence are separable and distillable into engines? E.g. reasoning, creativity, knowledge, foresight, etc.

In my opinion, the process you describe when transforming the AI approach into an engine is simply related to our understanding of why it works and how.

And this can clearly be done to cover a wide range of useful things.

But I would make two observations, first, neural nets are much more closely related to applied statistics than biology mimicry. The historical name should not matter much but it does because humans are driven by emotions, even in research.

Second, AGI is still too badly defined to be seriously discussed.

Every time people can observe some progress in the AI field, they are tempted to make predictions that, in my opinion, are impossible to make.

We're getting closer. That's all.

> first, neural nets are much more closely related to applied statistics than biology mimicry

Well, aren't biological systems really really refined statistical systems?

Maybe, but real neurons are not closer to artificial neurons than a digestive system is to a furnace or a barrel of acid.

There is a very loose relation between them, because neural nets were historically inspired by the way real neurons are connected.

This is not saying anything about the capabilities of neural nets, I think they can be extremely potent, up to AGI level, but not because of biological mimicry but thanks to Computational Equivalence.

https://worldscienceu.com/lessons/2-6-principle-of-computati...

I have a theory that the fact that the net on its own is a decent player is precisely its greatest weakness. Obviously the deep NN must be doing some type of search, but this search is maybe slower than the engine search, because it can't interact with the various engine cache tables, and feed into the statistics used to heuristically order moves.

Stockfish NNUE is a much simpler neural net that's just a really strong, static evaluation function, not doing its own search and therefore the engine can search further and get the most out of the data(transposition table, continuation tables etc) generated by that search.

I think the AlphaGo architecture makes more sense for Go than the chess family of games, but that's a different discussion altogether.

Yes, it does seem plausible (or even likely) that a deep net (AlphaZero is 80 layers deep) is doing some kind of implicit search from layer to layer.

Copilot has been really useful to me, and if you know what you are doing it's a massive time and mental energy saver.

You can let it draft an initial approach to a small task and then you refine it, I've found this works well, and in practical terms, I end up less tired after working in collaboration with Copilot.

I don't expect it to give me the perfect answer, and it doesn't remove the need for tests (which it can also help creating). But as an assistant? It rocks.

> CoPilot changes the kind of work the engineer has to do, from authoring to editing, but the amount of work is not reduced.

Disagree based on my own experience using CoPilot, but it would be interesting to think about ways to fairly test this.

> Reweighting these potential choices with the most likely given the current context and showing only the most likely is a solvable AI problem.

Not sure about other editors, but for JetBrains IDEs at least this has been a thing for a while: https://www.jetbrains.com/idea/guide/tips/enable-ml-code-com...

Me too, It has reduced the amount of work and by extension allowed me to have more productive work sessions.

These AI tools are not static. They are products with well funded teams behind them constantly iterating to make them better.

I don’t dismiss them, but I do put a low weight on arguments of the form “AI is not there yet” give how far it has come in the last 5 years. By 2030, I can see a product understanding the context of a system from multiple repos and, given an intent, producing code and deployment infra that adds new features respecting constraints

This is good. A single dev will be able to do the work of a full team and then every small business will be able to develop software that suits their specific needs with a couple of dedicated employees

There just aren’t enough “full stack” machine learning folks who can quickly solve these problems.

The pure programming people have trouble scoping out which solutions to try due to lack of experience. The pure ML people code in Python notebooks and have little visibility into these issues.

Both folks could easily learn the other side and help, but it’s surprisingly rare to see.

Copilot is a verbose savant heavily afflicted by Dunning-Kruger... but an extremely useful one.

Do you remember how Googling was a skill?

Learning to Copilot, Stable Diffusion, GPT are exactly the same thing.

Copilot's full power (at this time) does not exist in generating reams of code. Here are a few things it excels at:

- Snippet search: Say you can't remember how to see if a variable is empty in a bash conditional, ask.

- Template population: Say I have a series of functions I need to write in a language without good meta programming facilities. I can write a list of the (all combinations), write one example, the ai will pick up on the rest.

- Rust: If I get trapped because of some weird borrow checker issue with `fn doit(...`, I begin rewriting the function `fn fixed_doit(...`, and 9/10 times Copilot fixes the bug.

> I begin rewriting the function `fn fixed_doit(...`, and 9/10 times Copilot fixes the bug.

This blew my mind. Thank you for sharing, I never thought of approaching it that way.

> Say I have a series of functions I need to write in a language without good meta programming facilities. I can write a list of the (all combinations), write one example, the ai will pick up on the rest.

If you are coding in a language that requires tons of boilerplate, the solution is to use a different language that require less boilerplate.

Why isn’t “have an AI write the boilerplate” an equally good solution?

Asking earnestly.

Because you still have the boilerplate. When code has lots of boilerplate it's harder to find the important stuff.

Not everyone choses the language they code in and boilerplate amount is not the only consideration when selecting a language...

The basic problem is that machine learning systems don't really understand what they're doing. So most of the time, they do the right thing, and when they do a totally wrong thing, they don't notice. Works fine for advertising, not so much for self-driving.

Look at those systems that generate images or text from a prompt. Usually the results are good, and sometimes they are totally bogus.

As I point out occasionally, the big hole in AI is "common sense", defined as not screwing up big-time in the next 30 seconds. Until that gets solved, AI systems can't be trusted very far.

The control theory people are trying to fix this, so they can use ML to build control systems with safe behavior. The math is really tough. Way beyond me. See IEEE Transactions on Control Systems Technology to follow this. People are trying to make control theory math and ML math play together. Control theory usually has continuity - if it does the same right thing at 0.21 and 0.22, you can be confident it will do the same right thing at 0.215. ML systems do not have that property. Which is why we see those image recognition demos where some minor change in the background noise totally changes the result.

Can’t we create a model that asks questions when the other model gets something wrong. The model will get very good at asking questions and then the other model will update accordingly..millions of people using the system will keep it continuously improving itself.

Won't the watcher model then need a watcher? And the watcher watcher model too? Etc.

That’s basically what a GAN is — they’re great, but not a panacea.

Doesn't really work that way in that it still reduces to an overall system that doesn't understand what it's doing (someone else mentioned GANs, they are a great example of how unwittingly incorrect ML model output can be - go look at the out-takes from any image generation model)

A better approach is good uncertainty quantification (this is not at all trivial) to quantify when the model is and isn't qualified to be making a prediction.

> The basic problem is that machine learning systems don't really understand what they're doing. So most of the time, they do the right thing, and when they do a totally wrong thing, they don't notice.

Agreed, but how is the different from humans?

“Instead CoPilot attempts to solve the most general problem, have the engineer state their intent and let the AI produce the code.”

And here in lies the rub, 90% of the time, the engineers intent is wrong. That’s what real pair programming will help you with and what an ‘ai pair programmer’ will not. It just helps you build something that’s probably wrong faster.

I actually find the "build something that is probably wrong faster" really helpful. I frequently write a function and a quick bit of documentation about what it should do, let copilot finish it, realize my intent was wrong, rethink the intent, etc. If I didn't have copilot I would follow the same process, but it would take a lot longer to realize my intent was wrong.

It takes some time to learn what copilot is capable of and how best to prompt it, but I'm my experience it very quickly pays off.

Tangent notes but I would really like an AI writing my commit messages and pull request summaries. Is it a thing?

Try pasting your diff into the GPT-3 playground (or just a Copilot enabled edit window) and type "write the commit message" and see what happens

> AI is software based automation. That we imbue any automated process with intention, emotion, and intelligence, is a defect of our human software

Matt Calder thinks he's teaching us something about AI, but he's really just teaching us something sad about Matt Calder.

Wake me up when AI will join the customer call, negotiate features with product managers, and figure out exactly what needs to get built.

I already use gpt3 in my daily life to refine details for things I'm building for work, and to advance personal goals. If you give AI all the pertinent requirements, it's great at coming up with solutions. That's despite the fact these early language models are optimized for coherency and not correctness. So I wouldn't think we're too far off from waking you up.

Do you have any examples you can share?

I've used a predecessor of gpt3 quite successfully for decades:

https://www.amazon.com/Mattel-Games-Magic-Ball-Retro/dp/B014...

https://en.wikipedia.org/wiki/MegaHAL

In a post-dall-e post-stable-diffusion world I’ve completely 180’d on this argument as defending your job worth.

I used to think you’d struggle to replace an actual illustrator/artist because clients always want something specific. But really it isn’t about you giving them what they want it’s about them having to deal with you at all.

Why would I message the illustrator, negotiate rate, have a meeting then have back and forths trying to get what I want when AI can be running from the first moment without interacting with anyone else and it can be spitting out versions of the thing before you have the actual person in a video call.

It can do this all day and all night then someone just had to choose what version of the thing they want or nudge it closer.

It happened to art and I don’t see why devs think they’re immune.

On a long enough timeline, there may only be unfortunately project managers.

(Still think building great and opinionated work has value. But low level generic work is on the chopping block as far as I can see today)

Re: Art, it'll be interesting what all this brings to the table. On one hand, tools for artists, on other replacement for artists. Turns out, we might be getting into polemics of what art is and it'll turn out that skills (as in trade) are part of it, but not the most important - it's artist's taste that sets them apart and drives them above. Now, everyone will face the same since skills barrier will be brought down.

>It happened to art and I don’t see why devs think they’re immune.

Probably because it's a different problem domain that actually requires precision and the requirements often are unexpressable in natural language.

Also, it has not happened in art. You can say it has happened in art when 50% of currently salaried graphic artists are unemployed.

>actually requires precision and the requirements often are unexpressable in natural language

Same sentence could describe art.

The first part couldn't and the second doesn't apply to the current (and I suspect any future) mechanism AI models use to generate...anything.

> Also, it has not happened in art. You can say it has happened in art when 50% of currently salaried graphic artists are unemployed.

It’s barely happened to anyone. There isn’t a general pattern of automation replacing jobs, so it’s not a reasonable thing to assume will happen.

Automation replaces tasks and makes things cheaper, both of which make jobs more productive and more worth keeping.

> "On a long enough timeline, there may only be unfortunately project managers."

You won't need those either. AI will be able to just use your neuralink implant to build a complete product based on some vague ideas, in minutes. Goodbye daily scrum meetings and quarterly planning.

In all seriousness now - who is going to be building this "AI"? What about the platforms it has to run on (both software and hardware)? The tools that team has to use?

Will AI find new ways to implement AI? Will AI research and implement ways to optimize itself across the whole stack?

Are you just going to let AI spin VMs for you in your multi-cloud Azure subscription with uncapped spend, if that's what it takes to build your product? Maybe AI will also investigate customer escalations and bug reports? Or memory corruption issues?

Look, I get it. You're excited about a computer computing. And maybe you've gotten used to the idea that engineering ends with being a code-monkey stitching together CRUD APIs in Python.

Nothing wrong with being excited. But please, have a bit of respect for the craft.

> In all seriousness now - who is going to be building this "AI"? What about the platforms it has to run on (both software and hardware)? The tools that team has to use?

as I said "low level generic work is on the chopping block"

I think the hard part is where the customer doesn't really know everything about their business process: it's extremely common to have a partial view of a complex system. The process of requirements discovery involves asking questions, pointing out inconsistencies, and setting priorities. Only at that point do you sort of have an agreement about the desired work and even then there are gaps that need iteration.

Wait until AI also becomes product manager and the client.

So it sounds like Their complaining about the UI of Copilot -Doesn’t seem to be a complaint about current capabilities of AI

Hmm https://youtu.be/PdFB7q89_3U

It's like focusing on building very fast transportation through a vacuum (which may eventually work after sinking an absurd amount of money) instead of creating fast train infrastructure which we know is possible and works. It looks like a moonshot but in reality is just a pipe dream and a waste of resources (imo, with today's technology anyway).

The linked video seems like most comedy - optimized for laughs instead of describing reality, I don't think it is relevant.

Imagine that you are a CEO of Microsoft. Someone in your company invented an AI tool that makes an average developer much better at coding (more productive, etc.). As a reminder: you company is a software company that itself employs tens of thousand of developers. What would you do?

a) Downplay the tool. Use it internally to completely crush competition by writing amazing software. License for exorbitant price to a select few partners, who will also crush their competition.

b) Hype it up as much as possible and license it to anyone who's willing to pay $10/month.

I think the rational choice is obvious, which also makes it blatantly obvious that Copilot is not seen as any kind of competitive advantage by Microsoft itself.

That logic suggests Microsoft should simply never sell development tools, or even productivity software, because they would presumably be able to leverage the advantage of those tools internally in their own interests to crush all competitors.

Microsoft has long taken the position that its interests are served by there just being more computers being used by more people to do more things (‘a computer on every desk’ etc) - not by monopolizing all computer activity. That’s more IBM’s style.

Why I’d it better to ship it than keep it?

If you have a widget that makes software developers 10% more efficient, sure you can use it to make your own developers more efficient, but if your developers currently account for 1% of all the programming productivity on the planet you just realized a value equivalent to 0.1% of global programming potential.

On the other hand if you ship it as a product in exchange for, say, 10% of the increase in value produced, you contributed a value increase to the world equivalent to 9.9% of programming output, and captured .99% of the value of the market for yourself. That’s much more than the value you could have captured yourself and you grew the market meaning you can extract even more value on the next go around.

>That logic suggests Microsoft should simply never sell development tools

You employ faulty economic reasoning here. Most MS tools are not unique and aren't even best in their class. In cases where they are close, MS charges a lot of money for the privilege to use them.

To put it in simple terms, if Copilot worked as advertised it would cost significantly more than VS enterprise. At some point of usefulness the logic related to selling vs using it internally would work exactly like I described. This is why many companies develop their own software and don't sell it at all even if they could charge for it.

My company develops software for internal use and we've even been asked about selling it by some customers who have seen it. We don't sell it because doing so would require a ton of work to decouple it from our other internal services/processes and generalize it enough to maybe be actually useable by other enterprises, would give us a whole second set of customers in totally different industries from that of our main product, and would generally be a total pain in the ass. I suspect that there's a lot more of that sort of thing than the other.

Isn’t Microsoft the licensee though?

https://blogs.microsoft.com/blog/2020/09/22/microsoft-teams-...

That's not how large companies works.

Microsoft is way too big to be able to make finely grained strategic decisions like that.

If a team within Microsoft want to ship something hard enough - and they can get sponsorship from the right execs (MS has thousands of execs) - they'll ship it.

Ah, the AI versus advanced automation debate and the Lighthill Report (UK, 1973).

[1] Report: http://www.chilton-computing.org.uk/inf/literature/reports/l...

[2] Video of the public debate: https://www.youtube.com/watch?v=03p2CADwGF8&t

AI is going to rule this world.

You cannot serialize your brain and transmit it around the world in 5 seconds, but AI will.

What a human can do is encode thoughts into words, and transmit them to people one word at a time with the hope they will understand them, but it's incredibly inefficient.

Humans have infinitely shitty and limited I/O. AI will have any I/O it wants.

AI won't need to encode their thoughts into words. They will be able to send a neural circuit directly to their peers over the network.

The handicap will be so absurd that we will be their pets.

“If an AI lion could talk, another AI could not understand him” - AI Wittgenstein

Copilot doesn't remove the thinking process for large problems. But it helps me with dozens of small things. Like adding log lines, variable names etc.

If you have AI generating code, then you'll have to:

1) have the software requirements codified to a near-coded state already

2) have a validation/verification test suite that validates the software generated works correctly

Both of which increase the overall complexity to the point that the AI will be of minimal value-add.

The AI can also write the tests and docs.

It can't write the tests, the tests are where you validate the AI code actually does what you want it to. That is part of the AI-human interface.

The input specification isn't the documentation. Requirements are problem specification/description, AIs can't read mind.

AI generated documentation is an interesting idea. Like unit tests for AI code, it would still require perusal/editing.

We'll get there :)

My point is, maybe, but at what cost? I like the other poster's analogy to transportation systems. While ambition should be encouraged, there's work to do, so much so when it comes to exploiting our new found software powers.

yes, there's a lot of work to do. The cost is me, rolling up my sleeves, and adding entry points to the systems that I build to take advantage of it.

I don't think people are just sitting around. From my observations, there's a mad race happening right now.

What’s the mad race that you’re seeing?

AI apps popping up everywhere, hugging face & google collab growth exploding, Nvidia going full tilt on AI

Stopped reading once the author started saying CoPilot is not useful.

Apparently you stopped reading, but since I never claimed CoPilot is not useful, it wasn't because of that. My claim is, I think, more subtle. The tools of AI/ML are useful, and can solve many problems. But they can't solve all problems, and specifically, they can't solve many of the problems (well) they are currently targeting.

Let a thousand flowers bloom, it doesn't matter "they can't solve all problems". There's enough space for traditional approaches, nobody's forcing anyone to use a neural net. Copilot is just a good code neural net, it's unique in its own way, but it's not even state of the art for a long time.

CoPilot helps me code (mostly Rails full stack stuff) a lot faster than ever, which is no small feat considering I've got 30 plus years of daily programming experience, half of that in Rails. Occasionally CoPilot produces output that is nothing short of miraculous, stunning even. Suggesting that it's a faulty area of research, that the creators of CoPilot were somehow overreaching (if that's indeed what you're saying) is imbecilic.

What's imbecilic is generalizing your own experience to others and their standards. I am yet to see current iteration copilot producing non-trivial code. But maybe "miraculous" for you means something else.

For me, miraculous is writing "trivial code" for me with a single key press where I would otherwise have needed to type out several lines of code by hand.

Copilot does that all the time. It's the best typing assistant I've ever used. And that's valuable to me.

Then I hope you described IDE autocomplete as "miraculous" as well. It's certainly saving me more time than copilot, that's for sure.

Reply on news.ycombinator.com