Thinking this was a trick question I excitedly explained how stupid it would be to build and train a network to approximate a function which could easily be precisely described with a tiny circuit or code statement.
The professor was not amused. He said that's what he would have done. After a few similar incidents he concluded the exam giving me an 8/10 saying my answers were perfectly correct but he didn't like my attitude.
“Something something, it’s better to spend training time on the parts of the network where we don’t know the function beforehand than to train a subnetwork to do a function that we do know exactly at the outset.” and still you need to be ready to be asked about how to backpropogate through your hard coded function.
You can argue that since the professor understood what was being said, the language shouldn't matter, but it does. If again you don't use the correct language you risk offending the listener so much he can't get past that. After all you are dealing with humans, not machines, and in either case you are responsible for clear communication.
However, life is very rarely optimal or fair. The only effect we can productively have on the world is through how we approach it, instead of focusing energy on how our actions are interpreted. It is unfortunate, but that is the most logical outcome. Focus on yourself, not on others.
I usually get along well with teachers and have enough tact to make do when I don't. But this person was such an extreme case. He was always grumpy, would regularly take several minutes out of lectures to call out and belittle any student who was late or he didn't think was paying attention. The only times he wasn't sour was when he was describing unethical and torturous experiments on animals. Then he was overly excited and giddy.
The exam was, as was common at that university, essentially an essay which was then reviewed by the examiner who would ask for any clarifications or push with further questions. This exam asked us to describe a system solving a fairly standard AI task using neural networks. My answer was a few pages split into a few sections: a brief overview of the whole system, followed by detailed descriptions of each part. Wherever I skipped details in the overview I'd written "(see p. X)". It took a bit of planning to get those numbers in there when writing that by hand with limited time.
When I finally gave the answer the professor skimmed through the overview and then proceeded to berate me for having neglected to write about some important detail. I politely pointed out that it was in the following section and that I'd included a page reference for things which I referred to before defining. He grumbled and kept skimming. After a while he complained about another thing he thought I really should have explained which was missing. I told him on which page he could find it and politely reminded him that he was reading the overview and that the details were in the next part.
This went on for a while and I got a bit more annoyed and a bit less polite each time, because it was getting ridiculous. After the tenth or so time I flat out told him that if he'd read past the overview, like I kept telling him, he'd find all the details there and if he didn't want to I'd be willing to describe any part of it orally but really, it is all there if he would just read it instead of complaining.
That's when he asked the question about a third network and I, still very frustrated, was relieved to get an actual question, and I was certain it was a trick question trying to get me to say I didn't think of it and then he'd call me an idiot and explain why it was unnecessary. I was really surprised it wasn't a trick and frankly a bit delighted at inadvertently insulting him.
When I told my wife (well, girlfriend at the time) she was a bit shocked that I'd been rude to a professor, but I didn't need those points and while it was very out of character for me I never regretted it.
I hope to live long enough to be mostly writing tests for a gloriously hacky code generator that gets it right 80% of the time
The fact that it’s not released (and also the fact that GPT-3 etc are still not publicly available) makes me suspect that these models are far too unstable for actual production use. It’s also why I’m getting a bit tired of these overhyped cherry-picked samples with seemingly nothing solid to ever back it up.
I think we can get models about 1/100th the size for general use. That's also the main reason Google is developing TPUs.
I don't buy this. OpenAI literally released pricing for GPT-3, so either they grossly miscalculated their cost base (unlikely) or there's some scaling/instability/resourcing issue preventing them from doing so (much more likely).
I think it's telling that they spent the last 6 months on yet another flashy demo (DALL-E) rather than actually productionizing GPT-3. It just feels like constant smoke and mirrors.
Most of the times a "fantastic GPT-3 result" is shown, you have to dig a bit and then you'll find out how it was primed[0] and how many different texts they had it generate. Then the one(s) carrying out the experiment go on and pick the most shocking writings. If you read all of the outputs (there are a few articles around that show you 5 or 6 different outputs) you'll see the variations that it took duing those. I understand that 5 or 6 is actually small, to get shocking results they usually go into de dozens of tries.
[0] usually the priming phrases are given, but depending on how much of a snake-oil-salesman the person writing/giving a talk is, they may even hide this part
That is, there is no relation between human brains and artificial neural networks, other than them serving similar purposes in particular environments.
Unless you are a dualist, I would say that it's reasonable to view that it is in principle possible to produce an artificial network accurately emulating the function and behavior of a human brain.
If you are a dualist, then there is no further discussion to be had as we are very unlikely to ever be able to prove anything like the existence of a soul.
Apologies if you were speaking to some more subtle nuance that I was unable to pick up.
> I would say that it's reasonable to view that it is in principle possible to produce an artificial network accurately emulating the function and behavior of a human brain.
I think that claim is far too strong. As a non-dualist, I do believe that it is possible to create an artificial "brain" that has the same cognition as a human brain. However, simply rejecting dualism does not tell you anything about what the artificial brain has to be.
You can further say that a non-dualist who accepts the Church-Turing thesis must accept that there must exist a Turing machine which has the same cognition as the human brain. Since the PCs we use are Turing machines, it follows that we should be able to program one to behave like a human brain, in theory at least (disregarding hardware requirements, of course).
Still, that does not mean that a brain Turing machine has to look anything like a neural network trained through gradient descent & back propagation. This was my point: artificial neural networks and the methods we use to train them have no resemblance to the human brain, and there is no reason to believe that they are the way to create an artificial general intelligence. So, there is no reason to be surprised that a neural network, especially one as small as any of the ones we have realized so far, doesn't exhibit complex properties of the human brain.
Artificial neural networks are just a statistical model that was once inspired by a very, very simplistic idea of what biological neural networks are. As we have discovered more about biological neural networks, we've abandoned any notion of comparing ANNs with biological neural networks in terms of actual structure.
This is all not to say that it's impossible for a complex enough ANN to actually be an AGI. It's just not going to be that surprising if it won't be, if an AGI program will look significantly different, and will be trained in completely different ways.
It's probably layered on top of fuzzier tasks we're better at.
Maybe a sufficiently advanced iteration of GPT could do deterministic tasks as slowly and unreliably as we can.
Only every time I read something similar, I think "surely no programmer could think this". Are you a programmer?
Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things.
That considered, I had a pretty strict qualifier in my above post which means the machine must perform better than the average human in the respective task and therefore I'd be more likely to die driving my own car than a machine meeting my prerequisites.
Humans are much, much, much more capable than the absolute state-of-the-art robots when it comes to doing things in an uncontrolled environment.
Would you prefer your pilots to fly your plane with no AI assistance?
First of all, when you actually understand a self-driving car stack, you'll realize those super-human reflexes are more human than you think. The stack is complicated and not only are there delays to be expected, some hardware syncing requirements guarantee certain delays in the perception pipeline. It's still better than a person, but it's nothing close to approaching instantaneous. Likewise, sensors can get dirty, and blah blah blah there are other weaknesses robots have that humans don't. My point is simple: robots aren't perfect. In fact, they are almost always much worse than most people realize.
> will make it less likely to get into an uncontrolled environment
You're misunderstanding me. I'm not saying less likely to get into an accident. I'm saying the world, where cars drive, is an uncontrolled environment - and the current state of robotics is such that humans are better for doing things in the real world. There is no "less likely to get into an uncontrolled environment" because by definition you are always putting it into that situation.
> Would you prefer your pilots to fly your plane with no AI assistance?
AI assistance is fine. AI replacement is not.
There is nothing that remotely resembles AI in the cockpit of any current airliner. All flight control logic including autopilot, autothrottle, TCAS & GPWS, ILS & autoland, and so on are based on simple feedback loops and programming techniques that go back decades.
Have to read the whole comment before replying. You can't just grab individual statements out of an entire argument and choose to go after those. I mean you can, but you can't expect someone to actually engage you then.
There are many implementations that can fulfill a set of requirements. Not all of them are created equal. The ways in which they behave as the system changes can be wildly different. Well-written code will be able to handle those changes gracefully. Poorly-written code may end up proving brittle and bug-prone. Generated code will be completely unpredictable.
Imagine you're trying to build a street network for a city. Some designs are much more predictable than others. If you've played Factorio, the distinction between a spaghetti base and one that has some design is abundant. Even if they currently fulfill the same requirements now, the ability to improve upon and reason about how it will behave after changes is vastly different.
"Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers."
Point to the part you're arguing against because you way extrapolated what "have usefulness" means I think.
Not only that, AI would also have to learn the principles of system design, performance, security, readability, maintainability. That's what makes "good" software. It's a far stretch to say that AI could achieve anything of the sort based on current abilities.
I also disagree with the premise that code generation by AI will be very useful for programmers, for the reasons stated above.
We are already at the point of useful and context aware code generation anyway which is why I've found everyone on this thread questioning it to be kind of funny, Microsoft was demoing complex generations a year ago. So we're well on our way.
https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb?t=29m19s
https://www.infoworld.com/article/3518429/jetbrains-taps-mac...