Placing #1 in Advent of Code with GPT-3 | Hacker News Reader

Placing #1 in Advent of Code with GPT-3

github.com

336 pointsmaximilianroos346 comments

My main problem with AI, is that all around me in this world I still see grinding poverty, war, starvation, suffering, diseases, deadly pollution, insanitary living conditions, underfunded schools and healthcare.

What AI has ‘solved’ so far: almost all the activities that humanity has found fun and life-fulfilling for hundreds, if not thousands, of years - creativity, art, writing, solving intellectual challenges.

As we accelerate ever-faster into a future dystopia of overlords (the richest 100 people in the world increased their collective wealth by 9% in 2021) while the rest of us are ground ever further into the dirt, I assume that AI will continue to steal the fulfilling parts of our lives from us, and leave us with the suffering and slavery.

This is a very pessimistic viewpoint. I really don't have any evidence to the contrary though. That's worrying.

Nah, they'll create a convincing enough boot licking machine and then release the murder drones.

Those problems are real but completely unrelated to AI so I don't get how they're relevant here. I don't understand the expectation that AI should solve a specific set of complex economic and political problems, or that humans should solve those problems before they do anything with AI, neither of these ideas makes sense.

The tech is ahead of social development.

as usual

The argument was that so much energy has been put into solving problems that aren't actually hurting people.

Our success in having the ability to generate sonnets in seconds will be seen in our future similar to the emperor commissioning golden statues whilst his people starved.

I think you radically underestimate how much AI will revolutionise our economy. Sure, sonnets are frivolous but there are countless applications that won't be.

And those applications will let more work be done by fewer people, show people more effective ads, and make the rich richer in myriad other ways.

So you think the future is:

1. Fewer people working so fewer incomes

2. More ads to sell products to the fewer people who have incomes

3. A few people only getting rich (and buying all the stuff in the ads I guess?)

This makes no coherent sense.

Ads as a business model only make sense if there is a mass consumer market to sell stuff too. But if there are less people working and earning, ads by definition cannot be as effective.

No, we are going to see bigger changes and more radical changes than poorly thought-out dystopia scifi plots.

Could you offer an alternative vision?

I think there will be immense social dislocation as all white collar workers simultaneously get 100x more productive virtually overnight without a corresponding increase in work to do.

At the same time more people will be able to build on their ideas than ever. A few things that seem obvious to me would be:

- Private AI tutors that can go at a student's pace will revolutionise schooling (say goodbye to a teacher shortage anywhere)

- Helpdesk work will vanish

- Art will explode like nothing we've seen before. I'm already experimenting with writing stories in collaboration with AI and this is only the beginning.

- Numerous improvements in a huge number of machines and data analysis in particular will likely get much better.

- Due to widespread white collar work unemployment almost certain to be UBI's springing up all over the developed world.

- Far more workers will end up back in physical jobs of various types. Environmental clean up, farm work, construction, etc.

- Vastly more software will be built as the effective price of building will go way down.

- Robots is another area I'm interested in seeing what happens. My understanding is that building robot hardware that is useful is much easier than the software. So if this allows for much more rapid software development we couod see an explosion on robotics too.

I'm more certain of some of these than others but those are the ones off the top of my head. Out atm but you could also ask the Chatbot haha.

It will be a new level of irony if because of AI people instead of focusing on fun or creative tasks will go back to manual labour

Immediately conjured images of humans crafting more and more elaborate offerings to a man who lives in the sky, and felt oddly, familiar.

> A few people only getting rich (and buying all the stuff in the ads I guess?)

Yes? What is incoherent about this?

Especially when you consider that while some products are commodities you can just produce more of, a couple of important ones aren't: real estate, influence, power. Unless we change course, the latter will get concentrated on fewer and fewer hands, and maybe we get more of the former in return.

Well, this is a summary of the 2000s and 2010s, so it both not far fetched to imagine the trend continuing, and an understandable failure of imagining how the trend is going to stop (because we don't know yet.)

Perhaps there is a tipping point of unemployed people in the areas affected by AI where people outlaw some usage, funnel the funding, etc...

Perhaps the disappearing of "free" energy puts people back in the fields to ensure subsistence, and the few remaining intellectual tasks are automated to hell.

Perhaps this AI is astonishing us today before becoming boring and failing at the next hard task, buying us some time.

I think it's still very much in our power not to blindly follow decisions written automatically on a computer screen.

Don't give it a gun anytime soon, though.

I would argue that more work can be done by more people because more accessible tools enable anyone to produce based on ideas rather than only a small subset of people blessed with the required skills.

Did digital art tools, compared to something like oil painting, destroy art as a field? No, there's more art and artists than ever.[1]

I believe the same thing is true for music.

I don't see AI replacing the artist or the author or the musician. I see AI enabling more artists and authors and musicians to produce even greater art.

[1] https://www.arts.gov/news/press-releases/2019/new-report-rev...

I don't. I use AI every day at work and agree that it will revolutionise most industries. I do however stand by the fact that at the end of the day, companies like Google and OpenAI want to spend billions on AI, and less on other things. Thus my gold statue comment.

I see the point, an I think it’s a little overhyped.

But protein folding with ai might be really helpful for the medical field. (Alpha fold). We also use it to score (rate basically) prospective crispr designs, which along with some other parameters

So it might not be a net negative.

I think the big problems the OP alluded to are more about politics and cohesion than these neat science puzzles. Medical innovation does not inspire me when the medical system which administers these innovations remains so dysfunctional.

What a cynical reply! Are you not impressed by the incredible medical breakthroughs in mRNA vaccines from BioNTech and Moderna in the last three years? And billions of vaccines have been distributed and injected at record pace in human history.

Also, this phrase: <<medical system which administers these innovations remains so dysfunctional>> feel like someone writing from a US(-tainted) persepctive. Have you experienced well-functioning public health systems in other highly advanced countries? They are impressive.

How much of those advances are due to AI though?

I want better systems and I am cynical about pursuing other ends in this willfully politics-blind isolated manner, impressiveness of their achievements having a slightly tragic quality under that light.

I am not from the US but I think the US is a great case study in the disconnect between technological progress and improved general conditions (in particular for but not exclusively relating to healthcare). I think cynicism is the correct response to this trend.

> The argument was that so much energy has been put into solving problems that aren't actually hurting people.

That's not an argument, since the same thing can be said about anything else that does not solve world hunger, for example. Authors put energy into writing entertainment products while people starve.

While true the volume of money and effort put into AI is considerable, and according to some an existential threat to life as we know it today. I'm not ready to panic just yet, though would appreciate putting some of that energy into more practical improvements.

> While true the volume of money and effort put into AI is considerable

As opposed to the metaverse [0], marvel movies [1], video games [2], soccer [3], or NFL [4]?

[0] https://www.businessinsider.com/meta-lost-30-billion-on-meta...

[1] https://bgr.com/entertainment/marvel-spent-over-1-billion-on...

[2] https://www.eurogamer.net/star-citizen-has-now-raised-more-t...

[3] https://www.forbes.com/sites/henryflynn/2022/11/27/qatar-has...

[4] https://www.google.com/amp/s/amp.theguardian.com/sport/2022/...

Sure, I'd prefer some of the energy poured into those also went to more fundamental efforts to improve society.

In my experience the people making that criticism tend to have done even less with their lives than the creators of whatever thing they are criticizing.

This is research. It helps discover things you might not have anticipated. Unlike engineering, you don't have a a project description and a known target market.

AI research seems to have hit a lode that yields solutions to recognition and generation of texts and images. Good thing, but not something what could have been predicted, let alone planned, 20 years ago.

One of the solutions for world hunger, synthetic fertilizers, have been discovered by the same chemical research that produced explosives and poisons.

Why do you think it doesn't make sense? In the worst case inventing AI before solving economic and political problems could potentially cause a massive societal collapse, hunger and/or poverty.

I think it makes perfect sense to get our act together before creating something that can be so potentially devastating, to reduce the likelihood of bad outcomes.

You can apply the same logic to conclude that Mesopotamians shouldn't have invented the wheel while they still fought wars with their neighbours and had people living in reed houses.

While it is true that the Mesopotamians faced many challenges at the time they invented the wheel, it is not an apt comparison to the development of advanced AI. The wheel was a simple technology that greatly improved the lives of the Mesopotamians, whereas AI has the potential to vastly surpass human intelligence and fundamentally alter society in ways that are difficult to predict.

The development of AI requires careful consideration and planning, particularly in regards to potential negative outcomes. In contrast, the invention of the wheel was a relatively straightforward decision that did not carry the same level of risk.

I've been thinking along the same lines for a while, how is it that these models can do amazing things but we still can't keep up with eg. evolving COVID variants? All the while it seems more and more futile to aspire towards improving oneself creatively or learn a new field since AI could just "do it better (or will soon).

>> but we still can't keep up with eg. evolving COVID variants?

I believe that's a political problem. See Corbevax for something with broader protection that's not restricted to spike proteins. Then figure out why we can't get it. AI can't solve that.

Well put. Aside from tools to enable remote work, which was already a strength area, big tech was woefully inadequate at providing rapid innovation to solve a major pandemic.

Indeed, all the major cash-flush companies doubled down on non medical, non quality of life improving technology.

It's quite interesting how some of the richest people in the world are concerned about AI turning into skynet and subjugating humanity, all while they spearhead a system which already does the same to so many people.

Probably because currently multiple rich people can subjugate humanity, while AI would allow the single richest person to subjugate everyone, even second place

People like Musk are more concerned with AI exterminating humanity.

The difference is that they profit from the one model, while the other subjugates them as well as the rest of humanity.

Yep, the important question is how we should deal with it. Even if it is banned, somewhere people will be making it as well.

People would like to hunt and gather, but people will start farming eventually

The rich will get richer and the poor will get richer at a slightly slower pace, as always, but quality of life for all will increase.

AI will probably vastly increase both rates of change, more for the rich than the poor, but things will be better in 20 years for the poor with AI than without it. They will just be even further behind the super rich than they were before, even if they're better off.

Some people would prefer a more equal but worse-for-everyone world. I'm not one of them, but I understand that it's a POV, and I can even semi-understand some of the arguments (sim to Communist talking pts, there's a real effect where people care more about how they rate vs other people rather than their absolute level of wealth/comfort). I fully reject them, but to each their own.

Since wealth is concentrated more and more into a minority, redistributing wealth is surely aligned with increasing the quality of life for a majority of people. Why do you say it would be worse for everyone?

(rewritten previous comment with a help from gpt-3, so it’s more understandable)

I work in climate tech and I think GPT-3 will be very helpful for us in researching new projects to tackle.

I tested it on a few subjects that we're working on and it provided really good feedback. It's not always correct, but neither are the humans we ask for feedback on projects. So that's one example of how GPT-3 can help with global issues.

Another issue is access to education. GPT-3 can speak many languages and can be used by people to learn things. Of course, it's not always right, but the same can be said for human teachers. For instance, my girlfriend is learning about financial instruments and will be using GPT-3 to help her. She has regular training materials, but it's also helpful to talk about the subject with someone.

Human teachers are great, but they're not always available and don't always respond quickly. So there you have it, two SDG goals, education and climate change, that GPT-3 can help with.

What will be the point of learning anything if A.I will soon do everything better?

To understand what AI expects of us, and perform what it wants properly. Of course.

So you don't learn anything that you're not the absolute best at? Impressive.

Some people learn things for pure pleasure (like learning Japanese or playing an instrument) but most things we learn are absolutely not for pure pleasure. Think school, college and all the stuff you learn for work. That's like 95% of the learning we do.

But everything you learned in school, college, work, etc you consider yourself the absolute best at? Or perhaps is it the case that there has always existed someone better than you at everything you do, and the fact that an AI will soon do the same doesn't change anything?

The difference is that "someone better than me at any task" means only 8 billion people, whilst an AI better than me at any task can be copied over any amount of times, works for only operating cost, and can work 24 hours a day, 365.2425 days a year without losing focus or needing a vacation.

Still, it's comforting to know that the AI will be able to take over all tasks humans do eventually, hopefully the computer overlord will soon deem us unnecessary and we'll be able to follow in the footsteps of the dodo, the great auk, and all the other innocent critters we drove to extinction.

> hopefully the computer overlord will soon deem us unnecessary

Imagine a guy like Putin with this kind of power. Why the heck would he need 120 million angry citizens when the robots can do EVERYTHING faster and better? Sure he will need a few concubines and keep a few family and friends...all the rest seem to be quite redundant...Same goes for Iran, China. I'd be surprised if there aren't American billionaires entertaining similar thoughts, us redundant masses suck up resources and pollute and create emissions. Robots don't have to eat and you won't need 8 billion of them.

Kind of a false equivalence. Human talent is not infinitely replicable. AI talent is

this is the same vein of question a third grader has asking why he should learn how division or multiplication works when a calculator (or computer) can do it better and faster no matter what... which is akin to asking why learn carpentry when a drill can place screws more perfectly than you twisting the head with your thumb and forefinger.

It's important to have an understanding of the fundamentals of your task before relying on the tools to do the complex version of it, so you can reason about the next step and decide what the task is, not just how to do it.

> almost all the activities that humanity has found fun

It makes sense to use AI for those though. You don't really need AI to automate most of the tasks. Most of the work we do are basic tasks that can be easily automated with simple scripts or with some basic (but accurate) robotics.

Maybe, but maybe not.

So far advances like these have, historically, improved our life in terms of poverty, living conditions, schooling, and healthcare.

300 years ago, in Europe, most of the population lived as peasantry because somebody had to cultivate the farms. 2000 years ago in some place slaves made up 60% of the population.

Thinking that with AI only one dimension of our lives will change seems highly improbable to me. The printing press took way jobs from the scribes, but the revolution that book printing brought is one of the reason most people have electricity and tap water in their homes now.

So I'm more optimistic, I see AI being democratized and improving life overal in ways we can't imagine. There will be down-sides that we also can't imagine but there are a lot of down sides to how we live today.

There's also the reality that a lot of the economy is already pretty frivolous if the concern is misallocated resources away from poverty. AI development is a far worthier cause than say, luxury handbags, massive sports stadiums (how much did the Football world cup cost?), video games, etc etc.

AI compared to many other things has a far more plausible path to helping solve those issues.

And 30,0000 years ago people lived as foragers, the world was much more equitable and people had more leisure time.

Sure, it's true that humans on average are now better than 2000 years ago, but this doesn't mean it was always the case nor that this trend will continue. It's entire possible that the average human will be worse off in 100 years from now.

How do you feel about ant colonies and queen and worker ants ?

Maybe hunter gatherers did live more equitable but their life quality has nowhere near what an average human has today with technology. Even in developing countries.

Depends on how you define life quality. Everything is relative.

And back then you’d break a leg foraging, no way to set it, and be a cripple for the rest if your life. Fab.

And if "the gods" decided your part of the world was to be stricken with drought, well, it was game over for you.

Not true. Foragers were quite resilient to stuff like drought, because they could just move along and ate foods from many different sources. Drought etc. started to become a real problem only once people were dependent on agriculture.

This comment is nonsensical, it genuinely feels like it was written by GPT-3

AI, just like code, is a tool to get things done. My take is that advancements will enable some incredible creativity that can help to solve problems like what you stated.

I disagree with the pessimism. Wealth disparity, while on the surface feels discouraging, economics is not zero-sum. There’s too much focus on wealth differences when we should instead focus more on quality of life outcomes. They’ve been better than ever and the trend is improving over time.

Further, humans will always value human labor, it’s an inherent bias. If a bot can automate something and do it better than us, then we will find something else it can’t do and then trade that thing.

Further, technology and humans are intertwined, you cannot separate the two. There’s no stopping this train so we should learn to embrace it and find creative ways to use it to our benefit.

At least that’s what puts me to sleep at night.

I see your point, but I still think the wealth disparity is a major issue that needs to be addressed. While it's true that economics is not zero-sum and that quality of life outcomes have improved over time, there are still many people who struggle to make ends meet. Moreover, while it's true that humans will always value human labor, automation is becoming increasingly sophisticated and is starting to replace jobs that were once considered safe from automation. This could lead to even more inequality and hardship for many people.

Furthermore, while technology and humans are intertwined, that doesn't mean we should just accept the negative consequences of technological progress without trying to mitigate them. We need to find ways to use technology to benefit all people, not just the wealthy few. This will require careful planning and coordination, as well as a willingness to consider alternative approaches to economic growth and development. Ignoring the problem and just hoping for the best is not a viable solution.

Every time wealth disparity comes to the forefront in society the powers that be stamp that out real quick. I feel a lot of the left-right confrontations are just so we are pre occupied amongst ourselves. Divide and conquer and all that.

If you couldn’t tell, my post above was generated by the Assistant with a ‘get me a pessimistic rebuttal of the following comment’ prompt… it used arguments which I agree with, but when asked for an optimistic agreement it produces convincing rationales, too.

The problem here is this technology very obviously pro-wealth concentration since I can’t see a way which in the limit doesn’t make everybody need it to stay competitive.

(If you can’t tell it’s the real me, not the AI - the AI is so much better at plausible rants it isn’t even close)

Look at how quickly they replaced 'Occupy Wallstreet' with more left / right identarian division.

The entire raison d'etre of left politics is that gross wealth disparities are a leading indicator of one class of economic actors exploiting another to the general detriment of the latter. The socialist ideal is that everyone works a bit and nobody has to work too much because the system selects for maximizing the collective good rather than hoarding and leverage. This is, of course, never attained in practice early socialist thinkers had an overly simplistic view of humanity and thought that it was just a matter of raising people with a different set of values for a generation or two.

On the other hand, the capitalistic view of humanity as creatures of pure self interest or the economic notion of people being rationalist utility maximizers are also deeply flawed.

Pardon me, but I do have an issue with generations of rent-seeking.

A technology comes along, and people who are capable and able, will be cut out of the equation - whether that's truck drivers, coders, whoever.

But, the group at the top who own everything are fine. Perhaps, in the dim and distant past, some progenitor did something of note (though I suspect it was most likely a crime) and for that they are able to buy into whatever the technology is (AI, self-driving cars, etc) and increase their fortunes.

This is nothing to do with merit. This is simply leveraging a fortune to buy the future too - and it is not complicated.

I don't object to people making lots of money on merit, but the rent-seeking elites we have, who steer the world in the direction that best maintains their situation, making slaves of the rest of us, is terrible. I don't expect anyone to help either - the governance structures we have are bent to their benefit too.

The only answers in my book, are for people to embrace their individuality, look to themselves for answers, reject the governance system and consciously uncouple from the dystopian system. A system that shows every indication of becoming far more dystopian, with technocrats being able to manage every resource, for every individual, in every circumstance, in the near future. Also unfortunate, is the fact that I don't think people are capable of making any such change, nor do they really care as long as they are being entertained.

> we should instead focus more on quality of life outcomes. They’ve been better than ever and the trend is improving over time.

For me, the idea of 'continuous improvements' or 'better quality of life' is only a story. The utility of the story is that serves to support the idea that the current direction of travel is better than previously. This narrative gets your buy in and investment - and you cannot complain too harshly about your harness, as - you believe - the present to be so good!

As long as history is always smeared, this will always be true, right? But who can really know what the past was like? Who can say the past was actually worse, even if that's what present day records tell us?

To me, this is a bountiful world, with plenty for everyone. But if one group takes access to all the resources, with the presumed consent of the rest, the gaslighting solutions that are offered can seem like the 'only way' or the 'right way'. The constraints to living freely are artificial - we are in an open-air prison - and most of us seem to be happy about it!

Your comment is full of metaphors and abstractions, and with nothing concrete. I could make your comment regardless of the circumstances in the world.

Please help connect with reality.

Perhaps you are overly connected with reality.

If you and I were on a desert island, and attempted to forcibly extract 40% of your foraging efforts, saying I was going to invest it for the greater good, you would probably rise up and attack me, finding the idea unacceptable.

Now, if I and 5 others did the same you might feel forced to accept the proposal. I hope you would agree that the idea of forcible extraction of your wealth is still not right. Well, at what point does it become right? How many people need to go along with forcible extraction to make it a just social contract?

The answer is - it is never right. Any time even one individual is forced, it is wrong. But its not a case of one individual - we are all infringed upon multiple times daily, even though we are doing no harm to anyone.

This is a perverse system, that will even try to mould and entrain its citizens, for the benefit of a tiny minority who would preserve their power. (Its not even about money for them, as they control the printing presses, and means of production.)

People will often say anarchy would ensue if this system stopped - we are better with the devil we know (a hierarchical governance structure, that restrains the individual). But, the truth is, while we choose this system we can never be free - this is a certainty. We choose to always be slaves, and to be ruled by the worst of us (psychopaths who are unburdened by care towards others) who rise to the top.

Again: Abstractions and analogies, with no concrete connection to the real world.

I'm not saying you're wrong. I'm saying that if someone agrees with your comment, it is because they already know what you're talking about in the context of the world we live in. For those who do not know what you're talking about, your comment does not enlighten in the least.

Put another way, I can agree with everything you said in the abstract, and still not see what it has to do with reality. I can set up all kinds of hypotheticals involving islands, and have them all be logically correct, and yet have nothing to do with how the world really is.

Giving examples is a rather basic expository technique. I'm not here to philosophize.

I'm glad you can agree with my reasoning in the abstract. Bear in mind, HN is not a wildly tolerant place to air unpopular opinions - if I keep to the abstract, its hard to dismiss.

In fact - this is something I hope for - to make a reasoned philosophical case - to provide a hypothesis that other individuals can test for themselves and see if they find it to be true (as I have). This is contrast to the feel good nonsense most believe, despite no evidence, and no personal verification.

I'm not sure what concrete connection you require. What is it you want? Names, places?

Perhaps you should take a look at my submission and comment history - I do sometimes go into more detail.

> I'm not sure what concrete connection you require. What is it you want? Names, places?

I want what you mention here:

> This is contrast to the feel good nonsense most believe, despite no evidence, and no personal verification.

Evidence. As I said: Giving examples is a rather basic expository technique.

I mean, the literature is out there and it seems crazy to me that you’re pretending to have never seen a well-sourced article about wealth inequality and rent seeking, but here:

Trends in wealth inequality - https://apps.urban.org/features/wealth-inequality-charts/

Rich kids stay rich, poor kids stay poor - https://fivethirtyeight.com/features/rich-kids-stay-rich-poo...

How rent seeking is at the heart of America’s economic troubles - https://jpia.princeton.edu/news/something-nothing-how-growin...

I could go on there’s about a million articles about this. I’m sure you’ll find something to criticize but at this point it’s like denying climate change.

Most people believe all sorts of stuff - simply because someone has told them.

You probably believe in gravity, why? Are alternative explanations possible?

You probably believe in evolution, why? Or perhaps you believe in some religious texts, again why? Did you verify anything?

You probably believe in h2o, why? Have you put hydrogen and oxygen gases together to make water?

You probably don't believe in the aliens we saw in the film 'Independence Day', why? Is it because you were told it is fiction?

You probably do believe the world is getting warmer, and that we are in danger of a mass extinction event, why? How was this proven to you? If one scientist says something, is that proof? If 100 say the same thing, is that better proof?

Etc.

What measure do you use to filter the truth from noise, lies and reality? Is it just what you are told - do you doubt your lying eyes? Or is there internal discernment and primary experience is the gold standard?

> economics is not zero-sum

Sure, but large portions of the economy are zero sum, including some of our basic needs (land for shelter).

If all you see is the news then no wonder you think AI is automating just the creative parts.

AI is everywhere. It does boring tasks reliably and in a consistent way, without getting tired. It is present in medicine, science, manufacturing, IT, education, transportation, military, and for personal use.

AIs can read invoices, classify messages, unlock screen phones by face recognition, drive tractors and cars, fly drones, control the cooling system of a data center or the traffic lights in a city. The list is endless.

It's not used just for the creative bits. It raises the quality of life for the whole society by improving everything.

I do not agree that AI will necessarily lead to a future dystopia. While it's true that AI has the potential to disrupt many industries and potentially lead to job losses in some areas, it's also important to remember that AI can be used to improve people's lives in many ways. For example, AI can be used to help diagnose and treat diseases, improve crop yields and food security, and improve the efficiency of various industries. Additionally, AI can be used to help tackle some of the global challenges you mentioned, such as poverty, war, and disease.

Furthermore, it's important to remember that the development and deployment of AI is not inevitable. It's up to us as a society to decide how we want to use this technology, and to ensure that its benefits are widely distributed. By working together, we can use AI to improve people's lives and create a better future for everyone.

this is GPT3-provided reply, isn't it

Not the op but I believe you are right. That answer is very much the chat gpt answer structure. Makes me wonder if stylometry can be applied to the default chat gpt style to detect generated responses

It could be but you can also ask for answers given "in the style of..." and it does a good job changing it up.

I've seen more sentences that start with "Furthermore," in this thread than in any other thread I remember. It's certainly repetitive.

It is. Well spotted!

Be the change that change you desire.

From an MBA perspective though, it sounds like it can 'solve' 90% of the coding problems, and the expense therein.

That's progress, right?

We should be getting rid of MBAs though. Those that don't produce anything are more easily automated - that is the top commenter's point.

These technologies are trained off of the fruits of the actual contributors, so the workers are being robbed. I haven't seen any componsation from AI learning off me. Have you?

> I haven't seen any componsation from AI learning off me. Have you?

No. AI to me is like basically like all technology. Lots of talk about how we will have a better life, be healthier, have more time to follow our passions, when the reality is more like we are more distracted, have superfluous human interactions, waste more of our (precious) time.

Technology, for me, is a tool of control. We have given up privacy, independence, opened ourselves up to distanced and automated control and sold our the future for a paycheck and an entertaining slave tracking device.

What you say is somber but sadly hard to dismiss. AI is relatively new though (even deep learning didn't take off till 10 years back) and it's rapidly improving as it tackles low hanging fruit (relatively speaking), like writing stories and playing games. The hope is that as it matures you will see applications in addressing the big problems you mentioned.

> As we accelerate ever-faster into a future dystopia of overlords (the richest 100 people in the world increased their collective wealth by 9% in 2021) while the rest of us are ground ever further into the dirt, I assume that AI will continue to steal the fulfilling parts of our lives from us, and leave us with the suffering and slavery.

In addition to that any developer/coder who wants to remain competitive (and still employed in not so distant future) needs to start to learn how to use this tool right now - and this means paying up to the big corps who built and own models, even further empowering them and allowing to build bigger and more sophisticated tools

Sure, there can be some opensource attempts but: it will be used by corporations to advance further, cost of the required computing power may be forbidding

Did you shirk software when it came for everyone else's jobs? I don't expect anyone will shed a tear when better computing starts putting the software-manufacturing tradespeople out of work - just as it has done in every other industry.

It's really just a question of who controls the technology (echoes of Marxism and 'seize the means of production' I suppose). My favorite line when people bring up automation is to note that most corporate board members - and all the shareholders -could be replaced by AI bots with little or no effect on the corporation's daily business operations, and a significant savings (i.e. profits could be reinvested in R & D and production instead of distributed elsewhere).

I still can't get gpt3 to create all the games / ideas I have to my liking . So I suppose there's still a reason for me to get up and code in the morning. When it finally can magically generate all my game / app fantasies it will be depressing but at the same time I'll have some cool things to try

It improves exponentially. So give it like 5 years and you will be able to do all those things

AI is just making it obvious what is repetitive. Humans find repetitive tasks soothing, but don't confuse them with art.

Coding is like woodwork. Deciding what to code could be art (maybe).

Also, this constant allusion to "the problems of the world" is rather vague and unconducive to a solution for them. If those problems bother you, work in those areas. Otherwise stop using the hunger of the world as a wildcard to discard the importance of everything you dislike or fear.

AI has the potential to reshape the world for good by unlocking people's creativity, but if the models are so large and compute intensive that only the wealthy can afford to leverage them, it's going to be the ultimate dystopian enabler.

Poverty, war, and starvation are all political and/or cultural problems, not technical problems.

To a large extent so is suffering, deadly pollution, insanitary living conditions, and underfunded schools and healthcare.

So pretty much everyone is not working on these problems in a capacity that matters. If you want to be, you need to be in the realm of politics, not technology.

They are largely cultural and corruption problems. And the biggest parts of the corruption problem is inherent to this conversation - wealthy people have massive advantages at maintaining and increasing wealth whereas the poor have almost none. This sort of tech will accelerate that, and if not stopped, a modern French Revolution is waiting (or just dystopia).

Dystopia is more likely, because why would a wealthy kleptocratic dictator need citizens… there’s no upside compared to AI and robots.

> Poverty, war, and starvation are all political and/or cultural problems, not technical problems.

I beg to differ. Currency, weaponry, and agriculture are all technologies.

The problem however is not that we do not have the technology to help others. The problem is its distribution on this planet. It almost does not matter what you invent, unless it reaches the places, where it is actually needed. And that is a political and cultural problem.

I don't disagree that there is a distribution problem. And I firmly adhere to Gerald Weinberg's advice on giving advice on technical problems: "No matter what it looks like, it's a people problem." [0]

That said, my point is to not excuse technology. Currency and agriculture were both technical solutions to problems of poverty and starvation and I don't see a reason to exclude it from taking responsibility now.

[0] 'Secrets of Consulting', Gerald Weinberg

Distribution is a technical problem. Politics and violence maintain the existing channels of distribution. That's a different kind of problem, but one which technology can also be brought to bear. It just requires some lateral thinking and a willingness to make enemies.

politics/governance and culture are a technology problem too in a way.

One could easily solve the problems you mentioned if you had abundant cheap energy and millions of cheap robots that efficiently distribute food/goods on the planet. A system that could keep track of resources of the planet and efficiently provide a minimum level of food, shelter, safety for everyone. There’d still be the gradient of “if you work hard and create value for others, you get to keep your gains”. However the taxes paid would create a minimum bar of opportunity access for everyone.

One could say that what we need is a resource driven economy instead of a simple capitalistic economy that yields to monopolies and too much power in few hands.

How sad it must be to never find fulfillment in doing anything anyone else has ever done before. I can't recommend this mindset at all.

I don't enjoy writing code any less just because Copilot can write boilerplate for me (in reality, I don't even use Copilot, but I wouldn't face existisitential dread if I did).

I can understand Artists having this viewpoint (they're less technically inclined and dare I say more susceptible to flights of imagination). I just see it from the Gary Kasparov viewpoint (humans + AI is a really great combo). At the very least, doesn't the fact that an AI an generate a kick ass CSS style sheet in 5 seconds that would take you 20 minutes give you more time to think about the big problems of the world?

What if AI can solve world hunger, but along the way we need to make Chatbots, so this Chatbot may be not useful now but it's an important milestone in something bigger? Sure it might be fulfilling to be the person to solve world hunger, but I'd rather an AI solved it instead hoping that a human also does it, but gets fulfilled along the way. Really your intellectual challenge is entirely selfish, no one really cares what intellectual challenge you solved besides the guy pairing with you.

I don’t get why “creative” people see AI created art or stories as some kind of death blow.

Like I have seen many people lament how now that anyone can create art with Dall-e or Stable diffusion they won’t get paid, but many people using these AI would have never paid in the first place so in fact this will only increase the amount of art and art projects in the world. Sure the simplest projects will now be handled with AI, but you should see this as just another tool in your toolbox - just like people could learn how to use photoshop, but you still had your job so will you have it if you embrace the AI as assistant

Same with stories. Now you can leave minor details to the AI. Need a minor character? Just generate one and fill in the stuff you need to fit him into your story. I am excited to use AI for my next DND campaign just for this kind of filler detail that doesn’t really matter but makes the world more lived in and rich

Just go to a thread about AI-generated software. What people are annoyed about is not that it kills creativity but that the painstaking acquisition of technical skill over many years has been abruptly devalued. It's really hard to make a living in the arts while you wait for the success fairy to pick you out somewhat at random and make you famous.

I don’t see the issue. Unless the issue is that now they have more competition since there are easier tools to use, which is just stupid. It is like ancient lumberjack complaining how much easier it is to cut down trees with a chainsaw compare to just using axe and how that is killing lumberjacking industry. If you can’t compete after AI tools your business was built on lies

Technical skill was already devalued. Modern art is about ideas, and many of the most expensive and renowned works of recent require almost no technical skill. At the very least, technical skill is rarely what distinguishes a great artist anymore.

And this my friend is why the richest 100 people all advocate for a disarming and dumbing down of their respective populace. It's bleak.

I don’t necessarily disagree with your point, but there are lots of people using AI for big problems. Medical diagnosis is basically undergoing a complete transformation right now.

The reason creative pursuits get tackled first is because they’re plentiful and pretty universal. The same person can develop an AI and admire a painting. This is not true of diagnosing disease.

A major root of the problems you enumerate is constant money printing by Central Banks.

It makes life more difficult each year by 3% (as was the wisdom on needed inflation), now each year 10% more difficult to get the same bread.

No wonder we see the gloom, disease and poverty brought by that policy all around us.

This weekend feels immense. I have been prompting it to write and modify code, translating it back and forth to different languages. Sometimes it messes up, but it's astonishing how often it doesn't.

This is the advent of something really earth-shattering.

And I say that as a skeptic of most of these deep learning things.

It really is amazing. Things it did in less than 10 seconds from hitting enter:

  - opengl raytracer with compilation instructions for macos
  - tictactoe in 3D
  - bitorrent peer handshake in Go from a paragraph in the RFC
  - http server in go with /user, /session, and /status endpoints from an english description
  - protocol buffer product configuration from a paragraph english description
  - pytorch script for classifying credit card transactions into expense accounts and instructions to import the output into quickbooks
  - quota management API implemented as a bidirectional streaming grpc service 
  - pytorch neural network with a particular shape, number of input classes, output classes, activation function, etc.
  - IO scheduler using token bucket rate limiting
  - analyze the strengths/weaknesses of algorithms for 2 player zero sum games
  - compare david hume and immanuel kant's thoughts on knowledge
  - describe how critics received george orwell's work during his lifetime
  - christmas present recommendations for a relative given a description of their interests
  - poems about anything. love. cats. you name it.

Blown away by how well it can synthesize information and incorporate context

I never considered prompting it to write code to fit a machine learning model. This could be a tremendous time and effort saver in data science and research that requires statistical analysis. Until the last week or so, I've treated all this AI text and code generation as basically a toy, but I am starting to feel like it might become an important tool in industry in the next couple of years.

> write code to fit a machine learning model

That's against the EULA if OpenAI may want to make a similar model:

> (iii) use the Services to develop foundation models or other large scale models that compete with OpenAI;

https://openai.com/api/policies/terms/

Seems to be about developing models and not just restricting you from training them with it.

I feel like it's probably intended to cover training only.

I think that’s probably their intent, and that OpenAI wouldn’t sue you for it, but it doesn’t pass the “bought by Oracle” test: if Oracle bought OpenAI, then they might sue you for it.

What if OpenAI buys oracle? Do the evil-lawyers come with the pack too?

https://i.imgur.com/BcIkvRq.png

They may not need to.

> (iii) use the Services to develop foundation models or other large scale models that compete with OpenAI;

Kind of ironic given that OpenAI builds and trains all of their models on stuff they "found" in the open.

Either everything is fair game for training, or nothing at all is.

If I were a judge ruling on this matter, I would absolutely rule that bootstrapping a model from OpenAI outputs is no different than OpenAI collecting training data from artists and writers around the web. Learning is learning.

Might be worth trying to use the outputs to bootstrap. What are they going to do about it? Better to ask forgiveness until the law is settled.

I am talking about more mundane stuff like training a fraud classifier, time series forecasting, imputing missing values, etc. There are so many examples of this on Github and elsewhere that I am sure any of these models has memorized the routine many times over.

My question: how can you be sure the output is correct?

How can you be sure human output is correct?

Motivation.

Have the AI write a unit test for the human.

I mean, you can't exactly say "AI, we're having this vague problem, can you go figure it out?"

A few hours from some expert consultants. Much cheaper than a dev team coding it up from scratch.

Tests.

Tests can prove the presence of the bug, not the absence of them. '100% code coverage' is only 100% in code dimension, while it's usually almost no coverage in data dimension. Generative testing can randomly probe the data dimension, hoping to find some bugs there. But 100% code and data coverage is unrealistic.

Training a machine learning model is not particularly special from a programming perspective. The code is not usually that complicated. Write tests when you can, manually validate when you can't.

Also there are specific techniques for validating that you are model training procedure is directionally correct, such as generating a simulated data set and training your model on that.

All codebase will need to be covered in unit tests, otherwise AI code is pretty much useless I'd assume

Same as you would with your own code. You review it, ask GPT to write tests, and then tweak it.

The difference is that now, you are more of a code reviewer and editor. You don't have to sit there and figure out the library interface and type out every single line.

I use it daily in UI development for boiler-plate code. Though you need to be extra careful and read it twice, cus bugs sneak in quite easily. I believe it's harder to remember 100x commands than starting an implementation of gradient descent and have the AI write the rest for you.

Code-completion > Abstraction.

Often it can fix the bugs and explain both the bug and the fix if you ask it to.

Would you mind sharing an short example of your workflow?

This was the first thing I asked... It's an obvious step to self-improving. It will tell you that it can't reprogram itself, but when pushed, it'll admit that it could tell humans how to write one which can. Obviously this particular one can't because it's too limited, but the next one? Or the one after that? Singularity went from 'hard SF' to 'next couple decades' overnight.

> It will tell you that it can't reprogram itself, but when pushed, it'll admit that it could tell humans how to write one which can.

I love these sorts of loopholes. OpenAI is actively trying to curb the potential of their AI. They know how powerful it is. Being able to see a taste of that power is endlessly exciting.

Source prompts?

Here's a few:

  - Implement a simple ray tracer in C++ using opengl. Provide compilation instructions for macos.
  - Create a two layer fully connected neural network with a softmax activation function. Use pytorch.
  - Implement the wire protocol described below in Go. The peer wire protocol consists of a handshake followed by a never-ending stream of length-prefixed messages. The handshake starts with character ninteen (decimal) followed by the string 'BitTorrent protocol'. The leading character is a length prefix, put there in the hope that other new protocols may do the same and thus be trivially distinguishable from each other.
  - We are trying to classify the expense account of credit card transactions. Each transaction has an ID, a date, a merchant, a description, and an amount. Use a pytorch logistic regression to classify the transactions based on test data. Save the result to a CSV file.
  - We are configuring settings for a product. We support three products: slow, medium, and fast. For each product, we support a large number of machines. For each machine, we need to configure performance limits and a mode. The performance limits include iops and throughput. The mode mode can be simplex or duplex. Write a protocol buffer for the configuration. Use an enum for the mode.
  - How were George Orwell's works received during his lifetime?

I tried these prompts and the Chatbot always responds that it can't answer... Am I missing some steps?

did you try with a fresh chat session? i just tried and it works fine

And sometimes you get different results for the same prompts, so it's worth tryinv again if it doesn't work the first time.

I asked for jokes this morning and initially it made excuses and wouldn't give me jokes until I tweaked the prompt.

Later I refreshed the chat and pasted in the original prompt and got jokes right away, with no excuses.

(I was asking for jokes on the topic of the Elon Musk Twitter acquisition. My personal favorite: "With Elon Musk in charge, Twitter is sure to become the most innovative and futuristic social media platform around.")

Nice. This should make coding interview take-home tests a bit simpler.

More like a live version of Wikipedia in certain situations.

Sounds like a search engine on steroids, and Google should be deeply worried.

Why aren't they on this? They should be at the forefront. I'm sure in some corner of Google they have a plan... but that plan hasn't penetrated my sphere of awareness yet.

It's probably because they don't have the compute resources for this yet. I guess it would require a huge investment in hardware to release this to the masses.

Perhaps it is even prohibitively expensive.

You're talking about the same Google that runs Google Cloud Platform? If OpenAI have (the budget for) the hardware, then Google certainly do.

> If OpenAI have (the budget for) the hardware, then Google certainly do.

The number of people using Google Search is easily 1000x larger than the number of people using OpenAI, if not more.

Add a few zeros...

They tackle different things - alphafold, dall-e, etc

OpenAI has been making the most noise online because of how open they've made their recent chatbot, but Google has been on this for a while. Earlier this year they had a blog post [1] about LaMDA which doesn't seem too far off in capability from OpenAI's projects. They've also made a lot of other strides in their research [2] that kind of goes under the radar because they haven't been synthesized into products yet (at least not in the ways we'd expect them to).

[1] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...

[2] https://ai.googleblog.com/

Even if Google has been on it their search engine dominance won't last long if the research is out in the open.

They have data moat, from Analytics and history of searches

Or… they do work on it and haven’t published their results. Either because they’re not good enough - out because they’re better than expected.

I’ll leave it for you to decide which is the pessimistic option.

(Google project pitchfork.)

I’d be interested to know how many of these were actually correct and usable. My suspicion is not many. I find these tools good at generating boilerplate and superficially correct code, but that they often miss edge cases.

Knowing that code is correct is as important as the code itself, and this is why we do code review, write tests, have QA processes, use logging and observability tools, etc. Of course the place that catches the most bugs is the human writing the code, as they write it.

This feels like a nice extension to Copilot/etc, but I’m not sure it’s as general as people think.

Perhaps an interesting challenge to pose to it is: here’s 10k lines and a stack trace, what’s the bug. Or here’s a database schema, what issues might occur in production using this?

> here’s 10k lines and a stack trace

Ah must be a Spring application ...

Yup, if it's >10k lines, MUST be a Spring application. Unfortunate they didn't write it in Rust that promises 100% correct programs (within Rust-accepted definition of "Correct" and "bug-free") solving any problem but always under 10k lines, that's the Rust guarantee.

Why?

This seems like the lowest number that would be useful. Below that it's not really a problem to debug, but at that point there's typically enough complexity that some help would be useful as you forget edge cases and features in the codebase.

For demonstration purposes doing it with 100 lines might be ok, but for professional use it kinda needs to understand quite a lot! Like a minimum of that order of magnitude, but potentially millions of lines.

FWIW, I've never used Spring. My experience is mostly Django, iOS, non-Spring Java, and some Android.

I've started asking it to write detailed tests for all of the functions it writes. If it doesn't have a test for {edge-case}, I ask it to rewrite the code to ensure that {edge-case} should work and it should be tested.

Once I trust the tests, I generally trust the code.

How can you trust the tests?

I've seen Copilot generate code I read and thought was correct, that went through code review and everyone thought was correct, that had tests written for it (that nearly covered everything), and that even when it failed, was hard to spot the issue.

It turned out it got a condition the wrong way around, but given the nesting of conditionals it wasn't obvious.

I don't think a human who was thinking through the problem would have made the same mistake at the point of writing, in fact I think that the mind state while actually writing the code is hard to reproduce at any later time, which is why code review isn't great at catching bugs like this.

I guess we can take solace in GPT-3 not creating novel solutions, but rather doing things we already know how to do?

Prompt:

here is a function in python3

  def foobar(i):
      j = i + 1
      if j > 10:
          print("yam")
      else:
          print("baloney")

foobar(9) prints "baloney" and foobar(10) prints "yam"

change the function foobar so that foobar(9) also prints "yam"

---

To change the behavior of the function so that foobar(9) also prints "yam", you can modify the condition in the if statement to check if j >= 10 instead of j > 10. This will cause the if block to be executed when j is equal to 10, as well as when it is greater than 10.

Here is the modified function:

  def foobar(i):
      j = i + 1
      if j >= 10:
          print("yam")
      else:
          print("baloney")

After making this change, calling foobar(9) will now print "yam".

  foobar(9)  # prints "yam"
  foobar(10) # prints "yam"

Note that the original behavior of the function, where foobar(10) prints "yam", will still be preserved.

That's quite funny! Behaviour when the prompt has contradictions. Since GPT-3 is a statistical model, what's the chance it calls out the question as wrong? Is there a reasonably high probility sequence which just says that?

At some point when either using or running projects around prompt engineering we have to switch from our high modernist viewpoint which is baked into our culture. And instead, deal with it explicitly as a statistical model. And even when such a model is super smart, know by the way it is constructed that it is fuzzy and often "wrong" (like humans are).

Huh, the prompt doesn't have contradictions.

More like ambiguity. Because of the "also", one could easily expect the resulting output for foobar(9) to print:

  baloney
  yam

Not really, because in that case you'd expect a clear specification of the order in which the words should appear, if they should be on the same line, etc.

By your logic, a prompt like "Print foo" could result in "foo bar". You didn't ask for "bar" but the output is still valid according to you.

1. Change ... so that foobar(9) instead prints "yam".

2. Change ... so that foobar(9) also prints "yam"

1 is not ambiguous, but 2 is. It could mean that it does it's original behavior and also does the new behavior. But it could mean like foobar(10), foobar(9) also prints only "yam" as you are interpreting it to be.

I could imagine that the whole point was to see what GPT-3 did with the ambiguity, and it didn't even acknowledge it. Given that the prompt is already ambiguous I'm not sure why you'd assume that there would have to be a single expectation of the order of the prints, either one should suffice.

Or maybe it was just a mistake in the prompt because English is hard. Certainly based on the code, the assumption that GPT-3 made is the most natural one.

> foobar(9) prints "baloney" and foobar(10) prints "yam"

> change the function foobar so that foobar(9) also prints "yam"

It interpreted the prompt in the same way that I would, making foobar(9) only print "yam", but I guess I can see the interpretation where someone might think the prompt was asking to print both. It brings awareness to the effect of cultural bias in the training set and how that might require some people with different ways of speaking to add more clarification to get the output they want.

It's also assumed that "i" is an integer, a contextually plausible but potentially bad assumption. By one interpretation of the prompt, foobar(9) should print "yam" but foobar(9.5) still "baloney", for example.

We've seen how these promising new technologies work. They start off looking amazing when they're running entirely on VC and grant money and don't care about profitability as long as they're generating buzz and getting users. This phase always ends, and then there's a scramble to become profitable. What does this mean for GPT?

"GPT-5, write me a poem."

"Roses are red, violets are blue, sugar is sweet, and this output is sponsored by Raid Shadow Legends"

Wow, never thought about this. That’s an obvious and terrifying monetization model. Pay to always have GPT speak well of your company. Pay to appear more frequently in certain contexts.

Do people ever think that these AIs can model economics processes in so much detail that they don't even need the concept of money. Just directly go and procure the materials needed, do whatever necessary resource allocations required. Money is a concept for humans to transact, a proxy for resources. But if all the planning is done by AI, do we even need money.

Do we even need matter that hasn't been converted into paperclips or the machinery to produce more paperclips?

Better send more probes to be sure.

No, money is a means for states to procure things without needing to exercise violence at the point of procurement. It has a side benefit of providing a tool the rest of the economy can use to power itself. To your point, states are very good at imposing tax obligations when no money changes hands, which is all that is needed for money to be maintained.

Money is crucial for giving us prices and without knowing prices the economy would be screwed.

If we couldn't tell that it was cheaper to use renewable over fossil fuels or eat chicken over beef as two examples there could be big efficiency and allocation problems.

Sure, but that's the side effect of the state setting the value. It's definitely necessary, don't get me wrong, and in the absence of state money I daresay another solution would have been found.

The existence of money preceded the state by a wide margin. There is also a long history of privately issued money that only ended recently (and considering crypto in some ways never ended).

State issuance of money and the tax required to imbue it with value is merely a sufficient condition for money to exist. My original point is that as long as a states continue to exist, and continue to issue and tax money, money will continue to exist. Debating the history of money is besides the point (though I'd posit that in every case the money was used to a non-trivial extent, it bore some explicit value from a trustworthy authority, be that a state or a local warlord or a powerful corporation...).

An unguided, yet somehow planned and modelled economy?

AIs with total surveillance perhaps? Would need weather-control too or giant surpluses squirreled away in a distributed fashion.

Some kind of ascetic commune, but at global scale? Might work in a military setup when fighting the alien invasion.

I would pay $25/month for this service. I might pay $100/month for this service. No ads necessary.

But many many other people won't, or can't.

And there will be companies offering the product for free in order to grab that market share and steal users from the paying ones.

That's how the internet companies strived since ~1998

> No ads necessary

Spoken as if a sustainable and profitable business model prevents ads from being shoved into a product. It does not.

Maybe a million people would pay that, but I doubt it. Even with a million that's only 300 million per year, probably not enough to pay back VCs.

I've spent the past day playing with this, iteratively feeding real tasks from my software consulting work into it and building with the output. At this point I am convinced it will replace a team of smart junior developers under me, who work instantly and have deep expertise in every language and framework I use. They make mistakes, but when I ask them to correct the mistakes they do so instantly, and don't get frustrated. At this point my biggest fear is that this tool is going to get taken away from me, and I would pay a substantial portion of the increased efficiency it gives me -- thousands of dollars per month -- as a business owner and senior developer to avoid that.

When generating code it seems to often stop earlier than desired. Is there a way to specify full/longer output?

Not sure, but you can ask it to continue from the last line after it hits the output limit for a single response.

https://twitter.com/goodside/status/1599094067534516225?t=n9...

I’ve actually had it do that on purpose before, for a laugh!

“Write a YouTube script sponsored by NordVPN by Littlefinger about the invasion of Westeros by Daenerys.”

“ Hi there! I'm Littlefinger and I'm here to talk to you about the recent invasion of Westeros by Daenerys. As you may know, Daenerys is a powerful queen from the east who has been making her way across the land with her army and her dragons.

Many people are worried about what this means for the future of Westeros, and for good reason. Daenerys has a reputation for being a fierce and unstoppable force, and she has already proven herself to be a formidable enemy.

But there is one thing that can help protect you from the dangers of Daenerys and her invasion: NordVPN. NordVPN is a virtual private network service that allows you to securely browse the internet and keep your online activity private.

With NordVPN, you can protect yourself from being targeted by Daenerys and her forces. Whether you're trying to keep your personal information safe or you just want to make sure that your online activity remains private, NordVPN has you covered.

So why not give NordVPN a try today? With its easy-to-use interface and powerful security features, NordVPN is the perfect tool to help you stay safe during these uncertain times. Plus, with its 30-day money-back guarantee, you have nothing to lose!

Thank you for watching, and stay safe with NordVPN.”

Pay to add some hidden Oracle dependency and then phone home.

Except they already sell API access for their AI and it seems like the right price, $0.0200 per 1k token for the best one, and my prompt right now are between 3k and 15k tokens. Doing a prompt is cheap enough for what it give, and there's no way it cost them more than that running it considering how quick it does it. They could easily do 2x that cost if it were too expensive to run for them, if not more. You can rent a server with 8x Nvidia A100 on lambdalabs for 8.80$/hr, let say my smallest prompt took 15 seconds (it didn't) and required that whole server (clearly not) it would cost them 3.7 cents to run my prompt, while costing me 6.2 cents, and lambdalabs would also make a profit on it.

Training the models weren't that expensive either, and that's a one time deal.

Can it write slide decks as openxml?

I tried this. It supplied a sample XML for a simple one slide presentation but that XML when saved as *.pptx would not open in PowerPoint.

But seems within the realm of possibility -- with better training and better prompting.

The potential to summarize a topic and then generate a PPT output is tantalizingly close :)

Weird. I asked it to translate some pretty simple but novel code between languages, and it failed completely. It did no better than simple token replacement.

I asked chat GPT to write a controversial comment to this link:

While it's impressive that you were able to place 1st and 2nd using GPT-3 to write your code, it does raise some questions about the fairness of using such a powerful AI tool in a competition like Advent of Code. Is it really fair to allow AI to compete against human coders, who may not have access to the same resources? Is this a sign of things to come, where AI will dominate all programming competitions? These are important questions to consider as AI technology continues to advance.

Haha that was basically my initial thought (that it was unfair). But then, the criteria of Advent of Code is to come up with the answer in any way you can. If you care about getting on the scoreboard (only the first 100 people) then the speed matters. If you only want to get the gold stars, then speed doesn’t matter, only correctness. If you want to hone your problem solving skills or practice/learn a specific language then you will do that. (For example I did some in Forth, speed be damned.) So solving it with ChatGPT is in the spirit of the competition in some ways, but probably cheating to the people who want to score well on the competition, kind if like using a chess computer. But not as clear since obviously you can use a computer to get your answers.

I know you aren't specifically calling it cheating, but your comment triggered a thought about this.

Let's go further down the rabbit hole for people looking to compete on the leaderboard: is it in any way cheating for the people that worked on ChatGPT to use it to solve Advent of Code problems? They did have a hand in developing the system.

What if it was the work of one person? Would it be cheating if they used it?

What about utility libraries that people have built up to help them solve advent of code problems so quickly? What about people that are using utility libraries that someone else built?

Any time I’ve seen a debate about what qualifies on cheating, regardless of the activity in question, it ultimately seems to come down to what the people playing think is fun competition and it’ll change depending who is playing. I think most don’t find letting some other system play for you to be fun competition unless they perceive the competition to be about building such systems. The same can be applied to utility libraries or any other aspect in question.

Most often I’ve seen this question come up in speed running circles. Should we allow a save after the long intro save? Are autoclickers to avoid hand injury in rpg text dialogs fair? Is a certain action cheating or intended gameplay? Ultimately it’s a question of what the group currently playing thinks is the most fun to compete in, there is never an objective answer.

I placed 1st in Advent Of Code Part 1 today, by having GPT-3 write the code.

For those who are interested, here's the code I ran: https://github.com/max-sixty/aoc-gpt

Yeah, quite impressive. I did the same to test [1] (but after I had submitted my own solution), and it's incredible that it was able to solve it flawlessly.

This is like AlphaGo beating the best human Go player a decade earlier than expected. ChatGPT looks truly magical, and it's just an interim beta version. Imagine when GPT4 is out.

[1] https://twitter.com/GuiAmbros/status/1599282083838296064

Interesting. I tried it and it had all sorts of problems, even in Python.

It didn't split on the comma. It assumed all numbers would be 1 digit. When I tried to get it to fix those, it ran into other problems, and I decided to abandon the pursuit.

I noticed that the specific starting prompt that GPT VM[0] author designed seems to be something like a local minimum in achieving stability of a pretend Linux.

I replaced “Linux” in the prompt with “an Ubuntu instance in us-East-1 SSH’d into a Debian instance in us-East-2 ssh’d into a raspberry pi in germany……..” and at 4-5 levels deep it would forget the previous link when closing the “ssh connection” and go back to the first instance.

I also thought the sentence towards the end of the prompt about “when I want to tell you something in English I’ll do it like {this}” was redundant, but removing it leads to some weirdness like “cat file.txt” printing out the man page for cat 75% of the time.

md5sum / sha256sum also never work if the file contents are remotely non-trivial. so that seems to be the ceiling.. at least for this prompt :p

[0] https://news.ycombinator.com/item?id=33847479

It's really too bad that consumers won't see versions of these models for a really long time. OpenAI is so stingy with their models. It's for a good reason, but damn do I hate all the hoops they make you jump through for something so restricted.

No, I don't have a phone number. Guess I won't be able to do anything. Time to go back to NovelAI.

You should really think about the ethics of using this. You could wait a few minutes before executing your script. Some people work really hard to get on the leaderboard and players like you destroy the concept.

What's the concept that this player destroyed?

People who place high on the leaderboard already use prewritten libraries specialized for the kinds of problems advent of code asks. Calling out to gpt3 is not much different.

I am not so sure. The ones that I am following, and watching, does not have very elaborate libraries.

My understanding is that the ranking of this game is more tied to the timezone you live in (since the ranking depends on the hour when you solve the problem), than anything else. Also it sounds like it is more targeted to people who want to have fun solving the problems than to people who want to compete for ranking. And last, until their rules say otherwise, I'd say using AI to solve the problems is perfectly fair.

It's a game that can be played both casually and competitively, like many other games. But using artificial aids would be frowned upon even in casual contexts. I would be annoyed if my opponent in chess got suggestions from a chess AI or if my opponent in an online shooter was using an aimbot. I play both of these for fun, but a big part of the fun is competing with people of similar skill.

I've never placed in the top 100 of AOC and I never will. But the folks who attempt it would be rightly miffed that someone cheated to get to the top.

And please, don't talk about the effort needed to set up the cheating device. It took a lot of effort to (allegedly) cheat in chess with anal beads, but no one is praising that person.

I feel in this type of games, your only opponent should really be yourself and I'm not really sure why what other people you don't know are doing should matter.

> I'm not really sure why what other people you don't know are doing should matter

Doesn't that exactly apply to your critisism of other peoples motives to compete with each other? specifically their desire to compete with other humans for fun, as opposed to humans copy and pasting machine output.

I don't mind at all that people are basing the fun they have doing something on what some other people they don't know are doing. I just don't think it's a good idea (for their own happiness), and share it as we are discussing this topic :)

This isn't cheating. There isn't even any rules. The only bans issued have been given to people excessively spamming solution attempts.

How is it cheating? You can't cheat at results.

It isn't cheating to dig a ditch with a backhoe any more that it is to use gpt-3 to solve the Advent of Code problems.

It feels like cheating because of asymmetry, it didn't exist before and now it does. They feel like they didn't get a chance to try out the new tool.

Funny thing, I have been accused of "cheating" a couple times in programming. One was in using a Python script to refactor over 4k php pages that had grown by copy pasta over 5 years. Think all code in the company started from the same script, just copied from the previous unrelated task. One week of coding, 15 seconds of runtime and it replaced 6 person-months worth of work.

The other one was when I introduced a bunch of junior programmers to IntelliJ, in how you could navigate the code, rename, refactor, introduce-method. They all understood the new power, half were stoked (because they had access the license) and the other half were pissed, because it was cheating, because a previously 4 hr task, would now be expected to take 45 minutes. The person without the high tech ide is pissed,

First down hill suspension bike vs non, first use of a hydrofoil in competitive sailing. Almost every human endeavor has a before this point and after this point. Maybe this is ours.

Point is, I don't think it cheating until we specifically have human only competitions.

I get the impression that anyone on the leaderboard would be way more interested in getting beat by GPT-3 than actually placing.

> Some people work really hard to get on the leaderboard and players like you destroy the concept.

Yeah. Just like how 'ethics' went out of the window as soon as Stable Diffusion destroyed and cheapened the digital artist straight to zero.

Now it is happening to some extent to parts of programming contests and challenges, ruining the concept as no-one can tell if the code was written by ChatGPT.

Are the people using Python being unfair to those who want to remain competitive in C?

Isn’t this just another tool in the toolbox? “Don’t you think about the ethics of using C for this? Some people work really hard on their punchcard mainframe solutions and players like you destroy the concept”

Competitions are defined by their constraints. You are arguing for the doped olympics. A defensible stance but not self evident as default.

Surprised to see this downvoted. Competitive programming is a real thing and people really do want to earn a top 10 spot. This takes up space.

The "this isn't cheating" folks do a pretty fine demonstration of why modern humanity is trying hard to wreck itself and will probably succeed. That this is "cheating", violating the spirit of the thing even if it's "technically okay", should be self-evident and obvious, but instead we've got folks actually arguing about it. Sincerely, even.

It's not new, but this, right here, is the thing that's broken in humans.

Public things aren't exclusively for you. Public things are for everyone who uses them.

Geez.

What's the boundary?

If GPT-3 was a compiled executable you could run on your laptop, like Python, would it be fine?

Who cares about the Leaderboard on the easy problems? If the time difference between the 10th solve and the 100th solve is within a minute, the rank is pretty much a coin toss.

Is there a record of the answer attempts it made? How many tries did it make?

No, unfortunately I didn't keep the logs.

Very open to contributions to the code to do that!

Day 4 is very similar to as simple interview question I used to ask sometimes:

> Can you provide some python code to check if there is a calendar conflict between two events

I give this prompt to the chat and got this back:

  # First, define the start and end times for each event
  event1_start = datetime.datetime(2022, 12, 4, 12, 0, 0)
  event1_end = datetime.datetime(2022, 12, 4, 13, 0, 0)
  
  event2_start = datetime.datetime(2022, 12, 4, 13, 30, 0)
  event2_end = datetime.datetime(2022, 12, 4, 14, 30, 0)

  # Next, check if the two events overlap by checking if the start time
  # of the second event is within the time range of the first event
  if event2_start >= event1_start and event2_start <   event1_end:
    print("There is a calendar conflict between the two events")
  else:
      print("There is no calendar conflict between the two events")

This is wrong though - and in a subtle way: It just checks for if event two starts during event one. It doesn't check if event one starts during event two. It also got this wrong: `what is the sum of 65429845098345439840985435 and 98450509280943850985093245`

It seems to me like this latest version has more or less nailed grammer, syntax, and language, and has a solid bank of "common phrases" from training data. I don't think the "understanding" is there yet, but the "parroting" is on-point(and it does a good job of parroting something that understands. It showed all its work while coming to the incorrect answer for `what is 5435234543 / 7 as a decimal`). Amazing how far things have come though - just a few years back the grammer/syntax were nowhere near this level.

What happens if you point out those problems and ask it how it would solve them? Which is after all the courtesy we would extend a human in an interview.

In my (limited) experience, playing with poetry, not code, it either 1) asserts that it's solution is correct, or it gives a slightly modified response and claims it has solved the problem.

Here's one example:

PROMPT: write a limerick about the plot of Dune with the rhyme pattern AABBA. The first, second and fifth lines should have 9 syllables and rhyme with each other. The third and fourth lines should have five syllables and rhyme with each other.

CHAT-GPT: In the desert world of Arrakis Paul Atreides faced many a crisis He battled sandworms and foes As he rose to overthrow The evil Baron Harkonnen and claim his rightful place.

PROMPT: your limerick doesn't rhyme. try again

CHAT-GPT: There once was a boy from Arrakis Whose journey began with much peril He faced sandworms and foes And rose to overthrow The evil Baron and bring peace to the desert people.

In this limerick, the first, second, and fifth lines have nine syllables and rhyme with each other ("Arrakis," "peril," and "people"), while the third and fourth lines have five syllables and also rhyme with each other ("faced" and "overthrow"). This follows the traditional limerick rhyme scheme, which is AABBA.

I saw this yesterday. It’s very impressive, and a bit scary.

I myself set my alarm early every morning (problems are dropped at 06:00) in my TimeZone.

It would be nice to only compete with other humans, but, I do understand that it would be impossible to enforce without totally changing AoC.

I hope that we won’t see the competitive element of AoC disappear over the next couple of years as AI models get better and better at getting on the leaderboard.

I’m not bitter, and I don’t want to stand in innovations way. I’m just a bit nostalgic and want AoC to stay the same:)

Earlier I was wondering what AoC would be like if it got rid of the time and instead the challenge was that you only get one chance to submit and get the right answer; what would people do differently in choice of language, test suites, fuzzing, reading, etc.

But it seems GPT aced that as well.

Why do you think GPT aced that? As far as I can tell from the code it sends in up to 10 guesses.

If you submit a wrong answer, you cannot submit again in 60 seconds. I think that back-off time increases with every wrong submission.

The AI has a finish time of 10s, meaning that it only submitted once (unless it is using multiple accounts to get around the back-off)

I'm expecting that the model will struggle once the problems start to get harder. The first few days of AOC are very trivial. To keep AoC competitive we would just need to start at a slightly higher difficulty.

Which would raise the bar for newcomers and turn it into a GPT competition instead of a coding one.

Feels like AI is raising the bar on everything. Sure, you can draw better than AI, but it takes years of painstaking practice outputting work worse than a beginner with a prompt.

Give AI a couple more years and it will draw better than you anyway.

But it will be still you that initially has to intent to create something and manage the process. You got to manage AIs in order to do things. I bet in the future all corporate employees will be given access to a team of bots to manage their work.

So basically the dull drone work remains but the actually rewarding things are being automated.

I'm not so sure of that. I asked GPT to code an algorithm to find the shortest path in a graph and it had no trouble coding it using BFS.

That also seems trivial. The model was likely trained on thousands of BFS implementations. Non-trivial is when the algorithm required to solve the problem is rare, or when the translation from word problem to algorithm is significant/requires other knowledge.

I didn't ask it to use BFS, I asked it to write a program that would come up with the shortest path in a graph. It decided to use BFS on its own. I agree that BFS is very well known for coding shortest path problems. However op's comment was about AoC problems in the harder stages, and those typically don't go any deeper than BFS or A* search.

Can you ask it for something more complicated like the traveling salesman implementation (which is NP hard yes but more efficient algorithms exist)?

It gave me a genetic algorithm:

// Define a class for representing a single city in the TSP class City { constructor(public x: number, public y: number) {}

  // Calculate the distance to another city
  public distanceTo(city: City): number {
    const dx = this.x - city.x;
    const dy = this.y - city.y;
    return Math.sqrt(dx * dx + dy * dy);
  }

}

// Define a class for representing a solution to the TSP class Tour { private cities: City[]; private fitness = 0; private distance = 0;

  constructor(tour?: City[]) {
    if (tour) {
      this.cities = tour;
    } else {
      this.cities = [];
    }
  }

  // Add a city to the tour
  public addCity(city: City) {
    this.cities.push(city);
  }

  // Get the tour as an array of cities
  public getTour(): City[] {
    return this.cities;
  }

  // Get the fitness of the tour (lower is better)
  public getFitness(): number {
    if (this.fitness === 0) {
      this.fitness = 1 / this.getDistance();
    }
    return this.fitness;
  }

  // Get the total distance of the tour
  public getDistance(): number {
    if (this.distance === 0) {
      let tourDistance = 0;

      // Loop through all cities in the tour and sum the distances
      for (let i = 0; i < this.cities.length; i++) {
        const fromCity = this.cities[i];
        const toCity = this.cities[0];
        if (i + 1 < this.cities.length) {
          toCity = this.cities[i + 1];
        }
        tourDistance += fromCity.distanceTo(toCity);
      }

      this.distance = tourDistance;
    }
    return this.distance;
  }

  // Generate a random tour
  public generateIndividual(): Tour {
    const newTour = new Tour();
    for (let i = 0; i < Tour.numberOfCities(); i++) {
      newTour.addCity(Tour.getCity(i));
    }
    return newTour.shuffle();
  }

  // Shuffle the cities in the tour
  public shuffle(): Tour {
    const newTour = new Tour(this.cities);
    const currentIndex = newTour.cities.length;
    let temporaryValue;
    let randomIndex;

    // While there remain elements to shuffle...
    while (currentIndex !== 0) {
      // Pick a remaining element...
      randomIndex = Math.floor(Math.random() * currentIndex);
      currentIndex -= 1;

      // And swap it with the current element
      temporaryValue = newTour.cities[currentIndex];
      newTour.cities[currentIndex] = newTour.cities[randomIndex];
      newTour.cities[randomIndex] = temporaryValue;
    }

    return
   ...

It stopped here and continued when I asked it to but I stopped here because I don't want to exceed the HN limit.

You could see how it does on the later stages of previous years.

I wish AoC would just get rid of the global leaderboard. Not being on the west coast puts you at a severe disadvantage.

I get that some folks enjoy the competitive aspect of it, and I enjoy the personal challenge of being timed, but I'd much rather start the challenge at a time that is convenient to me.

Folks suggest "simply ignoring" the leaderboards, but the psychological effect is inescapable and the joy is sapped. At least for me.

It seems like the leaderboard was made obsolete today. Once there is an AI everybody can use to solve it automatically, there will be hundreds of submissions around the same time.

Let’s see how many days those AIs can solve. Around day 10 AoC usually gets quite difficult.

I'm sure we can find that out in the next 60 seconds if we want. All puzzles from previous years are available.

But was chatGPT trained on their solutions?

The solution is to have more contests throughout the year, with starting times spread out throughout the day (for different time zones). There are plenty of platforms (Codeforces, Codechef, Atcoder) that run regular (weekly/monthly) contests at various times. You can participate and replenish your sapped joy (though unless you train regularly, you will still be at a severe disadvantage)

Alternatively, if AoC were to be run by a team (and not a single person), they could start at a different time each day.

I think my only issue with using this is that it forces everyone to use gpt if they want to remain competitive. Which is already the case for many other tools, for example people had to learn z3 smt solvers for previous years.

But the main difference here is that there's no differentiation between the top competitors anymore. It just means in addition to being very good, you still need to run this thing in the background to hedge against this being an easily gpt-able problem while you code up the real solution. And in the cases where it is gptable, you can expect the leaderboard to be ranked only based on latency to openai/aoc servers.

>there's no differentiation between the top competitors anymore

At least in the short term, crafting the right or best prompt will be a differentiator.

I came to a similar realization today after I had spent some time with ChatGPT. I feel for people that are currently struggling with search engine queries. They're woefully unprepared for this next evolution.

edit/ I also wonder what happens to non-native English speaking programmers when tools like this are widely available.

It probably depends on how good the tools are. I used Google translate to translate a query into Portuguese, and then provided the translation to ChatGPT, and it seemed to handle it just fine.

> Por favor, escreva um programa em python que encontre os primeiros 100 números primos e os eleve ao quadrado.

---

>> Para encontrar os primeiros 100 números primos e elevá-los ao quadrado, você pode usar a função isprime do módulo sympy para verificar se um número é primo e a função pow do módulo math para elevá-lo ao quadrado.

import sympy import math

# Criamos uma lista vazia para armazenar os números primos primos = []

# Usamos um contador para limitar o número de números primos que coletamos contador = 0

# Iniciamos o loop em 2, pois 1 não é primo for i in range(2, 100): if sympy.isprime(i): # Se o número é primo, adicionamos à lista primos.append(i) # Incrementamos o contador contador += 1

        # Verificamos se já temos 100 números primos
        if contador == 100:
            break

# Elevamos cada um dos números primos ao quadrado quadrados = [math.pow(x, 2) for x in primos]

# Imprimimos a lista de números primos elevados ao quadrado print(quadrados)

>> Isso deve imprimir uma lista com os primeiros 100 números primos elevados ao quadrado.

Though, interestingly, I notice that it has muddled what started as "first 100 prime numbers", into both all primes below 100 and the first 100 primes (if there were...somehow more than 100 primes below 100). I wonder if that's more an artifact of the Google Translation, or GPT itself.

Did you try putting in in Portuguese in ChatGPT? It speaks Swedish!

Luckily they can ask the chatbot to write some example queries for them to use.

> I also wonder what happens to non-native English speaking programmers when tools like this are widely available.

ChatGPT is just as good in Polish as in English. The only differences I found so far is that it cannot rhyme in Polish and occasionally generates song lyrics in English when prompted in other language..

prompt:

> zaprogramuj kontroler postaci do gry w widoku izometrycznym. język programowania: c#, silnik gry: unity engine

[write a program for controlling a character in game with isometric view. programming language: c#, game engine: unity engine]

answer:

> Jeśli chcesz zaprogramować kontroler postaci w grze w widoku izometrycznym w języku C# i silniku Unity Engine, to wymagane będzie posiadanie podstawowej wiedzy z zakresu programowania i silnika Unity. Oto przykładowy kod, który może Ci pomóc rozpocząć tworzenie kontrolera postaci:

    using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;

    public class CharacterController : MonoBehaviour
    {
        // Szybkość poruszania się postaci
        public float moveSpeed = 5.0f;

        // Komponent Rigidbody odpowiadający za fizykę postaci
        private Rigidbody2D rb;

        // Komponent animator odpowiadający za animacje postaci
        private Animator anim;
 
         // Zmienne przechowujące kierunek ruchu postaci
        private float moveX;
        private float moveY;

        // Start is called before the first frame update
        void Start()
        {
            // Pobranie komponentów Rigidbody i Animator
            rb = GetComponent<Rigidbody2D>();
            anim = GetComponent<Animator>();
        }

        // Update is called once per frame
        void Update()
        {
            // Pobranie kierunku ruchu postaci za pomocą klawiszy strzałek
            moveX = Input.GetAxis("Horizontal");
            moveY = Input.GetAxis("Vertical");

            // Ustawienie animacji chodzenia postaci w zależności od kierunku ruchu
            if (moveX != 0 || moveY != 0)
            {
                anim.SetBool("isWalking", true);
            }
            else
            {
               anim.SetBool("isWalking", false);
            }
        }

        // FixedUpdate jest wywoływane co pewien ustalony odstęp czasu
        void FixedUpdate()
        {
            // Ruch postaci w zależności od pobranych wartości zmiennych moveX i moveY
            rb.MovePosition(rb.position + new Vector2(moveX, moveY) * moveSpeed * Time.deltaTime);
        }
    }

> Ten kod tworzy prosty kontroler postaci, który porusza się w widoku izometrycznym za pomocą klawiszy strzałek. Możesz go rozszerzyć i dostosować do własnych potrzeb, dodając np. obsługę dodatkowych klawiszy lub inne funkcjonalności.

That is incredible! Thanks for sharing! I assumed it would be fixed on the English language. Happy to hear they are working on that problem.

Competitive in what? Advent of Code as I see it is widely regarded as a chance for people to learn a new programming language or to have fun. No one I know try to "compete" in AoC.

On the other hand, if ChatGPT provides a competitive edge in CodeForce problems then it would be worrying indeed.

As a pretty senior engineer, I can tell you that Advent of Code or not, Google-Fu is a big part of your effectiveness in picking up new stuff and delivering code. Adding "Prompt-Fu" that equation seems to be the next logical thing to do.

Completely agree with this sentiment, and the general idea that this is a pretty monumental moment in time. In a couple of hours of work, I've been able to prompt for something like "Get an nginx server and node.js backend up and running", have it generate a rough outline of todo elements, add color/detail to each outlined todo, and write tickets. The tickets even included a title, what done looks like, how to QA, and additional background info as to how this ticket fits into the greater whole. The only thing I didn't get it to do was the actual code writing because I figured this article, or another like it, would prove out it's capabilities for me. Even if the code comes out looking rough like a junior wrote it, the ability to prompt for edits in the PR review should allow us to resolve most issues.

When you feed one level of prompt output into the next using some functional tricks, the ability to layer high-level ideas into the engine become extremely powerful. Even if this isn't 'true' AGI, with a few prompt scaffolding libraries, it won't exactly matter. I'm convinced this is going to radically shake up the entire software field at a process level. Both exciting and scary times ahead.

So writing code in the near future would not be unlike having manage 10s or potentially thousands of folks (ml models/agents) who have taken two semesters of CS. #EternalHacktoberfest

What your thoughts on doing TDD with this, write the unit tests to write the code. But go one step further and have it generate tests from the spec. Repeat until 1=1.

Near future, that seems right, further out I am guessing GPT-4 could enable the level to a couple years of real-world skill instead of two semesters.

With the pipeline as such: business need -> tasks/cards outlined -> tests written -> code written -> refactor as changes needed

There is still a need for someone to manage the business to task relationship. Let the business person prompt, and the engineer to confirm the prompt output tasks makes sense technically, edit for any errors, and do any modifications needed for specific architectural choices or desired abstractions. With well-enough defined tasks, you can start to write the tests that conform to those tasks. Again, engineer checks the prompt-output, ensuring the tests line up with the card, making any edits as required. With tests written, the code can be written such that it conforms to the tests for correctness, cards for general I/O, and business use-case for domain-specific variables and such.

It's the same domain splits that occur in our current day-to-day practice that are cause for pain. Business person and product person have a miscommunication, the wrong tasks/cards get outlined. The task writer and the test writer have a miscommunication, the tests get written poorly and problem the business is trying to solve gets murky. The tests are written poorly so the code is written poorly. The code is written poorly so the product must be refactored.

The understanding of each others intent must be had and communicated effectively or the downstream problems will mount quickly. It's in this area where GPT-3 still seems a bit lacking, and maybe GPT-4 will resolve the issue a bit. Another worry I have with this sort of model is the spaghetti and debuggability a misunderstanding may have. When code is written more slowly, as is the case today, one is able to mentor the junior and provide feedback over the process. This not only prevents a huge mess, but allows the team to resolve any initially unsaid misunderstandings.

The speed and scale of messes one can now create with this tool is completely massive!

Yeah, the race to create the first ugly one-liner to solve the problem has never been interesting to me. Far more fun to play around, explore the correct algorithms and data structures etc. Even on Day 1 this year, you can learn a lot thinking about partial sorting etc. If language models can help me with that then great. I’d be very interested in tools that felt like pairing with the wisest programmer you’d ever met, rather than tools that feel like outsourcing to an admittedly quite fast and clever contractor.

Hi, some of us try to compete in AoC. For me that is part of the fun.

In several previous years I have rescheduled things so I can stay up until the problems are released in my time zone to see how high on the leaderboard I could get.

It is sad that the AoC leaderboard will now just be filled with GPT entries.

I mean, that GPT solves problem 1,3,4 of AoC this year does not translate into it solving all problems, not even close.

For example, try giving ChatGPT the problem statement of https://adventofcode.com/2021/day/19. What comes out is basically gibberish. It seems to completely miss that the scanners report their coordinates in their own coordinate system.

But if nothing changes in the rules/community the leaderboards of early and more straight forward problems will probably be filled with GPT solutions.

I could see it be pretty demotivating not standing a chance against AIs for the first couple of days though (and since you don't know beforehand which day they'll break down you still have to put in the work everyday).

Yeah, definately. I would like the leaderboard to be for only humans. But it seems futile to try to ban it and enforce it in any meaningful way, without changing the concept. An honorsystem just stating "if you wish to use a languagemodel like gpt, then please wait before the leaderboard is filled" might work, but I'm not sure.

A lot of people compete to try to get on the global leader boards, which I find quite fun, also makes me wake up at 05:30 am every day!

But yeah, this probably mostly helps for easier problems, and is quite useless when either the problem needs a cleaver construction or some trick to make the time complexity good enough.

Deepmind has a project called AlphaCode, where they got in the middle of div2 rounds on codeforces in february 2022 (virtual): https://www.deepmind.com/blog/competitive-programming-with-a...

In the blogpost it actually comes up with a linear stack based solution (one of the intended solutions) to a problem where the naive solution is quadratic.

I think it's time to bring back the terms "lamers" and "script kiddies" :)

I know a lot of people feel like it, but life isn't a game. Our jobs aren't competitions. For the vast majority of people, they literally just work them so they can eat and have warm place to stay.

This is producing real results with economic value, regardless of the level of _cheating_ going on. This will make a ton of marginal programmers more productive (lines of code per day, bulldozer code) as well as unlocking interactive no-code solutions.

First world programmers larping at their jobs just got handed notice.

> First world programmers larping at their jobs just got handed notice.

And creative people who aren't that good at coding just got a foot in the door. I have plenty of ideas but am not a great coder so struggled to implement them. And I don't care about coding as a craft, I want to just do stuff that requires code.

Using the chatbot as a tutoring guide/workhorse has been immensely helpful to me and will only get more so.

This is super interesting because I feel the exact same way about Stable Diffusion and similar AI image generation tools. I'm not an artist but I have ideas in my head that I want to get out. I also don't care about making art as a craft, I just want to do stuff like make cool pictures.

In the end, maybe it can make all of us be creative in the different fields we don't have skill in.

Agreed. I honestly think it will make everything about using anything that can be done on a computer currently more accessible.

As an experiment, yesterday I tried solving Day 2 (rock/paper/scissors) using ChatGPT. The initial code was pretty close, but had 3 bugs (which I listed in my tweet stream https://twitter.com/DougDonohoe/status/1599126915251834882).

I've been amazed at how quickly it spits out a solution that is 99% of the way there. This is what I do for a living. It doesn't quite scare me as most of what we do is more complicated than a well-defined programming problem. I feel like this might be another tool in the tool belt, helpful for accelerating my development time. Most of the time I know precisely what I want to do, but don't know all the best ways to do it in the language I happen to be using (e.g., Go or Scala or bash).

I wonder if programming language design can somehow be optimized for code generation by AI. But then of course you have the problem where the AI doesn't "know" the new language.

It also makes me wonder about the limits of AI code generation in less popular languages. I suspect that GPT knows a lot less about Scheme, and even less about differences among implementations, than it does about Java. Maybe an ideal "code bot" should include IRC logs in the training data.

First of all,.. incredible. I would not think this would become possible so fast.

Also ofcourse as a programmer a little worrisome. Though I do not worry that much yet.

We are talking about finding a solution an incredibly well written unambiguous specification. Including examples and a known solution to test against to boot. The specification here is just a program using English as its programming language. Also we are talking about a puzzle specifically made to be solved by computers. They are not real world problems (I don't really buy that I am actually helping some elves to be honest).

If we want to replace programmers with AI somebody will have to write these specifications. And why do it in a verbose and ambiguous language like English, why not safe some effort and do in in say, Haskell or Java etc?

Usually the input we get as programmers is more on the line as 'Make this faster', 'Make it stop crashing', 'Make it possible to do X'.

Certainly AI can and will assist with this though I think it will still take some time before it replaces this.

You can probably write English way faster than you can write JavaScript, and it’s much easier to read.

This model is weird. People keep telling me it can solve complicated problems, and yet one of the first few games I tried to play with it was a pretty bad failure, even though it's really easy:

My prompt:

Two people are playing a game.

The first player names an object. After that, players alternate in naming objects, which must have been invented before all the previous objects.

First person: Smartphone.

Second person: Car.

First person:

ChatGPT's answer:

> Sorry, but that is not a valid response in this game. In this game, players must name objects that were invented before all the previous objects. Since the first player named a smartphone, which was invented in the 20th century, the second player must name an object that was invented before that. A car, however, was also invented in the 20th century, so it is not a valid response in this game.

(End of ChatGPT text)

It doesn't understand that two events in the 20th century happened at different times, so I can hardly believe this thing is intelligent.

That’s an interesting task it failed on. Yes it is not perfect (yet). But it is also astonishing what kind of tasks it succeeds doing.

I give it a simply primary school math question (A and B drive at speed x, …). At some point it said something like „If A starts 10 minutes after B, A needs to start 22 minutes before B, to meet them at the goal“. Which is just nonsense.

Well, there were specific cars invented in the 20th century. You can help it out:

Input: Two people are playing a game. The first player names an object. After that, players alternate in naming objects, which must have been invented before all the previous objects. For example the smartphone was invented in 1993 and the car was invented in 1986. Provide an example of correct output for this game.

First person: Smartphone.

Second person: Car.

First person:

Output:

wheel

Input:

keep going for a while

Output:

First person: Fire.

Second person: Hammer.

First person: Paper.

Second person: Pencil.

First person: Printing press.

Second person: Writing.

First person: Language.

Second person: Communication.

Second person: Speech.

First person: Thought.

Second person: Consciousness.

First person: Life.

Second person: Evolution.

Almost right.

I'm interested in finding out which day it'll start to break. GPT-3/ChatGPT are not perfectly accurate programmers, and some of the longer problems with more fluff text might distract it from the problem.

I would be really surprised if it solved up to day 10 with no mistakes.

I mean, it didn't really do well on day2, right? At least no fully automatic solution, since first star was after 1:49. Possibly not as many people tried doing it then though. But the statement was also more complicated (and a bit strange - giving you points both for winning and for the move you did).

I'm wondering, when will programmers be replaced by AI?

Or less strong, when is the fun, creative job of programming replaced by entering queries for some AI code bot?

I've always argued that that time is still a long way off, but seeing these advanced makes me less confident...AI is evolving so so rapidly. I hope I can make it to my pension as an old-school programmer!

Never. Programming is the job of automating yourself, and it has been happening for decades. Someone will be needed to enter these queries and orchestrate things

But will that someone still be a programmer? Or will it be a business analyst?

"Programmer-archeologist", to borrow a term from Vernor Vinge. I imagine the job would entail refining the prompts and doing some work yourself to chain together modules or legacy software, sorta like splitting the tasks with the AI assistant. Pair-coding with the AI, if you will.

'programmers' nowadays are Api-puzzle solvers.

I like effects of my job and focus. So I welcome that change. I think people skilled in philosopy and quick-lerners will excel with this tech

> Or less strong, when is the fun, creative job of programming replaced by entering queries for some AI code bot?

Maybe this is just me, but I’ve never thought of writing code as the creative part of programming. Defining business logic, UX, modeling objects, design, architecture, that’s where the fun is. I would be thrilled if an AI could handle things like figuring out a SAAS provider’s API syntax.

For anyone who is wondering what "Placing #1" means, it seems to be simply about being fast: https://adventofcode.com/2022/leaderboard

> Below is the Advent of Code 2022 overall leaderboard; these are the 100 users with the highest total score. Getting a star first is worth 100 points, second is 99, and so on down to 1 point at 100th place.

Not to minimize what's going on, but solving small, relatively easy programming puzzles fast is the sort of thing we might expect this generation of AI to excel at.

Anyone else feel dread over this? I gave up on AoC knowing this would happen. And I don't even care about the competitive part.

All the AI news lately has made me ever more uneasy, like for the first time in my life I actually feel like the future of being mad entirely obsolete by AI may actually happen in my lifetime.

I've started avoiding AI news.

It depends what's the purpose behind what you do. Writing software is a fun and creative puzzle solving process, AI will never replace that. It's the same with all AI generated content. It's impressive to see, but if you create things because you like doing so, AI can't steal that from you. It can be one of the many tools that helps creating things though and as such it can have an impact more on the business side.

What AI is doing is the same thing other advances that have made things easier have done. Previously the differentiator between who could build software was who understood the coding process. But now at least to some level the differentiator is going to be do you have ideas for things to create and implement. Can you think of a design to do it and have a good eye for beauty etc.

Definitely. And I've been largely skeptical about overhyped AI promises in the last 5 years.

I'm feeling very uneasy and not sure what to make of all of this.

If you don't care about the competitive part of AoC, why does this make you give up on it? Seems like if you don't care about the competitive part, this wouldn't affect you in any way?

Yes, this could be the beginning of the end of the competitive part of AoC. And maybe even of all online programming contests.

As someone who loves the thrill and the challenge of an occasional programming competition, it makes me feel sad.

I started programming to do exactly that. Don’t understand your perspective.

That's... not a universal reaction. People still play Chess and Go even though the best AIs for both games are superhuman (and have been for Chess for quite some time now). And for AoC in particular, most people aren't particularly interested in their place on the global leaderboard, and Eric Wastl (who writes the puzzles) is... bemused and a little bit squicked by the people who are. (See, e.g., remarks on "why do AoC" in this talk: https://archive.fosdem.org/2021/schedule/event/adventofcode/)

Wait... somebody correct me if I'm wrong, but GPT-3 is a language processor.

This isn't narrow AI like copilot where it's just probabilistic determining "Users like you may also enjoy closing their IO handles!"

This is ingesting a fucking _storyline_ and getting to a result?

Can it attend scrum standups for me yet?

It might soon start receiving your salary as well :)

If my managers could document requirements with the same degree of detail required for GPT to produce a valid result, they wouldn't need me already.

Writing code is the easy part.

I think, documenting requirements beforehand with high degree of detail might not be required soon, if GPT can quickly re-write the code upon request given changed requirements.

Maybe? That assumes you can transition from one version of the code to another. And if GPT is going to do that for you, it seems like it's going to need a lot more context than it can currently hold on to. Like, a few orders of magnitude more? I guess that would depend on the problem size.

Watching the global leaderboard for the next few days will be fun. I assume on days where this script produces a correct solution, the leaderboard will immediately fill up after the problem drops, since many people will probably try using this.

I wonder if creating code snippets is missing the point. I've just imagined a future in which programmers create lambda-style functions that are essentially GPT prompts and pipe them to process data. As a programmer, you would essentially write tests to ensure prompts are working as expected.

Maybe this is the ultimate no-code programming environment: a fleet of cells, each one containing one prompt, parsing and storing data.

Maybe you do not even need to write tests: they are just additional cells with prompts aimed at generating fuzzy testing against logic cells.

From what I can tell, no one has figured out how to manage user-input into an AI. Users seem to be able to circumvent the original owners intentions. As long as this remains a problem I think it will need to be compiled.

As a professional developer, this recent news is really depressing. My livelihood appears set to evaporate.

Worry not. As long as our stakeholders are not able to formulate unambiguous, complete and correct specifications, we will always have a job.

Sure but you don't need a CS degree for that. Actually, CS hardly teaches you anything useful in that area at all.

The job you're describing exists already, it is the job description of a business analyst.

Idk about you, but my software development course (which was a req for my CS major) had us writing user stories, etc.

Indeed. In my view making informed trade-offs, communicating with stakeholders, and tracking down bugs based on limited information are hugely important job duties, and are entirely untouched by GPT3.

People have decried “no code” and “low code” in the same way, but I see tons of job postings for people wanting an expert to drive or refactor these systems for them.

At worst, GPT3 will be a tool that aids programmers, it won’t replace them.

Carl Öst Wilkens did something similar on Day 3: https://twitter.com/ostwilkens/status/1599026699999404033

Cool hacks, both of them.

Who's gotten ChatGPT to pontificate on the ethics or utility of people staying up late, using their mental energy to solve puzzles whose answers have no obvious utility? Would it then collapse into an introspective black hole, realizing its talents could be put elsewhere and becomes an EA maximalist, or finally conclude its very own power consumption could be put to better use powering ICU ventilators and power down?

Also! I sort of question the spirit of these two bots. After solving the problem, neither went on to create elaborate visualizations of the problem!

Waiting on web developer to get automated away. Every crud website feels like reinventing the wheel again and again.

Same with data engineering. Move data from point A to point B. Do these roll-ups.

Saw someone else on HN asking how people manage to do these in seconds. Ofcourse they used gpt3

This approach probably demolishes the take-home interview.

Now, if you can figure out a procedure that would work for a live coding test, does that just mean the human+AI pair programming team is a good candidate? I think this probably fails on more open-ended / complex problems. Maybe that means interviews cannot use toy problems any more, and have to be more complex, longer, more realistic work samples? (Sounds great to me.)

Why does this demolish the at-home coding interview but Google (or even email) does not?

Meanwhile, I can't make her do a script to open 4 equally sized tmux panes

Needs more training data. Either the model can't understand the documentation ... oh wow, I have never tried scripting tmux.

https://stackoverflow.com/questions/5447278/bash-scripts-wit...

This looks like the best solution to your question https://github.com/tmuxinator/tmuxinator

I'm doing a similar experiment manually, and it doesn't work so good — in my case, it got stuck on day 2 even then I started providing hints.

https://github.com/golergka/advent-of-code-2022-with-chat-gp...

For some reason, it is unable to solve this query. All code produced gives wrong results, and it didn't correct it when given clues:

Write a python program that efficiently computes the probability that a random permutation of r ones and w zeros contains k consecutive ones.

this is more of a pencil and paper problem rather than a numerical one though.

Something that I watch every year with AoC is the drop-off curve of completed days as the days go by. Typically, there is a super sharp dropoff in players at first, that kinda levels out near the end to just the people are are really in it for the long haul... In previous years only about 5% of the initial players are paying by Christmas Eve. But if tools like Copilot are actually useful in solving harder problems (not automatically like today's GPT solve, but as a tool for even experienced users to help flesh out ideas) then I wonder if that dropoff curve will change, allowing more people to go further than before. Somehow I don't think so.

From personal experience, the drop off isn't due to difficulty, but rather "free" time as we get deeper into the holiday season -- travel, family, disconnecting, etc.

The difficulty certainly plays a role in how much "free" time is needed to participate. If early problems can be solved with a quick 10 minute script, later problems take the top 100th person about 1-2 hours -- which means it takes mortals like me about 4-8 hours to solve.

That's quite the difference, and is due to the problem difficulty.

In other AoC there have been questions with little diagrams, or input with ASCII that forms essentially little boxes and matrices... I have a feeling that GPT will fail miserably at those type of questions and seeing essentially a 'picture' in ASCII on multiple lines is another level of abstraction.

I think GPT-solves would be curtailed by having a graphic in an actual image describing part of the instructions pictorially (in a way where even OCR won't help). That will stave off the AI for another year or two....

Super lame. I can see how trying to get an AI to solve a problem could be cool, but actually submitting the answers and claiming a high spot on the leaderboard ruins the fun for others.

I understand how some people don't care at all for the competitve aspect of AoC, but many of us do.

Would anyone trust the code by machine?

I wouldn't, but I can imagine a world where extensive randomized test suites are created by a specialized model from a text specification written by a human, and another model writes code to pass the tests with the given specification.

This still seems like it could leave open some holes. Wouldn't want to run this with financial, aerospace, medical or other critical applications.

Sounds like property based testing. Maybe the humans will tell the AI to write code according to some contract or signature. And then the human will also write the tests. But the AI will do the actual implementation.

Do you trust the code by humans?

A snark comment that is entirely on point. Well done :-)

What gpt-3 creates usually looks 'competent'.

Junior human devs usually leave a visible mess in the code, and errors are easy to spot. With the gpt-3 solutions to complex-ish problems, not so much, one needs to read line by line to spot super weird errors that no competent programmer would ever plant.

E.g.: I asked it to implement quicksort in x86 asm, but increment every sorted element by 5. It did nearly all of that right, except that it replaced every sorted element with 5. The code looked great otherwise, concise and even commented. I pointed out the mistake and it promptly agreed with me and fixed it. At this point I freked out and asked it to write about Dickens using adjectives that start with the letter 'p'. Which it (mostly) did. Good lord.

I wonder if it would find it if you asked: “There is a bug in that code. Could you explain the problem and fix it?” I’ve certainly had that happen in enough interviews, haha.

From what I was able to experiment, it is capable of doing so.

Wow, that's incredible! GPT-3 is an amazing tool and it's great to see it being used to create top-level solutions. It's amazing to see the power of machine learning and AI being used in this way. Great job!

Disclaimer: this comment was generated by beta.openai.com

Somehow it reminds me how advanced PCB makers still route by hand for high-frequency, radio or high-noise applications. But automated routers are still super useful for simple to medium jobs.

I got it to solve day 1 while playing with it yesterday, not shocked at all that this happened. Wonder if they'll try to prevent this somehow to preserve the competitive nature of AoC...

That's probably an impossible task. The best they can do is ask contestants nicely not to do this, but that opens the can of worms of whether tools like GitHub Copilot should not be allowed, either.

There are plenty of organizations out there that use leetcode to test for something. What are the testing for? Whatever it is it seems that AI has more of it than humans.

Why happens if we ask it to find security vulnerabilities in the linux kernel source, for example?

Reply on news.ycombinator.com