OverflowAI
stackoverflow.blog
stackoverflow.blog
> The future of collective knowledge sharing.
The future is feeding all answers into a meat grinder and then forming word like nuggets from the paste and hope we like this? I am sorry but you lost your mind. There is a very fundamental disconnect between LLM hype and reality.
Here is my latest experiment, I needed to search for a string in dangling commits and asked Bing: https://i.imgur.com/pQtpkH5.png You will notice my question was a link because I copied the title from https://stackoverflow.com/q/21903972/308851 and apparently that made it into a link (I had the question open because every time I need to do this I just jump to this question, I outsourced storing my knowledge to SO ages ago). Every single answer given by Bing is wrong, of course, because git log -g is not appropriate for this as the answer states. It's right there: "Commits that are (still) referenced by the reflog are considered reachable and not dangling". And here's the best part: the Learn More 1. Stackoverflow.com links to this very question. The language model have seen "git log -g" in the vicinity of the string "Search for string in dangling commits in Git" and thought it's appropriate. The irony.
There can be value in SO answers, there can not be in LLM as it's just automated plagiarism without understanding anything.
Then it becomes a self-fulfilling prophecy: a) "ChatGPT is stupid! Also, plagiarism! No value!" b) " I wrote a carelessly bad prompt and it responded with a seemingly bad answer!" c) " This confirms my biases!"
I find that when I provide it with enough context and a set of requirements (as I would with a non-AI engineer), it's very good at problem solving.
I’m not sure understanding is an inherent issue either. When I Google ”weather lisbon,” Google spits out a number. The app doesn’t ”understand” anything, but it still helps me plan my day.
I'm old enough to remember the memes campaigning to rename "software engineering" to "googling StackOverflow", so plagiarism isn't a point against the value of an LLM, and given the case of an LLM made by StackOverflow using StackOverflow data it's also not a legal issue.
Conjecture. We lack a mechanistic understanding of "understanding", so you can't make this claim. Human answers are also fallible, so how do you know that human cognition isn't also statistical pattern matching similar to what's used in LLMs? If it is, then they are understanding, their understanding just has flaws.
That's like saying "reading the future from chicken entrails is a skill".
Accuracy rates much, much higher than chance are not accidental. While you're pointing to a valid failure mode of LLMs, you're not using the correct words to describe it which is leading you to say nonsense.
If I could read the future from chicken entrails with as high a reply accuracy as I get from ChatGPT, then I would fully endorse that too.
You’re basically saying that the signal to noise ratio is zero so its utility is zero. I guess you must think all of the comments here where someone mentioned how it helped them solve a problem, also have a 0 (or at least dismissible) signal to noise ratio?
What signal to noise would be needed for you to say it’s not devoid of any information? What would the minimum accuracy need to be? 80%? 95? 99? 100?
The thing is there are many questions where it is a short path to a solution that can then be verified. The most useful way to use it productively is to induce it to give you pointers to things you had no idea existed. Then dereference those pointers via traditional means (ie google, man pages). Be your own null ptr (hallucinations) segfault handler. Critical thinking a hundred percent preserved.
It’s a little bemusing how people take issue with next token prediction trained with a log likelihood loss. In particular they think that getting a high accuracy extremely low perplexity LLM is unimpressive because of the low accuracy high perplexity autocorrect on their phone. No- the llm’s emergent generalization capabilities as nll drops really is a capital p Phenomenon.
But I feel that I don’t need to try to change anyone’s mind anymore. Everyone had a threshold logp where they’ll change their mind about this, whether it’s attained in the next week or long from now. Just state the nonzero perplexity that you would accept from a person thinking, and wait for the loss curve to meet it.
As always, the ones most vocal about it haven't even bothered trying out guanaco 70b 2bit.
Probably don't know what ggml, gptq, cformers or any of the others are.
They still base their assumptions on the openai gold standard.
Was: Dangling commits in Git are often referred to as commits that are no longer reachable from any branch, tag, or the HEAD. These commits usually occur when you create a commit and then move the branch pointer away from it without creating a new branch or tag. They can also occur during rebases or other history rewriting operations.
Git has a garbage collector (`git gc`) that will periodically clean up these dangling commits, but until that happens, they're still in your repository and can be examined.
Unfortunately, Git doesn't provide a direct command to search for a string in dangling commits. However, you can find these commits using the `git fsck` command, which checks the filesystem for errors and dangling commits.
Here is a way to do it manually:
1. List all the dangling commits using `git fsck --lost-found`. This will print out all the dangling commits and store them in `.git/lost-found/commit/`.
2. For each dangling commit, you can search for a specific string using `git log -p <commit-id>` or `git show <commit-id>` and then pipe the output to `grep`.
Here's an example of what this might look like in a script:
mkdir -p .git/lost-found
git fsck --lost-found
for commit in $(ls .git/lost-found/commit/); do
git show $commit | grep -q "your string here" && echo "Found in commit $commit"
done
Remember to replace "your string here" with the string you are searching for.
*NOTE*: This operation could be time-consuming if you have a lot of dangling commits or a large repository. Also, this approach might not work if the garbage collector has already cleaned up the commits. Make sure to be careful when manipulating Git's internals like this, as it could potentially corrupt your repository if done incorrectly.
>Integrating AI with code development is something that Microsoft has been doing for several years with its Github Copilot technology. Chandrasekar said that Overflow AI is not an attempt to replace Github Copilot, but rather to just provide more information resources to developers and the organizations they work for.
>“We’re certainly not looking to replace GitHub copilot, we’re very complementary to what you’re actually going to do writing code,” he said. “You need a really solid foundation on Stack Overflow for Teams to give accurate, validated and curated information.”
I really do not understand what that means.
EDIT at time of writing, link pointed to a Youtube video, now it's a text announcement, so I am gonna go read that.
Leveraging synergies to provide value to key stakeholders, through community engagement and partnerships with industry thought leaders.
The more buzzwords there are in the text, the less I understand the text.
What he's saying is that they're going to focus on delivering answers that only an internal knowledge base can supply. This would be complementary to co pilot which can only give you generalized answers.
Maybe its all unavoidable, but I don't want it.
I don't foresee an AI being able to replace an engineer for real production-grade custom software for a long time
At the beginning of the devlopment of computer chess, people said it would NEVER beat a reasonable player. Then it did. Then they said it would NEVER beat a seasoned tournament player, and it did. They said it would NEVER beat an international master or grandmaster...but it did.
People keep saying never. Our society can never fall...but societies have fallen since the dawn of time. You might be safe in your job, but what about your children or grandchildren?
Yeah, it's gonna suck for us, but it's gonna suck more for a lot of other jobs a lot sooner.
I guess the modern answer from a globalist is that back in the day, societies weren't as entangled as they are today, so they could go into freefall relatively easily with little impact by other societies, so they let them fall, especially if it was beneficial for them.
But today everything is so interconnected that if one fall, it would heavily impact a large part of the rest of the societies, that we cannot let it happen. Just look at how unimportant most people would consider Ukraine to be, before the invasion. But it has a huge impact on large sectors, and most countries would have this impact. Some areas of the world, like the Eurozone, even purposefully lock themselves in with their partners so if Greece gets impacted by something, member states have to help them as it'll impact them greatly if they don't.
So I guess my end argument is that there isn't really "societies" anymore, we're just part of a global one where the decentralized nature is so strong that we cannot really fail (globally) anymore unless there is something really big impacting all of us (like COVID).
Worse: thanks to psychology, even if a time traveller came back to now with such a list (and proof of being a time traveler), their examples would not feel real to most until they could be experienced directly — I knew that GPT-3 could code simple web pages just over two years[0] before ChatGPT came out, but it still didn't hit me emotionally until we were all able to play with it ourselves.
[0] August 8, 2020: https://youtu.be/_x9AwxfjxvE
Yes, and?
I mean, you clearly get what you put in the next sentence:
> "AI" is just a broad, abstract category that they both happen to share.
But, there's also no requirement for future chat bots to be Transformer models like GPT.
All we know for sure is that this blob of weights seems to be pretty good at a lot of different things that used to be things that only humans could do.
Thus, I think as another demonstration of "we used to use this as a proxy for intelligence and now we can't" it's fine to use chess as an analogy because it demonstrates exactly that thing.
No idea what comes next, nor when — it's still possible that "solving" general intelligence is beyond human capacity.
If you think your job is just writing code, then yeah, you're about to get replaced. But if that was the case, then you would already have been replaced by an outsourced team of developers that can get things done cheaper than you.
We as developers are the conduit between what the business needs and what computers can do. We're translators, and not in the "we write code" sense. A non-developer won't be able to use GPT to write any non-trivial system because they won't have a clue what to ask for.
And if someone thinks "oh but they will one day be able to list their requirements exactly and GPT will just do it", I will ask: Well, why hasn't that taken off with offshore teams, then? The answer to that is exactly the answer to why GPT poses no risk to anyone's career: Stakeholders don't know what they need, and it's up to developers to figure it out.
Honestly, even that would require major advancements, which is speculative at best. The last time I used Copilot to generate a few lines of code, there were three different mistakes. I spent much more time debugging than it would have taken me to just write the code myself.
It can do some things very well, but it fails all the time.
For the foreseeable future, I think we're firmly in the "this is just a tool" territory.
The BLS data has some insights into how the government regarded different job titles over the years.
The current one:
https://www.bls.gov/oes/current/oes_stru.htm#15-0000
has "Computer Programmers" https://www.bls.gov/oes/current/oes151251.htm
> Create, modify, and test the code and scripts that allow computer applications to run. Work from specifications drawn up by software and web developers or other individuals. May develop and write computer programs to store, locate, and retrieve specific documents, data, and information.
and "Software Developers" https://www.bls.gov/oes/current/oes151252.htm
> Research, design, and develop computer and network software or specialized utility programs. Analyze user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis. Update software or enhance existing software capabilities. May work with computer hardware engineers to integrate hardware and software systems, and develop specifications and performance requirements. May maintain databases within an application area, working individually or coordinating database development as part of a team.
---
This seems to match your views too.
2021:
https://web.archive.org/web/20210318001704/https://www.bls.g...
15-1251 Computer Programmers
15-1256 Software Developers and Software Quality Assurance Analysts and Testers
15-1257 Web Developers and Digital Interface Designers
Had
15-1131 Computer Programmers
15-1132 Software Developers, Applications
15-1133 Software Developers, Systems Software
15-1134 Web Developers
15-1021 Computer Programmers
15-1031 Computer Software Engineers, Applications
15-1032 Computer Software Engineers, Systems Software
Yes, generating new code is a fairly small part of my job too. But this is also a pretty common take re ai.
Maybe I'm wrong, but generative ai seems to be eating human capabilities for breakfast right now, and we're only ~8 months in. Today it can do a small part of a small part of my job, and rather more of a junior dev's job, but where are we going to be in five years time? I'd quite like to still have a paying job over that timescale.
People used to think that creating art or music or playing go or whatever, were activities that required a human meatbrain. Turns out its all just (effectively) a bunch of parametric equations. What if the non-coding skills you list will be similarly amenable/vulnerable to automation by ai?
One of these things is not like the others
(I'm not actually sure which of those three you were singling out and why; they're all different in their own ways).
There's another effect, especially with regard to information synthesis via AI: previously the vast majority of information was obtained with people's names attached to them. All Stackoverflow answers have names attached to them, and articles on the internet too.
Now, AI bypasses that, making information gathering more anonymous and less human. I already remarked in comments that information sharing has always had two functions in human history: bonding humans together and trading useful information. AI removes the bonding aspect and creates a pathological environment where information sharing is too anonymous.
Your comment is one of the first I've seen that actually talks about a longer-term effect of AI, but AI programmers and companies are too bedazzled by the promise of short-term financial gain to give one care about ethics, except for mouth-service.
Though we do need to prepare for AI replacement, I think we'll be much more successful pushing for things like UBI than pushing against things like AI assistants.
> Previously the vast majority of information was obtained with people's names attached to them. All Stackoverflow answers have names attached to them, and articles on the internet too.
> Information sharing has always had two functions in human history: bonding humans together and trading useful information
I'm sorry but I disagree. When I see StackOverflow answers or random articles I rarely remember the associated name, unless they're especially notable (e.g. Eric Lippert). And I'm not sure about the vast majority, lots of Stack Overflow and Reddit answers are under pseudonyms.
I'm in academia. I believe that when people contribute to the pool of knowledge, the knowledge comes first and their credit comes second; not to be forgotten, but not something most people need to know. When applying for jobs and bragging to people I can show off that I discovered or invented X, but people learning or using X don't need to know my name.
I don't think most people asking technical questions are looking for "bonding", they're just looking for an answer to the question. I talk with colleagues about things like opinions and the latest hot topic, but AI isn't going to replace these discussions.
That is already an unfortunate side effect of our current advanced state of technology.
> I don't think most people asking technical questions are looking for "bonding", they're just looking for an answer to the question. I talk with colleagues about things like opinions and the latest hot topic, but AI isn't going to replace these discussions.
I never said they are looking for it. I just said that it is a function of information exchange. In other words, it is an evolutionary instinct that just occurs, and has occurred whether we want it to or not. AI for the first time will be poised to remove it.
AI might indeed replace your colleagues one day, since an entire academic department may not even need that many professors or researchers to run it, once it's augmented with AI.
I don't know where you've been this past year, but hand-wringing about AI replacing humans in the long term has been everywhere. It's gotten less common in recent months as the limitations of the models become apparent, but it's still prevalent.
With all of the money and effort put into GPT-4, and now all of the attention from 3rd party devs, we have very little to show for it? At best we have something that improves productivity by a few percent. It’s..ok. The more you use it the more you realize it’s very flawed.
I will never completely discount that future models will be fundamentally different, but I’d put the chances of that happening in the next 5 years very low, despite what we’re hearing from certain industry folks.
As for me, I'd put that at more likely than not.
If we are talking about specific workflows, I would agree with you that this is quite likely to occur.
Given all the people worried about AI safety issues[1], I agree it's likely we might not see any substantial quality[2] improvements for five years.
Even without that, there's no way to tell how close we are to a "solution" to the question of intelligence; it may well be that LLMs are to AGI what all the self-driving car startups you've never heard of are to SAE's Level 5.
[1] alignment: this entire area is described as "pre-paradigmatic" by the actual researchers 6 months ago
privacy preservation: anything that might come under GDPR
audience suitability: e.g. no adult content for minors
[2] speed/efficiency yes, but scoring similarly on the tests.
Seriously, try to find something relevant say for Ionic Framework , and most of the answers in StackOverflow are outdated garbage that was only relevant for Angular based Ionic.
The first thing StackOverflow should use AI for is to clean up those "hundreds of thousands of answers" so that the irrelevant things are removed. Right now StackOverflow feels like the library's "Technical Book" aisles around the 2000s, with tons of books about "Office 97", "Visual Basic 6", "QBasic by Example", and similar obsolete books.
Do we think that this launching a mere 2 days after the infamous Fall Of StackOverflow post is a coincidence?
I read something recent that the SO revenue is dropping and was showing signs of this before the recent AI push. In my opinion, this is because video is king with younger people and SO is geared towards people with beginner questions. You can find a huge amount of sw dev instructional content on youtube.
Yes, because it is harder to copy/paste, and set up as your own page.
Bit what a way to learn! 10 minute video vs 10 seconds of reading.
This is also noteworthy [1]:
> we're using AI to take existing knowledge sources (confluence, github, google drive etc) and provide an initial tagging framework and a series of questions and answers from them
What for? Presumably an LLM trained on a company's internal data, but with higher start up cost (time taken for initial tagging).
People asking questions will be angry because the bot okayed the question, why did the mods close it without discussion?
Mods will be angry because why did the bot okay stupid questions?
And it doesn't even go into the meat of the problem - stack overflow is supposed to allow everyone, including its competitors, free access to all the questions and answers. The questions and answers belong to the users, not to the CEO of stack overflow.
My conspiracy theory is Joel Spolsky saw the writing on the wall long ago:
> The company has been growing, too. Today we are profitable. We have almost 300 amazing employees worldwide and booked $70m in revenue last year. We have talent, advertising, and software products. The SaaS products (Stack Overflow for Teams and Enterprise) are growing at 200% a year. That speaks to the fact that we’ve recruited an incredibly talented team that has produced such fantastic results.
https://www.joelonsoftware.com/2019/03/28/the-next-ceo-of-st...
I think this hn comment summarizes it best
https://news.ycombinator.com/item?id=36890672
--
> In May I wrote about Stack Overflow's business, which lost $42 million over 6 months and had just laid off 10% of its employees. Since then, the company's fiscal year-end results came out. Despite growing revenue, it lost $84 million over the year ending on March 31, 2023.
Thank god Wikipedia isn’t run like Stack Overflow. As an end user, they have pretty much the same value proposition: user generated answers to my questions. Wikipedia is still doing well, meanwhile it seems SO is constantly being driven off a cliff by bimbos in management.
Not everything needs to be a damn unicorn. SO is an information repository. They need to accept that stop trying to “enhance” it with more crap because they don’t realize their median user is a junior dev who really just needs to serialize a Java object and isn’t going to pay or put up with any LLM-generated nonsense.
SO doesn’t need large language models. What they really need is a better model of what answers are good, what answers are outdated, and what answers should be expanded to include more info (and sometimes, what answers should be slimmed down a bit). Turn the top answer to popular questions into a wiki so that everyone can update it. And then add backlinks for questions which were closed for being “duplicates”. It solves so many problems SO has.
Another thing. This “comments aren’t for extended discussion” nonsense needs to go too. Any question could easily include a Reddit-style discussion tab to facilitate discussion. I’m sure much of it would be at least as valuable as the answers themselves.
Sure, but companies like Stack Overflow is set up in a way where they have to at least aim to be Unicorns, as people have invested tons of money in it with the expectation (hope) that it'll pay back at least 10x/100x, otherwise they probably wouldn't invest in it in the first place.
Contrast that to Wikipedia/Wikimedia which is a foundation funded by donations rather than "investments", and the difference between the two almost couldn't be larger, just because of that.
The Copilot Chat beta is able to answer this question by itself in VS Code, though not in Visual Studio.
sorry I couldn't help myself.