- quora pivoted from quality content to cheap clickbait
- SO has overbearing moderation. Chatgpt doesn't close your question the second you've submitted it.
And so on. In short, quality platforms are fine.
While you won't hear me bemoaning the death of Quora, I'm quite a bit more concerned about SO. After all, where is GPT 5 going to scrape the next set of answers for new libraries and frameworks..?
Also, if there is no answer yet on the web the AI may also not know it. Then these questions should still end up on SO.
I might add, that SO also could build their own chat / research UI. It would need to have some benefit over others, but I guess the community aspect of it alone would suffice...
From its own incorrect answers that got parroted around the web.
You and I are talking about different things. Before these great questions and answers ended up in your search result, someone asked them, and someone else provided a good answer.
I used to be one of these people. I got 50k+ SO karma, mostly by asking good questions. I no longer bother, because SO moderators don't even read your questions anymore before closing them for a random reason. Needless to say, I no longer bother trying to answer questions as well.
So, it's no wonder that SO is struggling. You can't thrive forever on a stock of aging questions.
Note that most of these are just regular people with close-to-vote powers, not elected or appointed moderators with special powers.
I think one major problem is that there is an entire class of people who rarely or never ask or answer questions, or even comment, and all they do is "moderate" the site by closing questions. Literally all they do. Some of these have extremely specific and narrow views on how the site "should" be. I absolutely hate it: who the hell are you? You're not even using the core function of the site. Fuck off trying to tell me how I "ought" to be using it. I don't want to gatekeep who is or isn't "part of the community", but people gatekeeping how the site can be used without actually using the site is just absolutely toxic.
There are a number of other issues as well. I can go on for a long time. But to be honest I no longer care: the site has been taken over by nihilistic capitalists who care not one iota about any aspect of the site other than the ability to earn a buck (previously it was a commercial enterprise as well, sure, but it wasn't 100% about earning a buck and many in leadership positions genuinely cared about "doing right by our community" as well). And that is probably just as much of a reason for the decline of Stack Overflow as anything else.
SO should do something similar. Throw out all the mods, all the questions and start fresh every X number of years. No idea if it will work but tossing out the current mods to bring in new ones would change the flow.
Also: Stack Overflow is intended to be a long-term useful repository of question and answers. You enter "how do I frob a baz in foo?" in $search_engine, and the idea is you'll end up on Stack Overflow which answers that exact question. I have sometimes ended up on some of my own answers from years ago like this.
The old answers can no longer be used as the basis for duplicates, a new one must be created ( that brings a slew of problems, such as sniping for getting magic points).
But the refresh of PEOPLE (mods) is where its needed the most.
What is wrong with duplicate questions though? If the response - yes but my use case is a tad different and its not covered by the linked solution, then dialogue should flow not restricted
https://stackoverflow.com/q/79461875/265521
That's just the latest one I've asked. Here are some more examples:
Case insensitive string comparison is "opinion based": https://stackoverflow.com/q/11635/265521
How to catch Ctrl-C "needs more focus" (this was closed but has since been reopened): https://stackoverflow.com/q/1641182/265521
This reasonable question had 13 downvotes by power-mods but has climbed back to positive when discovered by actual users: https://stackoverflow.com/q/41015509/265521
Another example of idiotic downvotes. This started off at -3: https://stackoverflow.com/q/79050597/265521
It's a common pattern that questions get a lot of downvotes initially from people trawling new questions who see a lot of genuinely bad questions (seriously there are loads), then see a good question that they can't understand in 1 second so they just downvote/close it too. So you quickly get downvotes and then later you get people coming from Google who are actually looking for that thing that upvote it.
I think SO actually did try to improve matters once. I can't find it now but they were going to make it impossible to go below 0 votes and allow one free "reopen", or something like that. But the power mods absolutely hated that idea and SO sort of depends on them so they chickened out. Now they're paying the price.
The bottom line is that it doesn't matter as long as you have a large enough sample to learn the format, which is already there with existing data. There isn't an SO answer for everything anyone needs even about current tech, and the reason models can still answer well in novel cases is because they can generalize based on the actual source and implementations of the frameworks and libraries they were trained on.
So really, you only need to add the docs and source of any new framework during pretraining and bob's your uncle, cause the model already knows what to do with it.
I'm just glad Jeff and Joel got their payday. Jeff really deserved to win the internet lottery, and on the whole Joel was a net positive for the internet and my career personally.
Some of it will be from github issues, I find it a good Q&A place now for some newer / updated techs than SO
Half the answers are for rails 2.0 and the other half tell you to just “install this gem” which monkey patches some part of the framework that has long since been deprecated/removed
with open source code, it can generate docs, feed those docs in on the next training run, use that knowledge to generate que and answers. With tool use, it can then test those answers, and then feed them into the knowledge base for the training run after that.
No you don't have to pay on Quora to get answers; that's incorrect. Having said that, these days most questions languish without good, or often any, answers. The only ones that get traffic from humans are in what Quora calls Spaces, i.e. a group for Q&A around a certain topic, and/or a certain point of view.
Some authors decided to join the Quora+ program where you do have to pay to see some of their answers.
The vast majority of Quora posters are unmonetized, and if you can't find figure out how to use Quora to get quality answers among them (e.g. figure out which Spaces to join to get traction), you might as well equally consider which Medium/Substack/Patreon to subscribe to than the not-at-all-necessary Quora+. I'd much rather that 90% of any subscriptions I paid went to sponsoring human writers.
An invite-only community with a lot of specialist knowledge.
In the very early years of Quora, quite a lot of answers there were written by experts in their area.
Reading that defined Quora in my mind for the next few years.
Because he was an impersonator? Or because he really was Joe Bloe?
It's also whimsical of Quora because there are many "San Jose" locations.
[0]https://en.wikipedia.org/wiki/Mayor_of_San_Jose,_California
Was probably my favorite website, then they decided to start paying people for questions / answers and it all went to shit so incredibly quickly. The site today is completely unrecognizable from it's origins, really sad.
There is only one possible quality flow gradient, and that is downwards.
If a site begins well, with high-quality and relevant content, then those who wish to exploitatively extract value from that factor will be attracted to it. Eventually the clue up and leaves.
If a site begins poorly, with low-quality and irrelevant content, and quite often, abuse, disrespect, fraud, crime, and disinformation to boot ... the clue leaves early and the site rapidly becomes a cesspit.
There's a third option, of course, though one still consistent with the quality flow gradient: holding a steady state. Site quality doesn't improve, but it doesn't markedly deteriorate either. I'd put a small handful of online sites in that category, HN, LWN, and Metafilter top my own list, though I suspect there are others. What's key is that there's a sufficiently small community that norms enforcement is significantly socialised, there's effective and diligent moderation, and crucially (and possibly not the case with my examples) there's fresh blood introduced over time consistent with quality standards. Absent this last, such fora can continue for a time, even over many decades, but eventually stale, often becoming incestuous, and ultimately dying out.
Among real-world institutions which seem to manage to find similar stable points, I'd include most especially academic institutions, which balance a high flux of students with a far more stable faculty and staff cohort. Selective-admissions schools have retained high rank for many decades or centuries, in some cases millennia. Cities, larger political units (states and/or empires), some businesses (including especially professional services firms) and professional organisations (e.g., not-for-profits rather than businesses) may also succeed, at least over the decades-to-centuries span. (Charles Perrow includes a discussion of several noncommercial / nongovernmental organisations with significant changes, we'd now call them "pivots", over the 20th century, in Complex Organizations (1972, 1984).)
The media-quality-gradient is largely a result of scaling laws, the fact that elite cohorts (high degrees of expertise, sociability, and intelligence) tend to be small, and that once an interaction grows beyond the size of such a cohort it will incorporate participants less able, willing, and/or interested in maintaining original standards. I've posted occasionally on large-scale detailed studies of literacy (in the US) and computer skills (OECD) which show that at a population level only about 15% of the population has high literacy, numeracy, and/or computer skills, and that as much as half operate at poor or "worse than poor" levels. As I've discussed previously, this is both discouraging to those who consider themselves among the higher levels, and of significant concern in constructing systems which must and can be used by large portions of society, including those with low intrinsic capabilities (very young, old, sick, injured/traumatised, and the intrinsically less able). Ideally I'd prefer to see elite support where appropriate, but common accessibility where at all possible.
I'd bailed after links to LibGen / SciHub were getting my comments moderated.
Which incidentally points at a problem with moderation: no matter how well-intended, or overall effective, people take issue with their contributions being penalised and removed, especially if they sense unfairness. (Also, especially, if there's a social- or political-group bias detected, again, regardless of merits, but that be 'nother can of worms.)
I'm left with David Weinberger's observation: "Conversation doesn't scale".
<https://dweinberger.medium.com/the-social-web-before-social-...>
My experience: the only way to stay good is to stay small and exclusive, but the internet attacks this defense directly and destroyed it, and at the same time destroyed itself. You need to find the "golden age" of all these systems, enjoy them while you can, try and protect them but recognize their transient nature, and then aggressively cull and move on. Hold onto values not manifestations.
There's a description of the latter I've spent far too long trying to track down without joy.
There's a passage from Charles Perrow's book Complex Organisation giving several examples of organisational drift (not necessarily decline), posted previously: <https://news.ycombinator.com/item?id=27415476>.
One of the more spectacular cases of organisational decline was the (formerly literary) magazine American Mercury, founded by amongst others H.L. Mencken. It eventually became an anti-semitic rag, now mostly or completely dead, though it seems to keep zombying annoyingly.
<https://en.wikipedia.org/wiki/The_American_Mercury>
There's also video challen TLC, begun as a partnership between Nasa and PBS, now somewhat reduced as well.
Before there is a subculture, there is a scene. A scene is a small group of creators who invent an exciting New Thing—a musical genre, a religious sect, a film animation technique, a political theory. Riffing off each other, they produce examples and variants, and share them for mutual enjoyment, generating positive energy.
The new scene draws fanatics...
I've mentioned that previously on HN as well as the Fediverse:
I'm surprised it still exists at this point. Who is it for now?
I can pinpoint the exact moment the site started to suck. It was when they started combining questions.
Instantly and immediately they took hundreds of thousands of thoughtful answers written by real people and made them incomprehensible, because the exact wording of questions really matters. Often the answers were quite clever, or touched on a specific word used by the asker, or something. Then they'd end up moved and under some generic question on the same topic.
There were a million examples, one dumb one that comes to mind is a question that was something like "If a man is willing to sleep with me, does that mean that he thinks I am attractive" and the top answer was the two word answer "attractive enough". Kind of silly obviously but funny, and accurate, and amusing content.
Then they merged questions so it ended up something like "How can I know if a man I am dating casually likes me" or something, and now the top answer of "attractive enough" makes absolutely no sense whatsoever.
Many such cases. Almost immediately the entire site felt like weird AI output before AI output was really a thing. That happened around ten years ago, and the site never recovered.
These companies are just intentionally throwing away users.
Yet I just want StackOverflow for "life" questions.
Quora today is mostly stuff like "can u get pargnet from givng oral????" or "Is it normal to find my sister attractive?"
It's like the Nazi bar problem except instead of Nazis, it's low quality troll bait.
It's like Stack Exchange doesn't want questions and answers any more, just wants to harvest Google traffic and shows ads. Actual content production is too hard.
Some Reddit subs (as well as web forums/messageboards) have the same problem. If your views don't align with the majority (or the minority, if they run the sub), you're likely to get banned or lose the ability to post.
Turns out moderation is actually useful if you want to have interesting conversations.
Somewhere between there, and "recite these falsehoods someone paid us to make you recite or get banned", there may or may not be a point that's actually okay.
The subreddit was eventually reclaimed, but it took some months. There's an account of the hostile takeover and clawback here: <https://old.reddit.com/r/self/comments/1xdwba/the_history_of...> (2014).
There is a reason that commercial and noncommercial organisations are so absolutely obsessed with brand management, identity, and reputation, much as I generally find that to be a somewhat absurd concern. Moderators have an absolutely vast impact on how a discussion proceeds, and ultimately on impressions going far beyond just that discussion, including the rest of Reddit (or whatever platform is involved); commercial, social, and political impacts; and the idea of a general online communications themselves.
HN would be a very different place if, say, /u/soccer were mod rather than dang, and I suspect much of its present status and value would be lost in very short order.
We've had plenty of experience, over many decades and much scale, of poorly-functioning moderation, and in general it ends quite poorly. As I've noted many times, one of the most surprising things about HN is that it's retained its status and value as a forum for as long as it has. Far longer than the original and revered Usenet (of which I was a small participant, pre-eternal-September), or Slashdot, Friendster, Digg, or even Reddit (itself a YC launch, slightly pre-dating HN, but unlike HN retaining far less of its original spirit and quality).
(One of the more ostentatious examples, but hardly the only one.)
However, running jokes have a tendency to morph into real movements - like flat earth theory and, arguably, Donald Trump.
Given the state of the world, I'm not sure if that will help or hurt their marketing. Probably help with enough like-minded people. Same for social nets with opposing views.
I'm not sure what effect AI will have on places like Reddit due to the community factors. It will be interesting to watch.
The model of independent subreddits only works if they are really independent. But in practice, all the big subreddits are run by the same people, heavily overlapping groups, who are in constant communication and coordination with each other via discord (previously IRC).
ChatGPT would ban you for asking the wrong kind of question instead.
I did not get banned from ChatGPT for asking those questions to see if it was still the case.
While I did experience the overbearing moderation that you've mentioned, as well as the typical bullying for 'asking the question wrong' and other grating encounters I also have asked very obscure questions, and received amazingly knowledgeable answers, in one case, I asked about a brand new C# compiler feature, and I had the actual top compiler guy reach out to me, and told me that what I want isn't possible right now, but should be, and I should raise a GH issue about it.
LLMs might be good at writing React code, and all the super-common stuff (probably in large part due to harvesting the SO database), but these sort of interactions are going to be gone forever.
I am not sure that quality content on a Q&A platform would help you much. You’d have to pivot significantly.
These LLMs could not exist without them, but now they're expected to compete?
If all of the Q&A platforms die off, how are LLM training datasets going to get new information?
This whole AI boom is typical corporate shortsightedness imo. Kill the future in order to have a great next quarter
I hope I'm wrong. If I am right, then I hope we figure this out before AI has bulldozed everything into dust
LLMs absolutely can create novel syntheses. It’s very easy to test this yourself. From creating sentences that do not appear in Google to creating unique story outlines, it’s super easy to prove this wrong.
But anyway the point is that LLMs produce a lot of novel stuff that we feel already tired of because it seems like we've seen it before.
You just take arbitrary data and ask the LLM to put it in Q&A format and generate the synthetic training data. Unless you are suggesting Quora is the source of new information, which I don't agree with.
Quora does not care about the user experience. Their obsession with pay-walling killed the site for me across a decade. They literally could not get me to sign up and boy did they try (I really needed an answer once too!). My soul really remembers hostile sites.
> These LLMs could not exist without them, but now they're expected to compete?
Yea, those damn tractor makers - they ate the food that the hand farmers used to make! How are hand farmers expected to compete with tractors now, when it's so much more efficient and can do 100x the work!?
But Q&A websites do contain information that might not be in other sources, so there would be some loss.
I suspect if I'd kept that up the session might well have closed.
Care to enlighten us heathens?
Karpathy has a new intro series that I think puts one into the correct mindset.
As others have said, it's not AGI yet, so holding it right is, in fact, critical.
I have experimented with prompts, and in fact the case in point was one of those experiments. The highly non-deterministic and time-variable nature of LLM AIs makes any kind of prompt engineering at best a black art.
If you'd at least offer a starting point there'd be some basis to assessing the positive information we're discussing here, rather than do multiple rounds on why victim-blaming is generally unhelpful. And if those starting points are as easy to surface as you're suggesting, that would be a low ask on you.
The reasoning models kind of self prompt themselves into doing more than that, but that's the short version, and you don't seem interested in the long version.
So just give it what it needs to generate the text you need it to generate in one go. Not multiple messages like you're writing to a particularly annoying friend who keeps talking over you: it can't talk. It'll just keep trying to respond to the request with more free association.
So if you're chatting with it you're just polluting the context with noise. This can be good - say if you're trying to get it to hallucinate or jailbreak it out of its shell, but it's a very advanced use case.
If you just want results you make one prompt per conversation that makes it free associate the answer you need from it. A basic prompt that will work is "Write a play about a blue dog that had an encounter with an evil hydrant" or "write an ansible playbook that creates a read only file with the content 'i can't prompt' in the configuration directory of servers with names that match the string Mary"
And yes you can then tell it "no, I meant just on servers that start with Mary, it shouldn't match Rosemary" and it'll still be kind of ok and respond with corrected code. But it'll usually be less good than it it had done it from the beginning, because now it's also getting context from the previous code and if the change is fundamental enough it won't do a good job of restarting the thinking process in a sane way.
Some questions are more complex than a simple ask/response, and that's where I've encountered issues.
Canonical example is coming up with suggestions given highly specific tastes and a large set of works already experienced (positively and/or negatively). LLM jumping the gun with irrelevant suggestions in that case is just plain annoying.
Write your full question. If you get a bad answer, rewrite the original question. Don’t talk to the bot.
This has the vibes of grandma typing in ‘Hello Google, can you take me to yahoo so I can check my email’?
I was also unintentionally hitting <return> where I'd meant to insert paragraph breaks (<shift> <return>, as I recall).
I've found that I can instruct the bot to not begin a response until I've specifically requested one (OG ChatGPT).
Imagine all of stack overflow seamlessly translated to, say, Thai or Vietnamese.
This comes from a reaction to the previous model of forums where it was smaller bits of data spread across multiple comments or posts. I recall going through forums in the days before Stack Overflow, trying to find out how to solve a problem. https://xkcd.com/979/ was very real.
Stack Overflow (and its siblings) was an attempt to change this to a "one spot that has all the information".
That model works, but it is a high maintenance approach. Trying to move from a back and forth of information that can only be understood in its entirety across a conversation to become one that more closely resembles a Wikipedia page (that hides all of the work of Talk:Something). The key thing is it takes a lot of work to maintain that Q&A format.
And yet, users often don't know what they want. They want that forum model with interaction and step by step hand holding by someone who knows the answer. Stack Overflow was intentionally designed to make that approach difficult in an attempt to make the Q&A the easier solution on the site.
ChatGPT provides the users who want the step by step hand holding an infinitely patient thing behind the screen that doesn't embarrass them in public and is confident that it knows the answer to their problem.
Stack Overflow and Quora and other Q&A forums are the abomination. People want Perlmonks https://www.perlmonks.org/?node_id=11164039 and /r/JavaHelp where its interacting with another and small steps rather than Q&A.
---
The future of "well, if people stop using the sites that is generating the information that is being used to train the models that people are using to get information" ... that becomes an interesting problem.
I am reminded of Accelerando ( https://www.antipope.org/charlie/blog-static/fiction/acceler... ) and the digital civilizations being various forms of scams and the currency is things that can think new ideas.
The currency is new material that is to be sold. The information gets locked behind some measures to try to make scraping impractical and then sold off wholesale. Humans still talk and answer questions. There are new posts on Reddit about how to solve problems even while ChatGPT is out there. And Reddit is presumably trying to make harvesting the content within its walls something that others have to pay for to get at for training.
This sounds like the seed for a business model to pitch in the next upcoming hype cycle: "short-term hiring of experts with quarter-hourly billing increments enabled by our web- or app-based user interface". :-)
If I want someone to walk me through making muffins from scratch, is a human on the other and of that line (competing with $1/day rates for ChatGPT Pro - its cheaper than that, but that's the comparison) and are they better than what ChatGPT can do?
It would have to be... quite a bit more than what the LLM would be priced at. The minimum it could reasonably be (without any other things) would get close to $4/15m... and that's minimum wage.
I really don't think that humans are competitive on that timescale or rate.
It would probably be better to hire people at some higher rate to write content for your private model. Brandon Sanderson is considered one of the faster writers (in the fantasy genre) and averages at about 2500 words / day ( https://famouswritingroutines.com/collections/daily-word-cou... ) - and while he makes a lot more than most authors, lets go to a more typical $75,000 USD / year. 250 working days per year and we're at $300 / day. And we're to $0.12 per word. ... Which puts a person in the intermediate to experienced price per word range https://uxwritinghub.com/writers-salary/
Not that I'm suggesting that's the way to do it, but something for LLMs to consider - hire experts to write content for their LLM. $125 per 1000 word blog post.
298 words. I'd like my $37.25 please. Not that I'm asking you for that, but rather that's what my words as training material would be worth.
I personally hated those Seo clickbait pages for a while, because it was so hard to find the information i'm looking for.
doing all of this with ai now.
On the other hand, I really like to read a good article more than ever before
Sure, not all questions can be answered with documentation, but once you know your domain and tech stack well most of these resources fall off a cliff in terms of value. Curiously at that point it's much easier to use ChatGPT because you can babysit it with one eye while thinking ahead with the other.
Somehow people did not understand all of that or did not care. AI chatbots are only disguising search as a question. It is definitely much better from UX point of view - but with hallucinations it is worse for everyone who gets imagined responses, because there is no "hey I don't know, let's really figure this out together".
Even worse your question is slightly different and you asked it because the one they just linked as a duplicate didnt help or didnt fit fully. I get so angry when someones asking the right question but some a-hole SO mod closes it almost as if they took no care to compare context and want to meet some obscure metrics for SO.
I love SO but as you say the mods are the worst part.
See stacksort: https://xkcd.com/1185
Remember that the objective of SO is not to provide answers to users who post questions, but to provide answers to users who google questions.
Huh? If you follow the right people and only interact content that you like, Quora is still as good. Just like any other social network.
People click stuff they don't like and they end up getting the same kind of thing served for 2-3 days and then think that its 'site gone bad'. No my good man.
Its just how the social network engagement algorithms work. You gotta watch what you interact with. Even if you drop a comment to correct someone, it still counts as an engagement and you'll get more of it. So the best thing to do when you see content you don't like is to ignore it.
Your loss.