How's business at Stack Overflow? | Hacker News Reader

How's business at Stack Overflow?

jlericson.com

54 pointsjlericson62 comments

I will never understand why SO canceled 'Talent', their job board. We got by far the most good people in our team through it. If it was unprofitable, why not raise prices? I was actually surprised at how cheap 'Talent' was compared to other options. I can say without a doubt that we'd have gladly payed more (well, not "gladly", but it wouldn't have been a big issue). The UI/UX was really good, you could very efficiently track communication to hundreds of candidates and it had lots of good ideas integrated, clearly made by people who cared about this and knew a lot about how hiring software developers works. A lot of hours must have gone into this, it's a shame this is just gone now.

While 'Talent' technically still exists, just having a company page there does absolutely nothing and we canceled that one immediately.

It was good. SO seems to have very bad internal decision-making capability, at least since the founders left.

Seriously. SO jobs was great as a job seeker too, great interface, high quality companies and less ghosting/bs than other boards. I'm not looking forward to my next job search now that it's gone.

>I will never understand why SO canceled 'Talent', their job board.

From a related article: https://jlericson.com/2021/04/27/reading_tealeaves.html

"But since it was finally canceled well after the acquisition, I suspect it was no longer growing and, perhaps, unprofitable. I don't get the feeling that the new CEO was interested in this line of business and it could be that Talent was starved of resources until it was easy to eliminate."

Simple: It wasn't making money fast enough for the parent company's investors

Yes, pretty much what this article also says, but I find it hard to accept that as an answer. I can't imagine that this had a lot of maintenance cost. As far as I could see, the whole thing was excellent and pretty much finished, I can't remember ever having to consult anyone at SO for support. They could have tripled the price and we would have payed, because if you compare how much hires/$ we got out of SO compared to for instance LinkedIn, this still would have been a no-brainer.

I also wonder if they could have spun it off as a small, profitable (after price hikes and expense cutting), but non-growing company that existed to buy advertising space on StackOverflow. That might have been too big a distraction though.

It's just a shame that they destroyed what, to me, was the best job board out there.

Not to be too glib, but investors are looking for sure things and on the internet advertising is a "sure thing"

For these kinds of company deals where they don't have 20 years of consistent revenue with a consistent product, like Coca-Cola or Apple sales, investors are going to push the company into doing things that they can understand easily.

I think the line you're quoting is actually from the article we're commenting on?

Yeah same, for some specialized roles we looked for months and managed to snag an excellent person in just 2 weeks from SO's Talent ads. Literally the best eyeballs to put job ads in front of are the ones looking at the relevant problems. It's even nicely tagged by category, tech stack etc.

Plus one - I got my current job via stackoverflow quite soon before they scrapped it and found that it had a higher quality of (interesting) job listings than LinkedIn etc at the time. Was very surprised to see it go

I used Talent for a while. For the price I always felt it was fairly high touch from the customer success reps at SO.

I could easily see this making it unprofitable or barely profitable.

Agree though it was a great talent pool. I’d think there is still an opportunity there for something more self service.

A few years ago, working at a startup, I tried to convince our founder to pay for SO talent. IIRC, there was a minimum 6-12 month commitment, and while not crazy expensive, it was too much money to lay down on a lark. Since there was no way to 'try before you buy', there was no way to know in advance if the outlay would pay off.

It was a great experience for job hunters, but that value was not apparent to employers.

Compare this to LinkedIn, where you can post a job ad for free, and pay a relatively small amount to 'promote' your ad. Do this, and you immediately see the quantity and quality of applicants increase, so it's easy to make the case to pay, even if in the end you'd pay linked in more than SO talent's upfront costs.

That SO talent didn't have a self-service, pay-as-you-go option was a major miss, especially for an audience of tech companies, where this kind of business model is common, if not expected.

I'm a long time StackOverflow user.

A few days ago I had some confusion about how to use a React Native library and docs were also not super-clear for me.

Asked StackOverflow. Then I said "let's see what happens if I copy paste the exact same question into GPT-4", and did it.

Had my answer in 30 seconds, which explained the issue very well and worked. A few hours later I got an answer on SO too telling almost exactly the same, which was right.

Anectodal for sure, but ChatGPT and similar tools will keep evolving faster and will become only better.

Not surprising to see decline in SO. But I believe there will be an equilibrium as LLMs will need that training data too.

I've been learning to code for the past year. StackOverflow was hugely important in the beginning. I haven't used it in probably 2 months because ChatGPT does an excellent job of not only answering my questions, but also allowing me to ask follow up questions like "what does that specific line of code do?"

Ok, but where is ChatGPT getting that answer?

I think this is a major problem with LLMs. You often want to know where the answer came from not just so that you can verify the correctness, but also so you can do your own research, learn adjacent topics, and solidify your knowledge.

Had my answer in 30 seconds, which explained the issue very well and worked. A few hours later I got an answer on SO too telling almost exactly the same, which was right.

Wow, how strange...

If you imply the person copy pasted it from ChatGPT, it's probably not as the answer was from a very reputable user especially on that framework, and even if the essence was the same as the GPT answer, it wasn't in the same style.

Without the answers from SO, ChatGPT quality will decline as well. Wait 2-4 years for new tech come up, ChatGPT won't have the training data and noone is answering on SO anymore. You would just get wrong answers.

LLMs can navigate sources and generate both documentation and potential answers directly from sources.

Many questions on SO are exactly about things that are not covered by the docs of the tool/language/library in question.

The ChatGPT of 4 years from now will get its answers from the copilot-enabled editors of people who know how to do the thing. If we're lucky it'll also be paying those people royalties, but I'm not holding my breath.

How does it distinguish between people who know how to do the thing and people who use copilot, but are terrible at their jobs?

vscode is owned by microsoft and could send runtime results back.

so is github...

It gives bad advice to users, from which it learns that that advice didn't work, and only the good stuff gets used going forward (may take some time to retrain and incorporate user feedback). I expect a proliferation of advice that works well enough to convince you that your problem is solved, but which might be subtly broken in ways that you have to come back to.

Tracing the fix-it sessions backwards in time to the broke-it sessions (such that the AI can learn from its mistakes) might be tricky unless we somehow manage to connect copiloting sessions through git history. Imagine a big database of changes where the AI is waiting to see whether the code persists, or if somebody comes along and fixes it.

ChatGPT can answer the homework and lazy developer questions that comprise 99% of SO, so maybe SO will just have the hard & interesting questions.

One can hope, anyways.

ChatGPT can evolve. If it doesn't know the answer you could be identified as an "expert" in the field and you could be asked for an answer. This is how ChatGPT will help connect humanity with all our experts. Basically we'll all be training the model and getting information from the model. We're going to have to get serious about thinking through oversight so bad actors can't corrupt valid information.

This is wrong especially for programming where a program can execute a program. GPT-x will learn by itself from programming. No data training will be needed.

I’m most surprised with you getting an answer from SO, instead of getting your question shot down by overzealous mods

This works for now, but I wonder what will happen for brand new Framework X in 3 years time if people have stopped contributing to sites like SO. Where will ChatGPT get the knowledge to process and summarise and feedback to people? Blog pages?

As I understand it, someone needs to be creating the information that goes into the top of the funnel. I don't think ChatGPT can create an answer from scratch.

That's exactly why I wrote the last part.

It seems like there's an opportunity here for SO to use an LLM to provide an immediate response to questions, then allow the community to discuss, clarify, expand, and even correct the response.

This could even be done collaboratively with the LLM to improve its training with real expertise instead of feeding back into itself.

This would be a bit of a product shift for SO though, and some people would undoubtedly be opposed to it and leave the platform. But if LLMs pose a threat to the platform it may be better than to alternative.

Considering SO's corpus is liberally licensed, couldn't anyone take up this project?

If for-profit LLM's can be trained on the public content of for-profit Q&A sites without paying anything, then there won't be any public content of for-profit Q&A sites.

For-profit Q&A sites will have to either:

* try to restrict access (some sort of technical and/or legal barrier)

* shrink and become non-profit (Wikipedia style)

* fold

There are echoes in this conflict from the open source database vs cloud providers and licenses. It is the type of conflict we'll probably see more and more as different business models compete in the same global digital space.

I don't know if this SO CEO person is actually good at his job, but every time i try to read his blog posts, he comes across as a completely lobotomized corporate drone who speaks in nonsensical non-statements and is completely disconnected from the actual community of people who contribute to his platform. I have a suspicion that he might be interested in just pushing the Teams product and doesn't give a rat's ass about the public SO website. But, maybe I'm wrong and he just sucks at blogging.

>> Layoffs are what happens when a business unit is spending money like it's going out of style and needs to reign it in a bit.

Not necessarily. There are a number of other reasons e.g. revenues and profits are great but CEO wants even more $$$, or more charitably: strategies change and business units become genuinely obsolete.

I'm honestly not even trying to be snarky, btw. But one can't infer any more that layoffs means dwindling funds.

Agreed - my understanding of the recent layoffs at Google etc. is that the companies were making lots of money, but did layoffs anyway.

(And the stock price increased, as now they'll have fewer costs and, at least in the short term, therefore make even more profit.)

Stack Overflow? Haven't used it in years. It was awesome when it first came out and for a little while afterward, but soon the karma whores swooped in, the answers became poor, and questions that were actually pertinent and had good answers were marked 'dead.' I just stopped going. Haven't even heard anyone talk about Stack Overflow in some time.

I don't use StackOverflow anymore since ChatGPT launched.

It's kind of not fair because a large part of ChatGPT being able to answer code questions correctly comes from the SO corpus... but... well life's unjust.

I still like looking at SO, not for the answer per se, but to see all the other possible solutions. Even if they're wrong, I usually learn something. Either what not to do, or a neat trick I had not seen before.

User content on StackOverflow is licensed openly (CC-BY-SA), as it should be. So it's absolutely fair game for reuse.

I have a lot of sympathy with people whose all-rights-reserved content has been repackaged by LLMs and similar, but that's not StackOverflow. SO got free contributions from its users; the company didn't write them itself.

The problem isn't one of legality, but practicality. If SO becomes a ghost town, all LLMs suffer, including OpenAI's.

This is a perfect case study showing how it literally doesn't matter what your business is or what kind of community you've built, or operating practices you've developed; if the company has sold itself to investors, and investors decide "we want our exit" the company will turn into whatever it has to in order to "give investors their exit" and "return capital."

Lets look at an article that talks about how they got into this situation (https://jlericson.com/2021/04/27/reading_tealeaves.html)

Relevant quotes:

"We can never go back to the golden years of Stack Exchange and we probably don't want that anyway."

"This might seem odd until you understand the intended audience: investors. It's been over a decade since the first round of financing. At this point, private equity has invested more than $150 million. It's about time to find an exit strategy."

"Instead, Stack Overflow will position advertising as a way for companies to promote both their products and their perks for potential employees."

"Stack Overflow's investors want to get back their investments plus a healthy return. Let's say each investor wants at least 20% return on their investment per year."

"The VCs want a bit more than Microsoft is willing pay. So the company needs to start making it's case for a public offering. The only story that will sell is growth—ideally "hyper growth".

"I’m writing today to share that I’ve made the very difficult decision to reduce our workforce by about 10%, or 58 employees."

Lets sum up shall we:

We created this great product that people love. We then decided that we needed to grow really quickly and so we sold the future of the company to people who only care about money. We then sold the whole company outright to new investors (Prosus which is a "tech investment" firm) because the original investors are demanding a payout because "it's been long enough" apparently that they have not gotten their payout.

The new company then demanded growth, which led to over-hiring on the false bet that they would increase revenue via propaganda. That didn't work so we're breaking our agreement with 58 people that we were betting on (most likely without telling them specifically that their employment was conditional based on a hypothesis) in order to make sure investors do not get upset.

Sounds great, lets keep doing this.

Lol they also sold to Naspers. A company which started with slave trade as far away from programming and tech as one can hope for. If you sell to random companies who have blood money, shocked Pikachu face won't help you after.

The integration of Teams on Stack Overflow was a really weird thing. I do buy the argument from Jon Ericson in this post that integrating the internal Q&A into the Stack Overflow page is a large benefit for developers. I don't think this works for everyone, but having too many places where information might be that are not connected is a big problem.

They way they did that was really weird and in my opinion a big mistake. The account system used to be directly tied to Stack Overflow accounts. This had all kinds of problematic consequences, e.g. how to handle SO moderation for users that had a Teams account. I don't think this kind of integration can work.

I do think they went too far in the other direction, and some optional integration would be useful for people that use Stack Overflow anyway.

"By September 2022, Stack Overflow had 1 262 paying teams generating annual recurring revenue of US$50m"

so an average revenue of $40,000 per team.

I did not expect this.

The key word is "average". I'm assuming the former Enterprise customers (large companies like banks who pay for many, many users) are folded into the Teams product now. In any case, a few big customers will skew the average. I'd be interested in median, but that would be a lot less impressive in the annual report.

biggest danger for advertising revenue is ChatGPT. I visit it much less often since GPT. ChatGPT is just so much better at answering questions. I feel like I'm 10 times more productive with GPT. No more searching 1000 duplicate stack oveflow questions to find this one answer that accidentally refers to your specific use case. Long time ago I also used to contribute to site, but this was really emotionally challenging because of so many noob questions and general toxic community.

I guess SO has really bleak future both because of competition from GPT and contributor burnout.

Is there a future where people stop using Stack Overflow, so ChatGPT doesn't have answers to new programming issues and questions to train on, and so it is suddenly blind to generating new useful answers?

Absolutely. But there's no obvious mechanism to counter this.

It won't be such a night and day difference. LLMs can still train on official and unofficial docs (someone also mentioned that LLMs can read source code?). But the results will be worse in the absence of the additional Q&A style content, I would think.

> Business isn't great if you are cutting 10% of your workforce just 2 years after being acquired

> But across-the-board layoffs force managers to cut people who probably ought to be retained.

There are plenty of companies that aren't in trouble doing layoffs to shed dead-weight with the excuse that everyone is doing layoffs. Similarly, profitable companies are skipping raises this year bc everyone else is skipping raises.

Skipping raises, or giving raises significantly below the inflation rate, which is the same as a pay cut...

If the information in https://newsletter.pragmaticengineer.com/p/the-scoop-48 is correct, it wasn't an across the board layoff.

> About 30% of product development was let go and these techies were the largest affected group. Of the 58 people, about 40-50 of those let go were in the formerly 150-strong product development organization, I have confirmed with current software engineers at Stack Overflow... I have to wonder if StackOverflow has given up on innovation, and is looking to maximize profits while it still can.

I only saw this detail after I wrote the post. Here's a primary source: https://www.linkedin.com/posts/dalecook_opentowork-activity-...

We are going to see a lot of this sort of thing. It's like the internet all over again, stuff you used all the time starts to feel old and naive in comparison to new paradigm tools. What is surprising is quite how quickly this has happened, even 2 years ago SO would have been a solid go to, only game in town.

Quoting https://blog.codinghorror.com/introducing-stackoverflow-com/ (2008).

Stackoverflow is sort of like the anti-experts-exchange (minus the nausea-inducing sleaze and quasi-legal search engine gaming) meets wikipedia meets programming reddit.

Maybe a new new site for programming questions will come along.

SO is completely dead. GPT-4 is far superior in 95% of what I used to use SO for. I have over 50k karma on SO and my visit to that site has basically went from very regular to almost none while my productivity has ballooned from using AI. If I can short that company I would.

but if everyone stops using stackoverflow for questioning and answering, isn't a matter of time before also gpt won't have the answers? Of course this concept is applied to SO and every other website

Somewhat off-topic, but the article mentions Google docs not having a way to embed/format code nicely.

This is no longer true - you can insert a code "building block" that functions much as you'd expect, though it wasn't obvious until a coworker pointed it out

Ooooooh. You gotta install an add-in to make it work. Example: https://workspace.google.com/marketplace/app/code_blocks/100...

ChatGPT is replacing SO for me. Just had an experience yesterday where the first few answers were not quite what I needed (they weren't incorrect, just didn't meet my requirements), so the ability to ask follow-up questions to narrow down the answer resulted in a solution in minutes which could have taken hours of searching and testing. (Still had to test, of course, and one advantage of SO that ChatGPT doesn't have, is that you can be pretty sure a reply with lots of upvotes will work out of the box.)

Reply on news.ycombinator.com