StackOverflow Is Down | Hacker News Reader

StackOverflow Is Down

stackstatus.net

59 pointsagluszak52 comments

Open articleView on HN

SE has been an incredibly well ran company.

- FREE

- Permissive content license

- Doesn't require me to log in to read an answer

- Few / limited ads

- Data available at archive.org

- Still independently owned. Imagine AWS or Microsoft buying it. Oof.

I think about LinkedIn, Github, ... the other large platforms we use for work, and SE is the only one that hasn't gone downhill.

No, I don't want them to open source the platform. No, I don't think you can do better. Yes, a few mirrors for redundancy would be excellent, but not urgent. Discoverability here would be the challenge for mirrors. They've been down less than a day total for me in 10+ years of using them.

> "Still independently owned. Imagine AWS or Microsoft buying it. Oof."

Owned by a VC firm, isn't it? https://www.forbes.com/sites/vijaygurbaxani/2021/06/08/the-1...

after https://vcnewsdaily.com/stack-overflow/venture-capital-fundi...

You're right. That makes me very sad.

> Doesn't require me to log in to read an answer

The cookie dialog being different on every site is infuriating if you’re not logged in.

I hope they are financially well because it is one of the few online things I think it works.

it is only bad they are having some many outages these days. I am not complaining for me. but it means something is happening there

And this is at least the 4th or 5th time within 10 days that either the website is down or performance is badly degraded.

So bad for a website that is basically the go-to resource where IT professionals get their answers.

So bad that the whole platform is closed-source, so we can't even spin multiple instances.

The main resource used by developers and sysadmins around the world is locked and centralized, and when that central place goes down all the answers go down with it.

We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them.

And, most of all, we need alternatives that are open and decentralized, and we need them right now.

If you use brave browser it usually is able to point you to corresponding web archive pages when SO is down. Nice to have it directly integrated into the browser

I think stack exchange provides data dumps via archive.org

https://archive.org/download/stackexchange

> we'll do the scraping for them

Awesome! Let us know when we can start using your new open source service, it must be mirrored and need it right now.

Stackoverflow's data is cc-by-sa 4.0 licensed and can be found here: https://archive.org/details/stackexchange

So nothing is stopping anyone to spin-up multiple instances.

It's much harder to work with the data if you don't have the code. SO is not open source.

https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-...

not so sure about that

That's kind of cool, thanks. How long does it take to do a web scrape versus a DB dump?

Here's a working link to Kiwix since the link on the page 404s: https://www.kiwix.org/en/download/

IMO it would be more interesting to not use their code and build some kind of specialized search service. One that learns to rank results from user metrics and feedback?

so like personalised google search, with the terrible results that that inevitably produces for code related content?

i am confused though. how do you think SO currently works? Both questions and answers are already ranked via user metrics (individual upvotes and historical upvotes) and feedback (comments)?

StackExchange is by no means perfect, but the copyleft content license is a key reason I feel comfortable contributing to the platform. (Which I assume is why they do it.)

> We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them.

I think that's already the case. Whenever I look up a technical issue, the first result is usually from SO; the next results are usually from websites that copied that first SO page

You're lucky, for me recently its the other way around, spam copies first.

FWIW, the Codidact project has been plugging away at building an OSS replacement for SE:

https://www.codidact.org/

A read-only, self-hostable copy of StackOverflow (and its sister sites) is already doable with Kiwix + the content packages they publish.

Content packages: https://wiki.kiwix.org/wiki/Content_in_all_languages

Example of how a site looks like (in this case SuperUser), hosted on my server: https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/inde...

And hosting it really isn't all that difficult: https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/

I’m not trying to be funny but it’s interesting how much panic one can read in the parent comment.

We all joke about the dependency on SO. It seems to actually be more true than perhaps we’d like to admit.

It’s unjustified panic. It’s freely downloadable and available everywhere.

It could sound like a joke, but when I applied years ago to be a sysadmin for an Antartica base, they let me travel with whatever I want.

My first option was a full copy of stackoverflow. No joke.

Now, I'm reading this 5 hours letter and everything is ok, but what a wonderful site.

They were down for several hours a week ago. It seems they've been having some sustained issues maybe? https://twitter.com/StackStatus/

For anyone looking on this from the future, the error page has Content-Type set to text/html but literally returns the text: "The service is unavailable." and a newline character.

        *   Trying 151.101.1.69:80...
        * Connected to stackoverflow.com (151.101.1.69) port 80 (#0)
        > GET / HTTP/1.1
        > Host: stackoverflow.com
        > User-Agent: curl/7.79.1
        > Accept: */*
        > 
        * Mark bundle as not supporting multiuse
        < HTTP/1.1 503 Service Unavailable
        < content-type: text/html
        < Content-Length: 27
        < Accept-Ranges: bytes
        < Date: Wed, 03 Aug 2022 14:58:39 GMT
        < Via: 1.1 varnish
        < Connection: keep-alive
        < X-Served-By: cache-lga21957-LGA
        < X-Cache: MISS
        < X-Cache-Hits: 0
        < X-Timer: S1659538719.045867,VS0,VE2
        < Vary: Fastly-SSL
        < X-DNS-Prefetch-Control: off
        < Set-Cookie: prov=f6b7f757-544e-1afc-3cc2-e13af9fc8932; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
        < 
        * Connection #0 to host stackoverflow.com left intact
        The service is unavailable.⏎

That's valid HTML.

A title-element is required for the HTML to be valid. It is the only required element AFAIK.

You mean I gotta start thinking on my own?

Not sure that will work.

I am using the Stackoverflow Keyboard. How am I supposed to type code now?

I had a good laugh, thanks. Never knew that was a thing[1] but it totally made my day.

[1] https://stackoverflow.blog/2021/09/28/become-a-better-coder-...

Time to take a long lunch. Hopefully it’ll be back soon, otherwise I’ll have to invent a reason to give to my boss on why I can’t get back to the office!

So what is their stack, or where can I find more about it? A totally custom webserver? Someone commented to this effect the yesterday [1], but I couldn't find much in the way of a good writeup, except this answer from 2009 [2].

1 https://news.ycombinator.com/item?id=32321726 2 https://stackoverflow.com/questions/676326/how-does-stackove...

I remember hearing that they are a .NET stack with C# and SQL Server.

The only reason I know that is because I was working at a company and discussing the issues the company was running into scaling SQL Server and Stack Overflow was used as a counter example of SQL Server scalability. It lead into a long discussion of caching on high read workloads.

I'll never touch SQL Server again if I can help it.

builtwith.com gives a lot of information, although the results page comes back with a "Misleading Technology Profile Warning".

Not specifically stackoverflow, but here is an answer on stackexchange [1] listing many of the parts that make up their stack.

I'm assuming stackoverflow runs as a part of all this, and not separately.

1 https://meta.stackexchange.com/a/10370

https://nickcraver.com/blog/2016/02/03/stack-overflow-a-tech...

Excellent, thank you. "Stack Overflow: How We Do App Caching - 2019 Edition" is particularly informative.

I also just discovered that “new” entry while posting this link and it was indeed quite a good read.

Scott Hanselman had an interview with the Stack Overflow Director of Engineering just a few weeks ago. They cover some high-level details there [0]. As I understand it's an .NET Core monolith with a very-large (~1.5 TB RAM) SQL Server backend.

[0] https://hanselminutes.com/847/engineering-stack-overflow-wit...

One thing about using SQL databases live in production for everything is that the architecture is amazingly simple, almost always performant, but if something goes wrong, it goes catastrophically wrong.

Amazon for new critical applications prefers NoSQL solutions for this reason since performance is flatter and not dependent on vagaries of the query optimizer.

From what I've gleaned through speaking to current and former Amazon employees you basically have to use DynamoDB for all new services.

IMHO the logical progression for applications with typical relational models is starting with an RDBMS in combination with an ORM, breaking out of the ORM when necessary for more complex/slower queries, ditching the ORM altogether when it starts to break, then evaluating sharding or moving domains into a NoSQL solution piecemeal. A well-architected monolith or series of microservices suits this paradigm which allows you to realize the productivity benefits of SQL and only worry about the deduplication, indexing, and consistency issues that come with NoSQL when it becomes actually necessary.

Then again if I were Amazon I'd probably plan for scale up front and be jaded by their experience with Oracle too.

Finally we can't thanks the spam copies, they assure the vital service of caching the site when it goes down.

It had the same issue the other day, at least for a few minutes, so possibly an ongoing glitch?

They're having stack overflow?

Cue jokes like "... and today was 32% less productive for developers worldwide"

Reply on news.ycombinator.com