Why does HN emit "Unknown or expired link" messages | Hacker News Reader

Why does HN emit "Unknown or expired link" messages

36 pointsjganetsk40 comments

I can't count how many times I've hit the reload button while browsing Hacker News, only to see "Unknown or expired link". I understand there are myriad technical excuses for this, but it still has a significant effect on user experience.

Am I supposed to just forgive this, because I'm a technical person and can easily get out of the jail that his error message puts me in?

View on HN

http://searchyc.com/expired+link

especially: http://news.ycombinator.com/item?id=163696

[edit: Thanks for the reply aston. My mistake]

He's not asking why it's broken, he's asking why he has to tolerate it being broken. It's not that hard to fix.

That doesn't matter. These types of posts do not belong on the front page of Hacker News. Either send an e-mail to pg, or post this in Feature Requests.

That depends. If HN is supposed to be an example of a high-quality product on the Internet... then this is definitely a topic worthy of discussion on HN. I'm not submitting a feature request. I'm trying to start a discussion about user experience.

But if HN is strictly a quick and dirty tool, then you are right. It is not worthy to make it a topic for discission.

It's the demo/test app for Arc. Arc is intended for quick and dirty prototyping of webapps. It is PG's position that a continuation-passing style is the easiest way to do that. The fact that links to continuations must expire is a byproduct of the fact that computers have limited memory.

It doesn't matter what you're trying to do. What you're doing is against the rules. These types of posts don't belong on the front page, and this specific topic has come up in the past and has been answered.

This is an unpopular opinion, but this needs to be said: to keep the quality of Hacker News relatively high, please follow the very few HN guidelines.

I'm not the one who keeps modding this thread up.

No, but you did set the ball rolling, even if gravity took over. There is still time for you to delete this thread.

PG has mentioned this isn't a priority amongst his list of other priorities.

Also, go ahead and enumerate the reasons why "it's not that hard to fix".

Countless other sites never had that problem in the first place.

That is a fallacy. (Several, actually: irrelevant conclusion, affirming the consequent, etc.) Man, I miss the old Hacker News crowd. They would never let you get away with making fluff statements that prove nothing.

I'm perfectly willing to back up my statements.

The general problem we're looking at is that fnid's (and the accompanying closures) are manufactured all over the place, and most of them don't need to stay around for very long.

However, there are some of these--in particular the ones created for sending a comment to be posted in a thread--that probably deserve a guaranteed lifetime on the order of hours, not minutes. Putting them into a different fnid queue is what I'd call a relatively easy fix, and that'd handle 95% of the complaints about expired links.

The other 5% are due to trying to reply to comment for a thread you opened a while back. I actually don't know why that's even coded the way it is. The reply link could easily just refer back to the post id instead of an fnid.

edit: I would probably be willing to fix this stuff myself, but as of late it seems pg's not been paying that much attention to news.yc. Also, last time I threw up a patch, I got burned pretty bad.

yea, he should definitely demand his money back...

The site is open source. Why not code a solution and send it along to PG?

As I understand it, the problem is that the web server stores the user session state in continuations, and old continuations need to be discarded regularly to keep the server from filling up. The error appears when the user tries to do something that requires a continuation that has already been discarded.

The easiest solution, then, is to simply increase the length of time that the continuations are held -- hence increasing the number of continuations held at once. If this were as simple as changing a number somewhere, I assume PG would have already done it. Hence it is likely that the server is already retaining as many continuations as it can at any one time, expiring them as it goes.

So the obvious solutions are: Get a heftier web server; rewrite mzscheme's continuations to take less resources; or rewrite HN to not use continuations. The first cannot be done with source code alone, the second has yet to be accomplished by Scheme's brightest minds, and the third is unlikely to appeal to PG.

(Edit: Not that I want to discourage anyone from improving Scheme, arc, or HN... just pointing out that the solution is harder than a simple bug fix.)

Could you write old continuations to disk? That way older continuations would take longer to resurrect, because they would come from disk, but at least they would still be around. You could probably store them for a few days before expiring them from the disk cache.

I guess this could totally thrash performance by causing a lot of disk I/O, but I have no sense of how likely this is. If you had some statistics about the rate of continuation-cache misses and how often cache evictions occur, you could probably get a good idea whether it's feasible.

The impression I get from the following newsgroup thread is that mzscheme doesn't support serializing continuations to disk... at least not easily:

http://www.cs.brown.edu/pipermail/plt-scheme/2007-February/0...

Perhaps it is possible under other Schemes, though?

Disclaimer: I haven't looked at the source yet.

My particular example was for browsing the thread list. Let's s/continuation/browse context/. You can generate exactly one browse continuation a minute, and hand that one out to everyone who visits the page during that minute. It contains the effective state for that continuation. So it's a little out of date... big deal.

Let's say a browse context is 1 kilobyte (which can fit a list of several hundred IDs and scores). With 1 gigabyte of RAM, you can store 1.9 years worth of browse contexts.

"Browse contexts" would not be able to be stored client-side, so the server cannot hand one out to everyone who visits a page. The server hands out an ID that references a browse context that exists on the server. Each browse context must be unique if you're logged in (otherwise you would see News.YC from someone else's perspective).

When a user is logged in, one browse context is generated per link-that-requires-a-browse-context. Each "reply" link references a new browse context each time a page is loaded, for example. So if there are 50 comments on a page, then there are 50 "reply" links, each of which references a unique browse context. (If you mouseover each reply link in Firefox, you can see the ID of its browse context in the status bar. If you refresh the page, each reply link then references a different, unique, server-side browse context.)

So it's easy to see why this could get a little expensive. It's essentially a time-consuming optimization problem to get rid of the "unknown or expired link" error, at this point. And it is not hard to work around.

Hmm... I notice, however, that the "reply" link takes you to a fairly similar page as the "link" link in the comment's header: both contain a reply box. The "link" link, however, doesn't contain a fnid in the URL -- just a REST-like comment id.

I wonder if these fnid-less URLs avoid generating continuations? If so, perhaps we could drop the "reply" link entirely and just label the "link" link as "link/reply"? If I understand your explanation correctly, that would both eliminate the error for initial replies and dramatically reduce the rate that continuations expire when using other site features...

Yes, you're correct. That would both eliminate the error for initial replies and dramatically reduce the rate that continuations expire when using other site features. Unfortunately, that would completely remove the ability to "reply". :)

When a logged-in user loads a Hacker News page, the server must generate a unique set of continuations for that user (one per each link that requires a continuation.) In other words, if everyone shared the same set of continuations, everyone would see the exact same Hacker News. The username in the upper-right corner would be the same, everyone's comments would be tagged with the same username, etc. Does that make sense?

So the "reply" link requires a continuation because you're adding a comment that's tagged with your username. But the "link" link does not require one because you're simply viewing a comment.

Did you click on the "link" link? It has a box and reply button just like the reply page. So removing the "reply" link wouldn't remove the ability to reply. I assume the inputs for the reply function are adequately captured by the URL (comment ID) and cookies (logged-in user) and thus don't require a continuation.

Oh snaps son. I got burned. You're right, I didn't click on the "link" link and assumed that it was only for viewing the comment. I sincerely apologize.

No problem. Now all we need is for PG to tell us why it won't work. :)

It's obviously a slightly different user experience, since users that aren't logged in won't get prompted to register/log in when clicking on "reply".

Keep in mind that getting rid of the "reply" link continuation would make the "Unknown or expired link" error less frequent, but it wouldn't solve the core issue of "each user requires a unique set of continuations". It would be a good stopgap measure in the meantime, though.

I'm not trying to solve every problem. I'm just trying to solve browsing the list of threads.

So, I'm not talking about reply. I'm not talking about links. Just giving me the main list of threads, that's why I wanted to call it "browse context". Not "reply context".

I also do not suggest sending the "browse context" to the user. I'm suggesting sending an ID to the user, while the server can store up to 1.9 years worth of browse contexts, so that I can hit reload on the main page for up to 1.9 years without seeing an expiration.

I would solve issues with reply separately. Maybe HN is meant to just be a demonstration of the power of continuation-based web dev.

i found when providing a comment of depth, i'll usually do a quick copy and paste of what i wrote, go back and post it quickly.

I don't know whether this is browser-specific, but I can still go back to the editing page and have my text there after getting an expired link, then copy it, reload that page, paste my post back in and submit. (this is on Opera)

Still annoying? Definitely.

No, you're not supposed to "forgive it," you're supposed to use Google or SearchYC, and see the reasons it's there and why it's hard to get rid of for yourself.

I guess this is acceptable for us hardcorehackertypes, but if users and happiness were the goal (as they are for the startups many of us are working on), this falls under the category of:

"The User doesn't care why this isn't working, or if it's a lot of work to fix it. Don't explain it - fix it. Every other message board works correctly."

Being as this is hacker news, I'm sure we can all deal, and understand the technical bits involved. Just saying, in this case, do as PG says, nawt as he does.

I understand that pagination of a list where items move up and down is hard. I understood that while creating this thread, and I'm still not convinced that it's not worth solving.

I work for Amie Street. We are an e-commerce site where the price of each product continuously changes. I am knee-deep in scenarios where things expire. Yes, it's an entirely different situation, but it has the same significance. Every time an Amie Street user gets an expiration message, we get negative feedback.

Fixing this remains one of the top-rated 'feature requests':

http://news.ycombinator.com/item?id=363

Yes this is pretty annoying - I've even seen it when I hit the logout button.

I always thought it was a security thing... it doesn't happen that often to me anyway.

This suck, somebody should fix it, i would do it if i wasn't a web dev rookie and a lazy incompetent fool. So that just leaves us with all the other smart people here. Any ideas?

You will be promoted to "web dev advanced" if you fix it. The best way to learn is by breaking and fixing (or fixing broken things)

Maybe PG doesn't know how to do that.

I think it's caused by people posting questions without searching for the answer first.

This has been addressed repeatedly already.

You could just stop using HN if you can't "forgive this" :).

Reply on news.ycombinator.com