Hacker News Reader

Comment by Fogest | Hacker News Reader

How does this actually indicate it had to do with the redesign? Sure the trend may be going up, but the site is increasing in popularity so that makes sense that it is going up. There are more people there to interact with and more communities people can join that they may be interested in following and discussing things in.

I am just wondering how this rise is attributed to the CSS change? It may have had some effect, but I'm confused how this single graph says that it is because of a CSS change?

View on HN

In fact, from my reading the rate of signups was increasing before hand and continued increasing afterwards, implying the design made no difference. That's what I can see from the trendline anyway, and the rest of the graph is too uneven to draw any conclusions.

Yes, exactly. The graph is full of spikes and I do not see any indication that there was any real change present from the CSS change.

They are wrong to draw the trendline the way they did. Since what they are claiming is not steady growth (then the trendline would make sense) but a shift in behaviour they should instead have drawn one line for average before the change, and one line for average after the change.

It is quite obvious that something has changed. Before the change there are five days above 7.5%, and after there are more than 10 days above 7.5%. The valleys are much higher too.

> It is quite obvious that something has changed.

Yup, but that's not an 8% increase in AR's. That's maybe an 8% increase in the rate of increase in AR's. That could be attributable to a LOT of things though (there isn't even enough context to control for network & seasonal effects, let alone whatever increased attention there might have been to sign ups after the rollout).

Ironically, probably the best way to isolate the impact to the launch is to narrow the time window and resolution to the minutes before and after the redesign launched.

More days above 7.5% would be true even if the underlying trendline was going up anyway.

They really should plot the moving average and include a longer timeseries - plotting the trendline before and after would be trying to prove what they are trying to show, rather than trying to work out whether what they are trying to show (our fancy new design wasn't a complete waste of time) is actually true.

This is why A/B testing is important.

Actually, it' why A/B testing is rife with false signaling. ;-)

It doesn't say anywhere that they used A/B testing, let alone proper A/B testing.

I have never seen reddit apply any kind of A/B testing. Never heard of them doing this.

A/B testing is supposed to be preceded with A/A testing to help with that though?

Doesn't really help.

The ugly truth is, the reason for Reddit's popularity growth in long term is /r/gonewild and its children. It's so integral part of Reddit, not even the femi-nazi-caterpillartrack-crazies dare to even attempt SJW'ing it.

/r/gonewild is not even in the first 50 subreddits with most subscribers. What makes you think it's the reason for Reddit's popularity growth?

Reply on news.ycombinator.com