How we built the new BBC Homepage | Hacker News Reader

How we built the new BBC Homepage

bbc.co.uk

154 pointsachalkley94 comments

Christ, so much hatred.

I like the desktop site, I like the mobile site, and I like and appreciate that you're giving us an insight in to your tech stack and how content is published.

Cheers beeb.

Does anyone know how much of this development is paid for by the UK tax payer vs BBC worldwide profits from DVDs etc?

I'm not sure website development should be paid by UK tax payers TBH...

Maybe at least some of the reason for hate.

My main hate is the idiotic cookie policy which makes everything look ugly, but I guess the EU is to blame there.

you mean by license fee payers.

Given the bbc is the 7th most popular site in the UK, and a clear leader over other news sites, I don't see why it shouldn't be paid for by the license. It fulfils its requirements outstandingly well.

(There are questions about whether given the move away from a tv based model the bbc should be paid for by tax payers, and that's fair)

It's ok they're building a new site. It's totally not ok that they created a steaming pile of crap, again.

Yeah but every few years they seem to reprogram the entire website in "whatever is most fashionable at the moment". I'm not sure that's a good use of the publics money.

You'd rather we kept with outdated layouts like this[0], forever? Part of rebuilding the website is updating the layout and usability of the website. In order to keep the website usable, the people administering it have to have better tools to organise content on it (as we all know, UX isn't just design, you can't just write a new template over a site with bad UX to make it perfect).

Developers use whatever is most reasonable to use at the time with an eye on it hopefully lasting 5-10 years into the future - much of the BBC website still runs on legacy, unmaintainable Perl, and they've learned their lesson from that.

[0] http://news.bbc.co.uk/onthisday/hi/dates/stories/december/4/...

I'm happy with that "outdated" layout yeah. It's pretty narrow on my screen - but so's the new one. The information is clear and accessible - more so than on the new one, actually. And I'd bet the filesize is smaller and the render time shorter as well.

Fine - but don't be surprised when a decade after letting their website rot, everyone's reading Murdoch's newspapers and the BBC has no reach.

The BBC was never supposed to compete in the race to the bottom. If anything I would think the opposite - a plain site becomes the voice of authority. Certainly in the print-newspaper world, the eyecatching, flashy designs are the domain of the tabloids; the more serious newspapers are more restrained, and the public trusts them more.

Either the BBCs "content" is good enough to win out, or it's not. The way that content is presented is of little concern. The BBC website of years ago works just fine.

You've really shown your biases in that statement...

You are clearly not representative of the target market. If preference were tested, I'm sure the vast majority of readers prefer more recent sites to that old style.

A dominant news site has an obligation to try to look good to it's readers.

Per their 2013-14 financial report, the BBC's income was 3.7b pounds from license fees and 1.3b pounds from commercial activities (= largely BBC worldwide profits).

http://downloads.bbc.co.uk/annualreport/pdf/2013-14/BBC_Fina...

And the BBC's mission is "to enrich people's lives with programmes and services that inform, educate and entertain". Building a website to better distribute the content they've already paid for seems like a no-brainer.

Disclaimer: I used to work for a subsidiary of BBC Worldwide. Plenty of bureaucratic nuttiness in there, but this seems like one of their more sensible moves.

> I'm not sure website development should be paid by UK tax payers TBH...

We pay for it mostly via the TV license fee. The BBC is not allowed to make money off UK citizens for any of its core services (TV, radio, news) - hence why some BBC links for the rest of the world are blocked for us, as they contain advertisements.

BBC News is primarily intended for a UK audience, so it makes sense that it's paid for by UK taxpayers. It's distribution infrastructure, just like the Freeview transmitters and the Freesat satellite - why wouldn't it be paid for by the taxpayer, who overwhelmingly demands online access to news and TV catch-up?

And I totally hate that they're wasting taxpayers money on all this crap.

Subtle. The alternative is biased commercial TV which prefers ad-spattered sensation over what the BBC produces. But I'm sure you prefer that.

Besides, the BBC amasses fortunes selling their programs abroad; Top Gear in its original form and in the formula license is shown in over 150 countries worldwide. And that's just one of their productions.

I'm not against the licence fees. I'm not comfortable with the money being wasted on a project so obviously broken by design.

I for one don't mind that they use tax payers money - because for all its many and varied biases* (national & cultural blindspots, party political) - its output is valuable and often of benefit/interest beyond Blighty.

* BBCs insistence that it is unbiased is unrealistic imo (but not as outlandish as Fox's 'fair and balanced'). Its more of an ideal rather than something anyone/org can achieve. I admire their attempts to achieve some kind of balance, although sometimes the BBC can filter too much thats not 'centre-ground' (but less so than US networks).

I'm ok with their coverage. They've just made it less accessible. Just another one in a long line of over-funded IT failures.

There are always mudslingers following change - its so easy to hate.

I for one think it is fantastic the BBC are sharing this information and am liking the direction they are taking. It might not be perfect - I too have experienced some issues with the site over the past few months - but I'd much rather a BBC that looks to refresh itself and attempt to improve the experience for their users. Glitches happen - as long as they are fixed then it is all good with me.

Of course it's easy to hate, when something that used to work is now broken and useless. It's not about glitches, it's a thoroughly broken design.

Seems fine to me, if only there was a different site for you to visit

I'm mostly complaining about their new Android app, which was rolled out alongside with the new backend site.

As a pretty typical London commuter, I expected this thing to be usable offline, just like the old app. But no, it's broken now, useless for me. Not reading any news any more.

Hmm. While it's nice to push rendering out to clients for saving your server CPU, that can lead to a suboptimal mobile experience, requiring more CPU and battery power on the mobile device to work on the javascript and render the page. There are other factors, for sure, but you want to be keeping client side javascript down to a minimum.

Taking a quick spin through yslow in the mobile browser suggests they've got a number of areas to improve on to make the time to screen significantly better on mobile devices (even on a fast connection here it took several seconds to even start showing me content, and several more before it had finally loaded everything)

Given the world wide reach of the BBC, expecting high speed and low latency networks seems like a bad idea. In the US, 3G & 4G typically see 90-100ms latency per request. Mobile Yslow is reporting that they've got 21 javascript scripts alone on the page. IIRC The android browser will limit itself to 4 threads retrieving content typically so that's (21/4 * 100ms) 525ms just lost in latency requesting the javascript, let alone actually downloading it and the overhead of the javascript renderer. It's also pulling in content from 21 different websites, so at the bare minimum that's 21 DNS calls being made (with the same latency penalty!) A bunch of those are being done just to load a single piece of content too, which is a little crazy.

Don't get me wrong, the site looks good.. it's just for a 'mobile-first' experience, they seem to be missing the all important time-to-screen and giving the mobile user a lot of work to do.

A useful tool from google for analysing the site for both mobile and desktop: https://developers.google.com/speed/pagespeed/insights/?url=...

and a good talk from last year's Google I/O conference on optimising the mobile experience: https://www.youtube.com/watch?v=WrA85a4ZIaM

Plus this is the only time I've ever seen "mobile first" interpreted quite so literally (i.e. on a desktop with a reasonable connection the mobile stylesheet loads before the main styles and content and is visible for just long enough for the eyes to register the layout, producing a horrible "flash of unstyled content" effect).

n.b. plenty of people looking at the BBC site are still being shown the old style.

I've been wondering about that unstyled "flash" on BBC myself and whether it's just down to some chrome extension or something I had installed interfering with things. Glad to hear it's not just me.

The main sites from my last job load the mobile view/content first, then load in the desktop experience as appropriate (some are still in the process of transitioning to the newer design). However they were done in such a way that it shouldn't result in any jarring transitions. A couple of examples:

https://portal.ehawaii.gov/ http://energy.hawaii.gov/

How true is that though? Given that native apps render client-side too. In a much more optimized, device-specific fasion, I admit, but still - that too is client-side rendering.

Your comment inspired me to use the great webpagetest.org service to collect a trace on iOS using a simulated 3G network condition:

http://www.webpagetest.org/result/150217_BB_Q71/1/details/

There's definitely plenty of room to optimize the time-to-first-render – in particular, switching to an async model would make a huge win since rendering is currently blocked by 6 stylesheets and 7 JavaScript files, most of which appear to be specific to certain content sections.

Totally agree that it's not ideal to push rendering out to clients -- and this article specifically says they're using React.js and rendering the pages server-side, so they are sending a rendered page to the client (along with the js code to perform further front-end behavior if js is enabled, but content is still there without js).

I do love when companies share this stuff. Thanks.

> Rather than using PHP or Java (that was the requisite of the Forge platform), we have chosen a non-blocking framework, NodeJS with the Express framework . This allows us to serve more simultaneous requests, increasing the performance of the application.

I don't doubt this is true, but its worth noting you can get good performance out of a "blocking" framework too. Node.js does better than others in some situations, but in this its not a snowflake, and is in some regards worse.

But I will criticize priorities: I think this is too much fad, not enough practical. the experience is notably worse than the old site, and it seems like they just threw buzzwords at their problems instead of really crafting a solid solution.

I think where they're coming from, maybe this makes sense - they needed to overshoot from their previous platform. But I think they'll find it problematic in the long run and change some of their approaches and release a new site, and that will be a good platform and serve their needs for a good while.

I agree that it sounds very faddish, but that might be the whole point of the blog post - attracting talented developers. Manchester has an up and coming startup scene, which presumably the BBC is competing with in the hiring pool, and brandishing their hipster credentials may be one way of doing that.

I must confess I'm quite surprised that they would get more performance out of NodeJS than with a decent Java framework. Java has its problems, but it's usually fast.

They do mention prototyping with Scala and Clojure. I also doubt NodeJS beat the JVM out and out in performance. Maybe they went with Node because it is easier to hire web developers that know javascript than it is to hire web developers with knowledge of the JVM

Java developers are not exactly a scarce commodity.

When you rewrite sites, that is often what happens. It will almost always be buggy at first, then it stabilizes as those bugs are fixed.

There seems to be some kind of bird theme going on with their CSS classes.

  class="distinct-component-group container-buzzard"
  class="distinct-component-group container-pigeon"
  class="distinct-component-group container-macaw"
  class="robin sparrow-container"
  class="sparrow-container sparrow-columns"

Tons of tech buzzwords, yet the blog intercepts middle-clicks. I want to open your links in a new tab so I can read them when I'm done with your blog!

Its not intercepting Cmnd+Click (mac)

Linux / Chrome middle clicking doesn't work.

How can they break something so basic?

Also not working on Windows 7 / Chrome.

Though working fine on Windows 8.1 / Chrome

Edit: Actually, only working fine on the first link. All subsequent links get captured

That's weird - middle click still works fine on Firefox/Linux, but doesn't work on Chrome/Linux.

Hard to believe this was intentional.

It's not just the blog - news.bbc.co.uk also intercepts middle clicks to external sites. Middle clicks on links to other bbc.co.uk pages work fine though.

Is it an accident or intentional? Anyone know?

I've noted this to them in their feedback forms before. It's been like that for over a few months. If I recall correctly when the Guardian (uk) was developing their current version under Beta the same problem came about for a short while.

I believe it's unintentional - I expect it stems from JavaScript event tracking (e.preventDefault(); logEvent(e); window.location=e.target.href; Or something to that effect).

> Tons of tech buzzwords, yet the blog intercepts middle-clicks

They broke the middle-click? Good god, shut it down!

It's like they monitored the front-page of HN everyday for a year, made a list of the top mentioned technologies and then gave that list to their management.

Their manager said "Your goal is to write a blog post mentioning each one of these technologies, and add links so it appears in dark-bold-blue text. If we don't have a project using a trendy tech-stack... you bust your ass and get something up and running."

The engineers balked... "but why?"

Then the boss said, "I'm tired of being ignored in 'Who's Hiring' on Hacker News. We make this article and they'll all come begging us for jobs. At the BBC we don't wait for news to come to us, we make the news. And THAT'S what we call journalism kids."

> It's like they monitored the front-page of HN everyday for a year, made a list of the top mentioned technologies and then gave that list to their management

Or perhaps those technologies were actually useful for them? I don't know, just a thought.

This is extremely offensive to the team that built this product. Why the hate?

I've been using the new design on mobile for a while here and the user experience is awful compared to the old design. Although the parent comment might seem a bit patronising, I think he's right - they've focussed too much on the technology and design fads.

Compared to the old site, there's a lot of "big text", too much of a focus on images (when you drill down into the site) and less on text, and low information density.

Worst of all (for me) - when loading on latest stable Chrome on Android, with a very fast internet connection, the page appears loaded but as soon as I scroll a number of the images disappear and the layout changes. I have to scroll back to where I wanted to be. Lots of scripts, client-side rendering, etc. It's overkill for a site whose focus should be content, not interaction.

While I'm sure the team built a technically impressive website, I don't think they've built the "right thing" and I certainly don't think it's better than the old one (from a user perspective).

The guardian have also similarly screwed up their mobile implementation, with at one point they broke the back key because they had expandos that couldn't remember their state.

But both these sites still suffer from "the flash", where it renders once, and then renders again half a second lafer with everything moving. It's disorientating and simply a bad UX.

Also the BBC used yo have this terrible experience for a few months where it would flash up a load of code, but they finally seem to have fixed that.

IMO both these sites are presently advertising that the responsive trend sucks ass.

I have started to realise that compared to the desktop sites it's almost impossible to get information out of these mobile sites at anywhere near the speed or information density I previously used to simply using the built in pinch and zoom and desktop layouts.

Like you scan broadsheets and desktop layouts, but the mobile you get a big, useless picture and a story title.

I like the design of the new BBC and Guardian websites, but I agree that if they're aiming for mobile first, they still have a way to go.

In particular, 'the flash' you mentioned on The Guardian means that often, instead of iOS 8 Safari scrolling when I try to scroll, it activates a link - or worse, an ad! It's infuriating, and it's significantly less usable than their old site in that regard.

For a truly bad mobile site, though, try going to Cricinfo: http://www.espncricinfo.com/ Go to an article, then go back. Notice that:

1) Your horizontal scroll position in the article list is lost.

2) The page renders, then re-renders exactly the same (for me at least), with a jump to the top of the page in between.

Don't invite Ricardo to parties.

If you believe that the technologies they selected are inappropriate then maybe you could explain why?

I love your comment, but I think the reality is more that they already had all the hipsters on board, the technology choices just followed naturally from that.

This mentions using Node/Express to help with concurrency. Ok. But this also mentions using 'isomorphic' javascript with React, and also not completely reliant on cache. React.renderToString (and all other isomorphic options -- Rendr, etc) are synchronous and slow. (rendering a large page in React w/ static data would take ~600ms; whereas the same page with with Mustache in Go for example would take ~30ms). End result, decreasing concurrency, and hurting overall performance.

OP - can you shed any light on how this is actually impacting your performance? Or maybe things that you had to do to get around the problem (ex - details of 'module level' cache with Redis etc).

I have not yet tested the new mobile site, but if the design of the normal site is anything to go by it will have been designed in hell by some teen demon reading too much Kafka.

Most of their tests take half a second each to run? Some examples: "The first story displays an image and not an alt text attr if one is defined. (793ms)" "The module banner color is BBC News Red (552ms)"

The 13 tests shown in the screenshot take an average of 577ms each, a total of over 7.5 seconds. 13 simple tests with wild variance in execution time. Checking a module banner color? 42ms. Checking a module background color? 552ms. So a 12x increase for checking a different color within a module?

Those tests are going to rot because of the expense of executing them and they'll be discarded. Over a 500ms average per test just isn't sustainable, particularly at the scale required for checking every conceivable kind of background color.

I work on a rails codebase with some tests that take 20 seconds to run. The entire suite is normally 5 minutes. I made a ruby gem to take the entire thing down to 20 seconds, which is much better for flow.

But I agree those are definitely smelly tests with questionable value return, and would not be surprised if they do indeed rot.

Thanks for sharing the tech stack. Would love to read more about the curation kit and the microservice approach.

I agree. More corporates should give these insights and don't care for a bit of bashing when they do.

They mention using BEM (https://bem.info/) Any idea what it is? The site is not at all clear.

CSSWizardy say it better than me: "BEM – meaning block, element, modifier – is a front-end naming methodology thought up by the guys at Yandex. It is a smart way of naming your CSS classes to give them more transparency and meaning to other developers. They are far more strict and informative, which makes the BEM naming convention ideal for teams of developers on larger projects that might last a while." http://csswizardry.com/2013/01/mindbemding-getting-your-head...

But basically: agreed rules for how to name and format css to make it maintainable and performant in big projects.

Thanks. I dont know why they couldnt just say that on their homepage!

Essentially, the BEM website is the worst possible rule for learning the underlying principles of BEM (ie what makes it good), instead it's all about their tool that attempts to manage it all for you.

I worked on a project that used that and while I understand the reasoning I hate the ugliness of it.

The BBC produce quite a few intereting write ups of their R&D process alongside odd things they are hacking on here: http://www.bbc.co.uk/rd

Well worth a read.

Home page displays a bunch of javascript before rendering the rest of the page. Doesn't happen all the time though.Maybe you guys might want to look into that.

It nice to know what is happening in the BBC and the tech they are using. Additionally, It is appreciated they spend the time writing up what they have done/achieved.

Rock on!

Actually it doesn't look too bad. Its similar enough to the old one it doesn't seem like when Facebook upgrades and you feel lost. Shame the quality reporting has gone down so much over the years.

Amazing insight into how large organisations manage projects.

Reading the article reminded me of this: http://motherfuckingwebsite.com/

I wonder to what extent they are still using Perl.

The days of the BBC being a "Perl shop" have long gone.

But they do still use Perl, for e.g. BBC Genome is a new project written in Perl - http://www.bbc.co.uk/blogs/internet/entries/cedd1f30-ac95-32...

I love the BBC. Very interesting news, fairly balanced.

I hope I can play their videos on the new website, as before they all used to require Flash.

In Firefox 29.0.1, on the new BBC News homepage, wherever I click, it takes me to same story, even if I click on whitespace.

The new "BBC News (Beta)" website is bloody ugly. They've gone for a "flat UI" thing, but it looks so clumsy that it reminds me of the internet from before people used gradients.

Haha has the internet groupthink gone full circle on 'flat' now?

Tech is fashion whether we admit it or not.

wow you use nodejs. how amazing that is

Pleasantly surprised how 'up-to-date'the beeb is.

For those that aren't aware, the BBC has one of the worst cases of "Not invented here" syndrome you'll ever see.

And yet all I see in the article is off-the-shelf libraries and frameworks, and they came from a commercial CMS - where's the NIH?

You won't find any in that article, but the BBC did build their own jQuery once upon a time (now long-since abandoned, of course, as many "NIH" projects tend to be): http://www.bbc.co.uk/glow/

The new BBC page is barely usable. Not to mention that the new BBC android app is thoroughly disgusting. Bring back the old one, please!!!

I believe there is a bot that monitors websites for changes, and whenever they change enough it posts randomly generated strings along the lines of:

> The new <sitename> design is terrible! Bring back the old one!

Whether it recognises the irony that it also moaned about the 'old one' when that came out, I'm not sure.

Have you used the previous BBC android app? It worked. The new one is totally useless.

No, to be fair, I haven't. It's just tiresome how every single design change by any large website generates comments exactly like this.

As somebody that has done a fair amount of usability testing, that much of why people think redesigns are unusable are simply because they were used to the old version and haven't yet spent much time with the new version. But if you show the new design and the old design to somebody that has used neither, they frequently prefer the new design. And over time, everybody kind of likes (or at least gets used to) the new design.

That doesn't mean that all redesigns are necessarily good, or even that redesigns are necessary, but the kneejerk "it's unusable!!!!!" is so cliched.

In this case "unusable" is fully justified, it means that the entire (and, I suppose, important) class of use cases had been eliminated: it does not work offline. And the system is designed in such a way that there is no hope it'll ever be fixed.

I doubt they did any proper UX testing at all, they did not collect user stories, they did not consider all the use cases. They just hired a bunch of hipsters eager to add as many buzzwords as possible to their CVs.

> I doubt they did any proper UX testing at all

Really? The BBC didn't bother doing any UX testing on a redesign of their flagship website? Really?

This reads like you're just point-scoring at this point.

If the site worked on the Tube before, and now it doesn't, that's a pretty massive failing.

That point was added after I made the comment.

What do you not like about their new Android app? I use it daily and greatly prefer it to the old one.

Obviously the tone wouldn't do at HN, but this Reddit post in /r/unitedkingdom mentions a lot of the problems with the new mobile site and the Android app.

http://www.reddit.com/r/unitedkingdom/comments/2vymm2/does_a...

Re: Android:

>Jackal___ 121 points 1 day ago >Their new Android app is very very slow to load the news and uses a fuck load of background data.

>Quagers 72 points 1 day ago >Also it doesn't download the stories when you hit refresh, just the headlines and then downloads the story when you click the headline. Completely bloody useless on the tube!

1. Crashes about twice in 5 minutes.

2. Tiny cache, compared to the previous one, so totally useless offline

3. Intrusive UI

4. Much slower

Reply on news.ycombinator.com