RBS software issues likely to continue into third week | Hacker News Reader

RBS software issues likely to continue into third week

siliconrepublic.com

17 pointslucaspiller21 comments

I'm a customer with these guys. The problem started on the 21st of June, the only transactions on my account are small ATM withdrawals. No card payments or more importantly no salary!

Tried to pay the rent. Didn't work. Tried to pay the phone, didn't work, broadband? Nope didn't work either.

From what I understand its some old mainframe-ish style legacy infrastructure that broke down. Ulster bank downsized anyone who was working in that area a year ago and shipped it all out of the country.

The Register had some decent coverage of what went wrong [1].

If this does prove to be the root cause, it just highlights the costs of cutting costs irresponsibly.

For customers, as has been said elsewhere, second bank accounts and cash buffers can be cheap insurance against the incompetence of others.

[1] http://www.theregister.co.uk/2012/06/28/rbs_job_cuts_and_off...

Boy, some of that described experience is pretty familiar.

I can see this going down in IT folklore, "Remember to do test your backups and roll back plans, or you might end up like Ulster Bank"

I work with some people who used to work for RBS IT, they tell me that the department was as bureaucratic as they come (presumably to reduce risk) but once all the hoops had been jumped through, it came down to single team who would put code into production, and they were often quite poor at it.

When I think about the scale of this mess, and the hundreds of millions that RBS is going to loose in compensation and lost customers, I hope that RBS (and in house corporate IT in general) starts to appreciate the need for automated deployment and fault tolerant IT systems rather than yet more process as the answer (I'm looking at you ITIL).

ITIL is not a problem, in fact it is a good thing that has happened to IT. However, the problem is the non-IT people running the ITIL. They end up sidelining the developers, or the developers badmouthing the processes, which causes all sorts of separation and issues. When people follow processes appropriately, things work out well and on time.

ITIL is a problem. It's extremely rare that an organization has problems because nobody's ever thought about how to do things better rather than having broken internal structure or incentives – ITIL can't help with either of those but the huge amounts of time and money it soaks up will make it much harder to actually do anything about them.

Or perhaps:

"Don't outsource all of your IT ops team or you might end up like RBS/Ulster Bank"

If you're an IT guy then that one is perhaps a more important battle to win.

In theory a separate company who's core competency is running IT systems should be less likely to mess this up, not more so.

However, retail banks are rapidly becoming sort of API hosting companies for the processing of financial transactions. Their core competency IS running IT systems.

If none of RBS's physical branches opened tomorrow, the UK would probably tick along just fine, with some inconvenience caused to people who deal mostly in cash. But fail to process payments for 24 hours, and the whole UK has problems. If this had gone any longer than it did, it's not hard to envisage a run on the bank and a domino effect to other banks. It's really quite scary how badly we depend on these IT systems, waaay more than Google or AWS.

In theory a separate company who's core competency is running IT systems should be less likely to mess this up, not more so.

That's true. Rather than "Don't outsource" the warning should be "Don't fire all the existing staff who understand your systems".

They didn't. They fired half of the team.

"to do a NatWest"

(Unfortuntely it slightly rhymes with 'best', but meh)

I couldn't help thinking of the backronym: "Really Bad Shit".

I think it would work better with some kind fo rhyming scheme.

Its really not an IT problem, the upside to this for RBS is they can keep a very tight grip of their deposit to loan ratios. They aren't bothered about the customers, its about keeping the bank above the waterline.

http://www.reuters.com/finance/stocks/incomeStatement?stmtTy...

Just look at how very close the Deposit vs. Loan values are, and also consider the timeline of this whole debacle.

RBS downgraded, more loans than deposits, suddenly a "IT problem" that lets them even out the odds a bit, so they can afford to borrow to cover the remainder, and gain much needed liquidity.

http://hat4uk.wordpress.com/2012/06/25/rbsulster-glitch-slog...

Given that both payments in and payments out seen to have been affected, this isn't really likely, not to mention that the fallout from events like these in terms of press coverage and regulatory interest would outweigh any benefit from such an act. Lastly, organisation of such an event would require way too many people to stay secret.

Is a Loan-to-Deposit ratio above 100% really so unusual that they would engineer an IT failure to hide it? Some banks seem to be operating which much higher values:

http://www.zerohedge.com/news/what-europes-loan-deposit-rati...

[Having said that, nothing would surprise me about RBS]

No. There will be an Irish Government investigation into this. They would never get away with manufacturing it.

I'm beginning to wonder if Ulster Bank have reached the event horizon beyond which they don't have the computing power to recover from this.

Interesting. I wonder what would happen in this instance. They have admitted losing some transaction information forever. Wonder what the financial impact will be

Thanks online news source for just reposting twitter comments without adding anything. Would have been nice to actually contact a poster to get more details or like...do more work than search for a hashtag.

Reply on news.ycombinator.com