HNHacker News
TopNewBestAskShowJobs

obeattie

1,215 karma · joined May 19, 2008

VP Architecture at Monzo Bank.

Twitter: @obeattie

submissionscomments
obeattie··on Monorepos: Please don’t
The real issue is that this is more nuanced than is appropriate for one-size-fits-all advice. "Everyone should use a monorepo" isn't helpful, but neither is "everyone should not use a monorepo."

Sadly, this article falls into the same trap.

obeattie··on SIPC Says It Has Serious Concerns About Robinhood's New Product
It's a bit jaw dropping that they've launched this without talking to the SIPC to check that they agree with the statement that they provide FDIC-like protection on these accounts. I wonder what happened here…
obeattie··on Vladimir Lukyanov's hydraulic computer
There's a little more info in this (somewhat wonderful) article: https://pruned.blogspot.com/2012/01/gardens-as-crypto-water-...
obeattie··on Monzo, a U.K. challenger bank, raises £85M Series E
We're definitely aware of the iOS performance issues and we're working to fix them. We're frustrated by it too.
obeattie··on Sqlite: Code Of Conduct
How immature. If you disagree with having a code of conduct, then not having one seems preferable to making it an insult to those asking for one.
obeattie··on Airplane Mode: A concrete box designed to seduce you into digital disconnection
Why do I need a concrete box with a leather cradle for this? My phone has a feature - conveniently also called “airplane mode” - that does this.
obeattie··on Should Rivers Have Rights? A Growing Movement Says It’s Time
>I’m going to leave out the west coast, which has a nutso way of doing things.

Just for interest, could you elaborate? How does the West Coast (of the US, I assume you mean) do things?

obeattie··on I made my own WireGuard VPN server
We use Wireguard extensively at Monzo. It’s such a workhorse: extremely performant, straightforward to administer. Having dealt with the horrors of IPSec before, I can safely say I’d never go back.
obeattie··on 1.1.1.1 outage explanation
Major credit to Cloudflare for publishing a clear, honest, and detailed description of what happened. I wish more companies would do this.

One thing I’d be interested to know more about is why it took 17 minutes to fix. While you can and always should strive to make them less likely, outages are inevitable, so how you respond is crucial. Here the outage was very obviously caused by a deployment that I’d assume was supervised by humans – why did it take 17 minutes to roll back?

obeattie··on Warning signs for TSB's IT meltdown were clear a year ago, according to insider
I’m Monzo’s Head of Engineering. A lot of the technologies mentioned above for us are accurate but some aren’t in use - notably PostgreSQL - and some others which are crucial are missing - eg. Kubernetes. The vast majority of the platform runs on AWS. Our engineering team has grown to around 60 now.

Our entire backend has been built in-house using modern tech. You can read a bit more about it on our blog if you’re interested: https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...

obeattie··on TSB Bank: Botched upgrade has left customers unable to access their accounts
The executives and employees of banks can definitely go to jail if they screw up hard enough. Those in tech aren't excluded from this.

(Context: I am the Head of Engineering at a bank.)

obeattie··on Mice compensate for extra exercise by reducing other physical activity: study
So I think the theory is that when we take up exercise, we move less during non-exercise time. They explored this in mice by tracking their movements with infrared devices.

But surely we have oodles of data from devices like Fitbits and Apple Watches to know whether this behaviour is true for humans or not.

obeattie··on An Update on Last Week’s Accident
I'm willing (and almost hope) to be proven wrong on this, but I personally find it hard to believe that the ambiguity around this in the statement is accidental. A statement of this nature will have been extensively wordsmithed.
obeattie··on An Update on Last Week’s Accident
> the fact that the Tesla driver received multiple "get your hands back on the wheel" notifications, as close as six seconds before the accident

But that isn't what their statement says. It says the victim had his hands off the wheel for six seconds before the crash, and that he received hands-on warnings "earlier in the drive." It does not say that during those crucial six seconds he was being warned. Nor does it explain the fact the car plowed into a barrier.

obeattie··on An Update on Last Week’s Accident
To me there is a stunning lack of compassion and decency in the response to these incidents by both Tesla and Uber.

Uber made sure to point out that the victim of their incident was homeless. Tesla is pointing out how the driver received unrelated cues earlier in the journey. None of this information is relevant. They’re trying to bamboozle the reader in an effort to improve their images at the expense of victims who can’t defend themselves.

I don’t understand why it is so impossible for these companies to act humbly and with a sense of dignity around all this. I don’t expect them to accept responsibility if indeed their technology was not to blame, but frankly that isn’t for them to decide. Until the authorities have done their jobs, why not show remorse, regardless of culpability, as any decent human would?

obeattie··on Kubernetes 1.10 released
We run all of our production backend on it at Monzo (we're a bank.) [1]

We first deployed v1.2 nearly 2 years ago, and I can say Kubernetes has made some amazing improvements in that time – in terms of functionality, usability, scalability, and stability. This release continues that trend.

We've invested a lot in it too with things like making sure our etcd clusters backing Kubernetes are really solid [2], and we've even added some of our own features to Kubernetes like allowing CPU throttling to be more configurable to get more predictable >p99 latency from our applications.

We've been through our share of production issues with it (some of which we've posted publicly about in the hope that others can learn more about operating it too [3]), but I don't think there's any way in which we could run an infrastructure as large and complex as ours with so few people without Kubernetes. It's amazing.

[1] https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...

[2] https://monzo.com/blog/2017/11/29/very-robust-etcd/

[3] https://community.monzo.com/t/resolved-current-account-payme...

obeattie··on Arizona Government Suspends Uber's Self Driving Cars from Roads
Waymo mentions this in their safety report, which I think is interesting, and relevant:

> Advanced driver-assist technologies were one of the first technologies our teams explored. In 2012 we developed and tested a Level 3 system that would drive autonomously on the freeway in a single lane but would still require a driver to take over at a moment’s notice. During our internal testing, however, we found that human drivers over-trusted the technology and were not monitoring the roadway carefully enough to be able to safely take control when needed.

> As driver-assist features become more advanced, drivers are often asked to transition from passenger to driver in a matter of seconds, often in challenging or complex situations with little context of the scene ahead. The more tasks the vehicle is responsible for, the more complicated and vulnerable this moment of transition becomes.

obeattie··on Facebook was warned about app permissions in 2011
Whether this is intuitive for you as a developer isn’t the point. The point is that Facebook gave your friends permission to expose your data, without making this very clear at all to you. It seems hard to argue that this was in users’ best interests.
obeattie··on Learning to operate Kubernetes reliably
I think this bit from "Borg, Omega, and Kubernetes"[1] (which is an excellent read) sheds light on this:

> The Borgmaster is a monolithic component that knows the semantics of every API operation. It contains the cluster management logic such as the state machines for jobs, tasks, and machines; and it runs the Paxos-based replicated storage system used to record the master’s state.

So it sounds as though Borg includes its own storage system. As I understand, Google has a set of (very complex) libraries written in C++ that implement Paxos/Multi-Paxos[2], which they have not open sourced.

[1] https://research.google.com/pubs/pub44843.html [2] https://research.google.com/archive/paxos_made_live.html

obeattie··on Understanding Go panic output
Nice article, thanks for publishing this. However:

>This panic is caused by dereferencing a nil pointer, as indicated by the first line of the output. These types of errors are much less common in Go than in other languages like C or Java thanks to Go’s idioms around error handling.

I often see assertions made like this about Go. My experience at companies which use Go as their primary development language with reasonably-sized teams (30-50 people working on hundreds/thousands of Go programs) does not agree with this. I would say nil pointer dereference errors are just as common as if they were writing code in C or Java. I don't have hard data to back this up though; perhaps you do.

obeattie··on MasterCard Ending Signature Requirements
It’s my understanding that this isn’t totally true. You are liable for the fraud if you have been “negligent” with your PIN and stored it insecurely or divulged it. If the fraudster obtained it through some other means then your bank is liable. It’s true that the merchant isn’t liable.
obeattie··on MasterCard Ending Signature Requirements
Mastercard did not just “finally” realise this. They realised it back in the 90s when they were involved in the creation of EMV, which replace the “signature” method of cardholder verification with PINs.

Signatures have been all but obsolete for purchases in Europe for years; I can’t even remember the last time I used it here. Everything is contact or contactless EMV now, authenticated by PINs.

obeattie··on Some .io nameservers are returning wrong results again
Also seeing this affect several of our .io domains.
obeattie··on CAP: Don't settle for eventual consistency
Plenty of Google's products use Spanner and have for years.
obeattie··on Getting Hacked, Lessons Learned
This is a bit of a tangent, but Authy allows recovery of an account without access to either the original device or the original phone number. Does anyone know what actually happens when you go through this flow?

https://www.authy.com/phones/change/

It says they may perform "additional security checks" in the help but what are they? The wooliness of this makes me very nervous. (I don't use Authy fwiw)

obeattie··on NASA Spacecraft Dives Between Saturn and Its Rings
Cassini has been the most incredible mission… so many fascinating discoveries and wonderful imagery. We really need to do more (far more) missions like this.
obeattie··on Building a Bank with Go [video]
Yep, that's exactly how it works.
obeattie··on Building a Bank with Go [video]
I'm the Head of Engineering at Monzo. We don't use floating point at all in our core banking systems, which are all written in Go by our own engineering team :)
obeattie··on Google Hangouts temporarily won’t support phone/video calls on Firefox
Regardless of how you feel about Google's motivations, this is a pretty dismal commentary on the state of browser interoperability.
obeattie··on Ask HN: Is S3 down?
Yes.

"We've identified the issue as high error rates with S3 in US-EAST-1, which is also impacting applications and services dependent on S3. We are actively working on remediating the issue."

← PreviousPage 2 of 7Next →