Solving `Passport Application` with Haskell
jameshaydon.github.io
jameshaydon.github.io
This blog encapsulates the problem of writing government services/software, which often results in strange outcomes.
Writing software for government is essentially the codification of centuries worth of Acts of Parliament.
Now imagine building the HMPO passport system, and then some underlying Law/Act is changed or repealed etc.
Now someone has to find and change everything that the Law/Act affected in all systems.
Now consider that the government frequently outsources this work to expensive consultancies who are motivated to elongate contracts and extract maximum value from the client... And ideally become entrenched.
All whilst building systems of varying quality and inflexibility so that the next time that a Law/Act is changed then this whole process repeats.
There is no central decision making authority to wrangle this problem (there used to be Spend Controls), which is why Government services delivery is so expensive.
Often, you'll find that procurement is handled by someone working off of an incomplete set of written specifications. The procurement officer probably has no expertise with or informal knowledge of the problem to identify problems in advance, and even if they do legal requirements (ironically designed to ensure fair and transparent procurement) might force them to disregard informal knowledge over the formal specifications.
After the contract is signed, sunk cost fallacies and optimism biases take over. If the project goes over budget it might be easier to cut back on features than admit failure or demand more money, and the project will go over budget once real users start seeing the in-progress work and change requests pile up. Project managers want to be seen managing a successful project; nobody's rewarded for being the first to predict failure.
This is a kind of principal/agent problem that the private sector partially manages with equity or deferred compensation, but the profit motive has a very limited role in government and you'd probably not want to give stock options anyway. Beyond that, even the political overlords have short time horizons of 'the next election', and announcing a grand new project is almost as good as delivering one.
A high-profile example of this problem is the Canadian federal payroll system (https://www.oag-bvg.gc.ca/internet/English/att__e_43045.html), which has led to billions of dollars in fix-ups over a decade or so.
The private sector also has the option (and usually the motivation) to just quietly disappear failed projects.
Wal-Mart buys an automation system for their distribution centre and it's a huge success? Tell the stock market. The system is a complete failure, we remove it and trash it? Uhhhh of course the secrets of our highly efficient warehouses are commercially confidential, couldn't possibly tell the stock market what we did or did not do.
Unlike in private companies, government employees are often very hard to fire, even if they tragically underperform. If your salary does not depend at all on your performance, you play by the book, follow the rules and don't particularly care about putting in good work. You have no incentives to do so. It doesn't matter how the contractor you picked performs, as long as they can't prove negligence on your part, you're not getting fired. You won't be promoted either, even if they deliver under budget and months before the deadline.
In a private company, the owner wants to make money, so there's an incentive to perform. Underperforming employees (or even entire departments) are simply cut. If a company fails at this too badly (all companies fail to some extend), the entire company underperforms and loses customers. Ultimately, the market ensures that companies grow if they're good at this and shrink if they aren't.
With governments, this is, again, not the case. You aren't going to switch to a competing passport system and drive this one out of the market, no matter how badly it sucks. Metrics won't change much, no matter how bad the UX is, as the system provides a critical service which has no alternative. This is the no competition problem.
Private companies also have more freedom to pick contractors. It's really bad PR for the government if their employees choose contractors through nepotism, so laws have been passed that standardize the contractor selection process. This means that, even if you know a contractor is not the right choice, you may be prevented from doing anything about this. Because al rules can be gamed, most government contracts are won by companies who know how to game the rules, not those who make good software and would be chosen by a knowledgeable employee. This is the no fraud problem.
Whatever the situation is in the UK, it is not hopeless. The government needs to stop being afraid of hiring actual employees and bring all development in-house.
While renewing a passport is one of the easiest things you can do, I distinctively remember it taking a good half-day (most of which is waiting in line jumping from one desk to another until you find someone that can check all the documents you provided are satisfactory and sign off for you).
Now however, if it's not a renewal but a new application, the difficulty sharply increases.
If you're applying for your very first personal ID, then you need to bring a birth certificate and another adult who already has an ID, but it's also something that's been trivial to do for years now.
It's a bit of a race to the bottom regarding outsourcing, there are good companies and there are good technologist's but they never last more than a few years due to poor contracts, poor decision-making and poor pay.
I've actually seen some very good companies deliver on time and under budget, but then the company fails because they did the job too well and follow-up work wasn't needed.
I think that for outsourcing companies "wage theft" and doing just enough to meet the contract are core components of their business. That is, they need to optimise for high fees, low pay and the bare-minimum in quality.
I suspect part of the gap here is, as you note, everything technical is outsourced and so there is not enough institutional capacity to envision and execute what would be a quite revolutionary re-architecture.
I find it extremely frustrating that the government is able to spend tens of millions on contractors for an IT project, but can’t pay market rate for in-house engineers, even if those would be cheaper in the long run.
Most roads will lead back to consultancies stepping-in because the government org will likely lack the skills. And it's very easy for a consultancy to claim they're experts, but throw an army of graduates at it.
Interoperability between rules based systems (and legacy) software need to be considered too. It's a massive problem that will grow with time... Most government departments have some form of every generation of technology.
I'd be cool to have a system that just needs to be fed "patches" in an append only way instead of having to scatter exceptions (if branches) all around.
Some logic programming languages can do that a little bit, but I think that the real problem is that there's no tech debt budget, and lawyers working around the laws can also profit from things being complicated. It'd be cool to have people devoted to simply laws to keep things as simple as possible.
I don't think it's because I'm used to Algol-based languages (C, Python, etc.). Every Haskell code I've seen is plagued with a plethora of operators which aim to make the code concise but it's not obvious what they do just by looking at them: https://academy.fpblock.com/haskell/tutorial/operators/
Isn't that pretty close to what something being intuitive means? Ie. something being intuitive means that it's easily understood or learned without requiring prior knowledge or instruction.
Once you understand what the combinators do in a particular context the code often reads very easily, and what it does is very intuitive and natural.
Best example I can think of is applicative parsers, where there are scattered <*> and <$> operators around, but if you just ignore them the parser just looks like a very intuitive description of what the expressions you want to parse look like.
If I didn't know programming and picked at random some Java program, I wouldn't understand much either. If I didn't take a single math course and I tried to read a math formula, I wouldn't understand the operators or anything either.
The standard should be: how much can you understand once you learn the basics and do a couple of tutorials?
Haskell has some up-front incidental complexity, but it's a difference of degree not kind from popular starting languages. It's easy to underestimate how much you had to learn about Algol-style languages that you've just internalized so well that you don't even realize. I've taught some complete programming beginners (high or middle schoolers) Java and Python and, especially one-on-one, I've seen how they get confused by things I did not even remember required explanation!
For example, in Python, people put : in the wrong place and don't understand how = works. "Basic" syntax like : and = are far more complex than we realize, we've just learned the basics so well that it's second nature. It's similar to how native English speakers don't even realize we have rules for adjective order ("big red ball" vs "red big ball"), while language learners run into it like a brick wall.
#haskell on freenode in the 2000s was a really fun place. I could not for the life of me convince anyone else to use the language.
the perl6 guys ended up inventing the new perl (now renamed raku)
https://rakujourney.wordpress.com/2024/10/12/raku-burritos/
the first perl6 parser (pugs) was written in Haskell btw
however, there are certain sub languages (slangs) where the use of punctuation chars as operators is very helpful, of course regex is the most compelling where the code is parsing one char at a time
my regex url {
^ # Start of line
[<http> s? '://'] # Match http or https
['www.' ]? # Optional www.
[<[\w\-]>+ '.' ]+ # One or more domain segments
<[a..z]> ** 2..6 # TLD, 2 to 6 lowercase letters
[ '/' <[\w\-./?%&=]>* ]? # Optional path/query
$ # End of line
}
it was inevitable that a simpler language - Python - would take the big slice of scripters who wanted something easier to learn and to read
I'm sad that perl no longer has a credible niche for script level coders who want to learn a more powerful and more expressive language than Python ...
It was such a great time and place for a young person interested in learning about Haskell.
yield*
function*
a?.b?.c
function f(x = 3) {
x ??= 3
...
a!.b
Type<T>
type A = B extends C ? D : E
...
In Haskell, the lack of parentheses for function calls plus currying means that to read a function call, you need to already know how many arguments the function takes, which I feel adds a new level of difficulty over languages where you can often guess what a function does based on its name, without looking it up.
As a result, often Haskell reads more like math, where without knowing each symbol’s definition, you’re lost.
I’ve seen cryptic JavaScript too, but less often.
While I agree with the general sentiment of what you are saying, note that the syntax has nothing to do with it, it is purely about Haskell using currying excessively. The syntactic translation between Haskell and JS is straight-forward and 1-1:
f x y z -> f(x)(y)(z)
f x (y,z) w -> f(x)(y,z)(w)
Yeah but the JavaScript operators are almost all universally understood. There are some exceptions and mistakes, like `===` which shouldn't exist, and `.?` which is new but fairly standard. But most of the operators are just standard maths stuff, normal array/field accesses, etc.
Haskell has normal operators plus a bunch of weird operators that you need to learn. Even if the raw number of operators is only 2x, the number of operators that you have to learn is probably 10x.
Especially when you consider that Javascript operators are fixed so you have to learn them once, whereas Haskell operators are user-defined and Haskell programmers love defining them.
* $ is "function application" - f $ x = f x, but lets you elide nested parens: f $ g $ h x = f (g (h x))
* & is "reverse function application" x & f = f x. This reads nicely in pipelines as "and"
* <$> is "fmap" what is like a lifted application over a structure: f <$> [1, 2, 3] = [f 1, f 2, f 3]
* <&> is "reverse fmap", which is nice when you have a big lambda
* Anything in <_> brackets is generally a "lifted" version of some operator
* And so on
Sure, you can play it on "hard mode " and do it with paper and pen for the lulz, but my experience was extremely efficient, fast, and straight-forward.
In comparison my Canadian passport renewal (damaged after 3 years) from California took 4 months. Was entirely paper based and had ridiculous requirements such as requiring a reference and a photograph stamped by photographer.
I know there's a digital pilot that's ongoing. That should be rolled out ASAP.
And they expire in 5 years, not 10 like many other countries.
Even though the consulate has my biometric data, I need to visit them in person.
My only saving grace is that they sometimes visit my city., and I can register there. But that registration is only valid for 1 year, and they don't visit every year...
And that is still 2h+ one way of travel for me, for something they already have.
don't they need to re-record them after X years? I thought that was the whole point of renewing an identity document/passport.
It is a shame that more businesses dont care about their customers enough to invest in such a well designed website. Well done HMPO.
I would have much, much lower expectations for a body shop like Infosys or Accenture or Thoughtworks. They probably wouldn’t implement the design system right without billing extra for attention to detail.
Anyway I had to submit:
Certified copy of my birth certificate, order from the UK general register office.
Original (not photocopy) of Japanese family register and translation.
Certificate of acceptance of notification of birth, original and translation.
Original Marriage certificate and translation.
Colour photocopy of every page of my childs Japanese passport.
Copy photocopy of the passport of an American or British citizen who confirms that child is mine.
The process is pretty unclear, and in general you seem to have to just keep submitting documents until they are satisfied.
In fact `applicant's father` (me) was born in the UK in this case, but `applicant's father's father` was not, the cause of the extra complications.
Weirdly this never came up for me (and I'm in a similar situation). I suspect what documents are ask for varies a lot by who is assessing the application.
Hope it all works out!
But why?
Qatar is a fascinating country, though.
I was asked to provide (I am British, wife is Hungarian, living in Hungary) both parents’ birth certificates and all four grandparents’ birth certificates.
My wife is not in touch with her father, so we had to submit a cover letter explaining the situation, which was accepted (although I don’t really see why it was relevant anyway, my son’s Britishness is derived from my Britishness, her nationality and the nationality of her parents is essentially irrelevant).
We also didn’t have to copy the passport of anyone else, but we did have to have a family friend do the confirmation online (family friend is a lawyer in the UK, so was on the list of approved jobs for this).
This is essentially happened to me, the "75% of the initial doc requests turned out to be irrelevant" was referring to this sort of thing.
All I needed to do was submit my own details, and a copy of both the marriage certificate and birth certificate for the child.
The assumption was made that as a married couple the child had me as the father, and as I'm British then so were they.
No references at all to grandparents, or even the mother's details, in the online wizard I filled out. Quite a quick process, which reminds me I probably need to renew their UK passport since it has expired a couple of years back.
The most difficult part of the process (not dealt with in this version of Passport Application but maybe a future DLC pack?) was actually finding someone who could certify my evidence (you are meant to submit originals but they keep the docs including passports for 3-6 months which is a bit unrealistic if you are living abroad). I can't remember the exact rules but it wasn't possible to use a US notary or a normal solicitor certification process and instead I needed to go to a council office.
After calling about 5 councils all of whom disavowed any knowledge of the process or its requirements I ended up finding someone at Islington Council who was delightfully helpful. But it was one of the more frustrating UK government interactions I've had.
One of the interesting rules the OP doesn't appear to mention is the 'mutable history' rule. Herein, the facts contained in a single document can alter over even short periods of time requiring that a document, already scanned into the Examiners system of fact recording, can be required to be returned for subsequent scans. Verifying documents have not altered in their facts is, some say, tacit acknowledgement by the Examiner breed that a multiverse exists.
The communication channel between the "NPC" telephone agents and the Examiners appears to be somewhat akin to prayer, only vague inferences can truly be received.
Aside, I suspect the OP maybe be a Mornington Crescent aficianado; a game seemingly similar in rule-topology to the aforementioned Passport Application.
Not to mention the passport office has forgotten previous decisions for my own passport where this sort of surname discrepancy has been explained away. (Including one passport where I am listed on a page of her passport!)
The NPCs are infuriating. I called, emailed and received a different answer every time. It was like a poor LLM.
After applying I had to explain to my mother that within the same batch of applications, one daughter required additional UK documents but the other one didn’t (the passport came), purely because one was born in the UK and the other one wasn’t. She rightly flipped out, raised it with an MP and it was resolved within a day.
It was interesting to speculate what exactly the one child whose passport came months later had done that warranted the Examiner's extra scrutiny. Something to do with reincarnation I expect.
Long story short, I demonstrated I was born in England and that I have citizenship (a passport) so no matter the possibility, it must be something that can be passed on. That doesn’t match the guidelines of documents required but it’s much less of a pain I reckon.
This is insane to me, coming from a country that simply has a central database of all citizens and foreign residents, and a citizenship check is probably something like a single SELECT on an SQL table.
I am willing to bet, with extremely unfavorable odds to me, that is not true
People born abroad are often citizens.
The British have had an insane number of changes in territory that affect citizenship.
Some of their current citizens have citizenship because they were born in Bombay (Mumbai). Others because they were born in Hong Kong.
Maintaining a reliable list of citizens requires regular registration, and loosing citizenship if you don’t register.
Despite being born in the UK to a British father my character wasn't previously British due to parents being unmarried and being born after 1983 but before the 2006 rules update.
I'm not totally sure when but I think some time in the 2010s the 2006 update became retroactive so my character was allowed to play. Didn't have to do too many side quests - the main one was retrieving father's birth certificate.
My character also had to attend the endgame ceremony and swear allegiance to the queen, which seemed odd for someone born in the country and lived here all his life
I can’t recall the article I read but this is a good presentation.
https://tomassetti.me/wp-content/uploads/2020/10/2018-04-10-...
- For the mynumber, first time, it was a simple application, and maybe a trip or two to the City Hall. This was at a moment where Japan was trying everyone to have/use the mynumber system so that might've made it easier on purpose.
- The second time I needed to book an appointment to request the application forms, which were (snail)mailed to me so then I could apply for the card. Yes, that sentence is as bad as it reads.
Now the crazier one is the driving license (conversion process):
- The first time I needed to get a translation, I went to JAL, showed my documents and paid, then received the translation. Took that and 2-3 documents, went to the driving license center, and gave the documents and passports and IDs. Waited few hours and had it done. Oh, I also got the Motorbike license for free.
- The second time I am applying now: need to get the translation, first register online for the translation, for which you need to create an account and a 8-step process. Hard, but still doable [1]. Then you need to book a meeting for the driving license center. But it's very hard to do so, in fact I couldn't find the link at all. I went in person and they showed me a QR code for the booking, I suspect this is hard to find on purpose. Every place is full and cannot be booked, except for 1 of them, 2 months later. Okay, I try to apply. First basic questions, sure, then asks for travel history on my passport that I need to input manually. Note that dates in one part of the form are on the shape of `YYYYMMDD` and in other they are 3 fields of "YYYY", "MM", "DD" (3 different inputs), having to write a dozen of them is maddening. Oh wait, but if I want the motorbike one, which was automatic before, I now need a document from the Spanish embassy as well, another side quest. I'm hoping they can at least provide it in Japanese. I guess I'm halfway that process now, got a meeting booked 2 months later.
[1] https://english.jaf.or.jp/driving-in-japan/drive-in-japan/ab...
(Spaniard here too...)
AFAIK this is only available for residents? People traveling can/should use the international driving permit. Both JAL (translation) and the center itself asked me for my resident card.
> do a short exam
Oh there's no exam needed at least! Unless you mean the eye test?
It’s apparently been a loophole for Chinese and Vietnamese tourists to get a license that is more recognized abroad then their own.
I think restricting this to residents makes sense; I’m more wary of making it more complex or difficult in other ways, but I also have that quest behind me so I don’t really care to be fair.
As to exam - that depends entirely as to where your “original” driving license is from.
Some countries just need a sufficient amount of paperwork (which sometimes is more difficult than the exam), some require just a theory exam, some require both theory and practical exam. In case of the US, it also differs state by state, because of course it does; why would anything ever make sense.
Yes, that's why I was replying that since we both are Spaniards "(Spaniard here too...)"
But the documents required (at least previously to recent changes) for the license transfer were [0]:
1. Application form for Japanese Translation of foreign driver’s license
2. A driver’s license (original, in principle)
3. A photocopy of a residence card or a resident record etc [...] only required for licenses written in Arabic or Russian and licenses issued in certain countries[...]
--
0: https://english.jaf.or.jp/use-jaf-more/drive-in-japan/foreign-nationals-licenseif you drive on the right at home, they will fail you the first time on principle. If you drive on the left (aus, brit) you have a chance to pass the first time.
It was a weird case, because the parents hadn’t actually been working for the foreign embassy for a while, and didn’t know they still held diplomatic status when their daughter was born.
They had entered the US as diplomats and shortly after decided to go local. That was probably their plan all along - lots of diplomats from third world countries do that.
It was a huge shock.
Kind of a “surprise, you’re not actually a citizen, even though both your parents and all your siblings are!”
She was deported.
My First Reaction: Given that the game can be read as a deconstruction of the concept of Nation States and Citizenship, why would the UK government run such a thing?
After a few minutes: Oh wait a minute...
>British passports are issued to those who have a claim to British nationality under the British Nationality Act 1981.
Has the British language really evolved that much in the last 50 years?
Legal copy written in 2025 could be considered arcane.
It doesn’t mean the language of the time is hard to understand.
Changes to Appendix EU
APP EU1. In Annex 1, in sub-paragraph (a) of the definition of ‘continuous qualifying period’, after “(b)(i)(ee) below”, insert “(or unless sub- paragraph (b)(i)(ii) below applies)”.
APP EU2. In Annex 1, for sub-paragraph (b)(i)(ii) of the definition of ‘continuous qualifying period’, substitute:
“(ii) (where the person has limited leave to enter or remain granted under paragraph EU3 or EU3A of this Appendix) any period(s) of absence which did not exceed a total of 30 months in the most recent 60-month period, as at the date of application or (as the case may be) at the date on which, under paragraph EU4, the Secretary of State is considering whether to grant them indefinite leave to enter or remain under paragraph EU2 or (as the case may be) EU2A, without a valid application under this Appendix having been made; or
(jj) any period of absence due directly to an order or decision to which sub-paragraph (b)(iii) below refers, where that order or decision has been set aside or revoked; or”.
APP EU3. In Annex 1, for sub-paragraph (c)(v) of the definition of ‘continuous qualifying period’, substitute:
“(v) a relevant reference is concerned; or
(vi) sub-paragraph (b)(i)(ii) above applies, where, under paragraph EU4 of this Appendix, the Secretary of State is considering whether to grant the person indefinite leave to enter or remain without a valid application under this Appendix having been made”.
But with the recent LSP decoupling of the IDE to the compiler, it is quite possible to make good tooling that is independent of any particular editor. It's just that the language's popularity is what induces contributors (as most are realistically after "fame" and "portfilio" when doing contributions).
It has a feature-rich LSP, code formatter, package manager, dead-code checker, configurable linter, thread debugger, memory debugger, vulnerabilities checker, and much more.
That’s just what is provided by external tooling. Then, it also has everything the compiler has to offer, which is bit more than what most languages do. For example, you can now compile to JavaScript or WASM.
No it isn’t. This is a stupid meme that people just dogmatically perpetuate.
Also, you can build pretty much everything in Haskell with a plain text editor and ghci. You get better code that way too!
Try France, or maybe Germany.