The Norwegian government's BankID system (used by all Norwegian banks, and also supported as a login mechanism across all the government's digital solutions, including the national paperwork portal where everyone files their tax returns) only migrated away from Java 18 months ago [1].

[1] http://www.dinside.no/931165/dnb-dropper-java-fra-november

I Germany, the web site of the German equivalent of the IRS uses a Java applet to do the Umsatzsteuervoranmeldung (don't ask me what that is in English, though).

So it's not banking per se, but it's finance-related.