Bloomberg Runs on 25M lines of Fortran (2006)
etrading.wordpress.com
etrading.wordpress.com
edit: Can mods put (2006) in the title? Seems reasonable...
And NO not knowing fortran doesn't count if at 18 at my first job was told oh hear is a book on fortran learn it your graduates with a fancy CS degree can do the same.
But I'd have to see it to believe it for a straight out of college kid and rudimentary familiarity with the language.
It's not like learning FORTRAN (or C, or Pascal, or COBOL) will make someone unable to program in other more markeatable languages.
Of course, if you're doing it just for fun, then I'd say there's no negative. But from a career perspective, you'd be better off learning more relevant skills.
Further, what you used your last job matters. Particularly if that job is the _only_ real job since you got out of school. If you just learned Fortran for fun or for a job in a long list of jobs and your github account is littered with good work in multiple languages, Fortran can indicate flexibility, diversity, and a a broad skill set.
If all I see is one job with Fortran for 2 or 3 years after school I might hesitate to hire you for a <insert language here> job at anything but entry level.
edit
Just a note, I think Fortran is a fine language and anyone whose done a decent amount of linear algebra work knows what a big role Fortran plays in analytics.
My main point is that you should care about what your first job(s) are coming out of school because they can shape what the next jobs are going to be. So make sure it's something you want to be doing.
Isn't it still commonly used for scientific computing? So, maybe it would enable you to work on other kinds of neat stuff than the latest javascript UI framework.
I have not known a single person in over a decade to go out and specifically learn FORTRAN as a separate language. Schools still provide graduates with C programming skills and any C programmer comfortable with the language can not only immediately read most FORTRAN, but can modify it as well.
As well as vast numbers of religious flamewars. I'm personally of the camp that (C || Fortran || "C with classes") + (Java || C# || Python || ${HIGHER_LEVEL_LANGUAGE}) is better for nearly any complex, performance-sensitive task.
I'm with you on combining the best tool for the job and sometimes having a Java/C/Python wrapper is useful, but you can retain your Fortran tried-and-true-and-tested code-base while adding feature enhancements compiling against the Fortran2003 standard and often eliminate the need for wrappers. I guess the practical problem is finding F2003 engineers is way more challenging than a Java or C# dev.
0: F90 was the "big" upgrade that came out which made Fortran "modern", but there have been continuous upgrades like F95, and Fortran 2003 (I guess this would be analogous to C++17) which make it perfectly fine to write.
I much prefer c/cpp, but fortran remains pervasive in scientific computing and being conversational in it remains a prerequisite for fields in high performance computing, IMO.
That's why all the Bloomberg functions have four-letter names.
Why be offended, rather than excited to get to come in here and to expound on what you do for a living? It's what makes the comment section great.
IMO, comment sections aren't really a great place for discoverable information, so maybe I'm thinking more in-depth posts need to be created that actually peek behind the curtain a bit so they are ranked better when searching. Right now, you can only glean information about certain areas/technologies from individuals that represent them if you know where to look. e.g. Here's a recent video interview with Matt Hunt from our Portfolio Analytics team regarding some of their problems/infrastructure https://www.youtube.com/watch?v=cMj2V3U4J3k
They could change that perception with being more open about how they go about developing software (like their GitHub account). But these article showing up every one and a while goes against their efforts hence why I can see why OP may get annoyed about it.
Sure, visiting Bloomberg and getting the tour leaves you with a wonderful expression, but those that I know that have worked there in the past wouldn't compare the work environment to Google.
Bloomberg is much more of an "East Coast" company, and feels like it.
Having said that, I had a Bloomberg Terminal subscription for five years. I friggin' loved it. Regardless of whatever might be behind the screen, it's still an amazing, powerful, gorgeous tool. I miss my B-Unit, oh so badly.
Check out the new SF office we opened up if you're looking for a more West coast feel.
On the non-technical side of things, they also struggle with hiring developers who are capable of (or willing to) work with Delphi, especially junior developers. A significant portion of their staff also doesn't know how to work with Delphi, so reassigning resources to match the needs of the business is often a challenge.
I imagine these issues would probably apply to Fortran as well.
Lifecycle of languages is a hard issue regardless of language. Unless you keep switching to the correct new shiny thing then you will have developer issues. I think their are a few more Fortran programmers than Delphi these days.
Python? Good luck. Every frigging month (practically) it's a breaking change. You have to work furiously just to keep your infrastructure working, or purposefully draw a line in the sand (2.7!).
I run Fortran every day (via NumPy and other packages), and some of us are investigating just switching to it for some projects (we need the math speed, and all the C++ matrix libraries involve compromises of one form or another). There's no reason I and my peers wont be doing the same in 50 years. It works, its fast, its debugged, and it is easy to use things like Python 5000 (by then over 70% of people will have made the transition from 2.7 to the 3.x version!!!) to do all the glue work that Fortran is not optimized to do.
Fortran codes ain't ever going to be a liability[1]. Python 2.7 on the other hand...
[1] because of being Fortran. Poor coding style was endemic in Fortran77, and that can be crippling, but to a large extent that is language agnostic. I acknowledge the required short variable names of 77 doesn't help, but people would not comment what the variables mean, and that is the real problem (IMO). Poor structure is another problem (masses of gotos). Yes, 77 doesn't make your life easy there, but discipline is possible. We can quibble about all this; my point is the 'modern' languages are going to be old quite soon, and we will face real porting problems then. Move 20,000,000 lines of JS and Python? Good luck.... Get a Python 2.4 code base working with 3.9? Ah, hmm, well .....
But Fortran? We build it and run it every day, easy peasy.
What? There was exactly one breaking point, from Python 2 to Python 3. Which "monthly breaking changes" are you talking about?
I know each issue is easy in isolation, but we're talking about old 25M line codebases here. If such things exist in Python it can be a maintenance problem.
I'm in favor of languages making breaking changes occasionally but there is something to be said for languages like Java or Fortran that are intended to keep compatibility forever.
That Fortran gets bashed and Lisp is worshiped seems quite incongruous.
One breaking point?
The majority of the libraries that I use don't even work properly in Python 3. The standard Mysql library for instance, which is used in the majority of ORMs (including SQLalchemy) isn't included in Python 3, which means you need to make major changes/try to use a different library.
The case of many standard libraries were changed, breaking lots of code. Example: ConfigParser to configparser.
Even simple things like print 'test' is now print('test')
These are only a few examples. There are lots of other major changes that break 2.X code.
I have been using Python 2.X for all of my projects and wanted so badly to change to 3.x for my new projects. However, because of all incompatibilities, I can't.
I don't think I'm alone on this either. Python 3.X still isn't being used by the majority of developers.
"Gurer jnf rknpgyl bar oernxvat cbvag, sebz Clguba 2 gb Clguba 3"
Bar oernxvat cbvag?
Gur znwbevgl bs gur yvoenevrf gung V hfr qba'g rira jbex cebcreyl va Clguba 3. Gur fgnaqneq Zlfdy yvoenel sbe vafgnapr, juvpu vf hfrq va gur znwbevgl bs BEZf (vapyhqvat FDYnypurzl) vfa'g vapyhqrq va Clguba 3, juvpu zrnaf lbh arrq gb znxr znwbe punatrf/gel gb hfr n qvssrerag yvoenel.
Gur pnfr bs znal fgnaqneq yvoenevrf jrer punatrq, oernxvat ybgf bs pbqr. Rknzcyr: PbasvtCnefre gb pbasvtcnefre.
Rira fvzcyr guvatf yvxr cevag 'grfg' vf abj cevag('grfg')
Gurfr ner bayl n srj rknzcyrf. Gurer ner ybgf bs bgure znwbe punatrf gung oernx 2.K pbqr.
V unir orra hfvat Clguba 2.K sbe nyy bs zl cebwrpgf naq jnagrq fb onqyl gb punatr gb 3.k sbe zl arj cebwrpgf. Ubjrire, orpnhfr bs nyy vapbzcngvovyvgvrf, V pna'g.
V qba'g guvax V'z nybar ba guvf rvgure. Clguba 3.K fgvyy vfa'g orvat hfrq ol gur znwbevgl bs qrirybcref.
Is there some reason you ROT13'd it?
I rot13ed it in case there's a dead-comment-reposting-killing algorithm on the loose.
Are you using lots of really tiny matrices? I would think pretty much all the libraries could just be linked with a good BLAS distro and then it would not matter what language that you use. Personally, I'm a big fan of the Armadillo C++ library.
Sorry I really love Python but I see it as a slow weighing beast that is always a good second best choice for everything it does. Great if you only know Python or only want to know Python but for production? Speed issues is the main reason why Python is not more widely used.
Its probably being mentioned as such because a lot of the reason people use Fortran is because of key high-performance libraries for particular numeric application domains -- not because Fortran is necessarily the best language, aside from the existence of the those libraries, to express higher-level solutions in those domains. And because Python has convenient wrappers for those libraries, such that where that is the motivation for using Fortran, Python is a reasonable alternative.
> Speed issues is the main reason why Python is not more widely used.
Sure, and that's why you wouldn't (with the current implementations available) want to write the low-level numeric routines underlying NumPy in Python.
OTOH, once you have NumPy, a lot of time Python makes perfect sense as the language to use for higher-level solutions that rely on the functionality provided by those libraries.
Here is NASA's benchmarks using Fortan vs MATLAB vs Numpy vs Java. pretty evident that Fortran blew everything away. https://modelingguru.nasa.gov/docs/DOC-1762
More typical is about 3× or 5×, due to iterating over the data many times (once per Numpy call) instead of once, thus bottlenecking on main-memory bandwidth.
But about 18 months ago, I took over maintenance of a small-ish Delphi application when the original developer left the company. It's not a super-fun language for me, but to me it was fairly easy to get into.
Do you know FORTRAN? Do you know any programmers who know FORTRAN? Do you know anyone who wants to learn FORTRAN? Unless you're in the community the answer to all of those is no. And I can't imagine it's a big or growing community. The obvious consequence of this is FORTRAN will become more and more expensive to run a business on as time goes by because fewer and fewer people will have experience with it or any desire to use it. It's going to be harder to find people to port it and even harder to find people to maintain it.
That's why it's a problem.
If you know SQL or anything about databases SAS is very easy to pick up. My background from University was Java and C I had few difficulties.
My House-mate is a statistician he works in finance and uses R my understanding is R is a bit more difficult as it doesn't have anything like SAS's Proc SQL never used it myself though.
Coincidently Fortran is used very heavily in models here as well.
As to it becoming "more and more expensive to run it", that kind of depends. Most of that Fortran code isn't touched very often. The cost of swapping it out is huge.
Swapping it out piecemeal isn't as expensive money-wise, but it carries large risk. Should it be ported out? Sure. Can it be, in any reasonable way? That's a harder question, and it's not as cut-and-dry as you make it sound.
a little
> Do you know any programmers who know FORTRAN?
many
> Do you know anyone who wants to learn FORTRAN?
many
> Unless you're in the community the answer to all of those is no.
Fortran is still widely used in mathematics and science. So if that's the community bubble you're referring to, then yes, there's definitely a niche there in my experience (but not purely for legacy reasons).
There are a million different versions of the same function each with a minor tweak to get the "right" behaviors labeled func_name_1 down to func_name_1000000 because everyone is too scared to fix the original function as they don't know what depends on it.
It's not that Fortran is the right tool for the job it's that it has terminal momentum.
> There are a million different versions of the same function each with a minor tweak to get the "right" behaviors labeled func_name_1 down to func_name_1000000 because everyone is too scared to fix the original function as they don't know what depends on it.
Sounds like a problem with development practices, documentation, and overall code lifecycle management -- and also employee management in terms of skill development priorities.
Those issues aren't particularly tied to implementation language.
This is just my experience with it. Somewhat to my surprise I really ended up liking Fortran, and the new code doesn't look at all like the old code. For what we did it was also slightly faster than C. If I were to write a numeric program I wouldn't think twice about what language to use. I feel like Fortran is an undervalued language for calculations. But then again, I am by no means an expert.
Now, if Bloomberg's code is a spaghetti monster (I have no idea), that is a problem. But that is a problem of legacy code, not a 'Fortran' problem.
I think it is a bad article based on hearsay by one ex-bloomberger.
This I think is the key point: lots of time when I've seen people mistake a practices problem like this (that result in expensive-to-maintain "legacy" code) with implementation language problems, they end up doing a conversion to a new language (often doing the worst thing you can do with a poorly specified, poorly documented, but basically working codebase, a "big-bang" replacement rather than an incremental one) without fixing the process issues, and so end up creating a new poorly-specified, poorly-documented, often less-well-working "instant legacy" codebase that just happens to be in a currently-fashionable language.
More concerning is the bit where the author says that they have "1500 coders firefighting, just keeping that legacy monster alive."
I wonder if that has gotten any better since 2006.
Fortran code isn't inherently technical debt, any more than code in any other language is.
> Do you still have a VCR? Isn't it a bad idea to assume VHS is a good medium to archive?
I fail to see the relevance of the analogy; its not exactly as if there is a shortage of -- F/OSS and commercial -- Fortran toolchains maintained and available.
Sure, you may be able to find tools to use Fortran in the near term. But how do you protect the future. Does it play well with emerging systems/protocols? Do you have developers to maintain it? Is there a clear path to upgrade to newer platforms and frameworks?
There may be challenges getting the talent you want with legacy systems and I believe you're building a technical debt as you'll hit a wall with your infrastructure and have to upgrade/migrate one day anyway.
The top comments on this posting are ex-employees talking about how they've been migrating many (most?) parts of the system to C++, so yeah, even Bloomberg management agrees staying on FORTRAN is probably a bad idea.
Most Fortran hate is actually F77 hate.
Fortran 90, on the other hand, is basically C with a different syntax. F90 (and newer: F95, F03, F08) really isn't a worse language than C, and there's no reason to hate on it when you wouldn't hate on C, but a lot of people don't know it exists because they equate Fortran with F77.
My own take on that was: "Wow, they surely must do a lot of performance sensitive stuff to use the fastest language available for number manipulation. Kudos to them."
I don't remember the exact figures but 25m lines of Fortran seems plausible, but it was still the minority of the code base. The majority at the time was in C and C++, with probably a few million lines of Perl and Javascript as well.
Plenty of the Fortran dates from the 80s, rewriting it in a modern language would be a huge project which lots of risks and limited upside. No significant new fortran has been written for a long long time (>10 years).
It's also a mistake to think of Bloomberg as a single piece of software, it's closer to an app store, there's are thousands of specialist apps (functions in Bloomberg terminology) and most users only use half-a-dozen, but which half-a-dozen varies significantly between users.
This makes it very hard to displace wholesale. There are individual pieces a competitor can go after but identifying a subset of functionality that's enough to convince users to switch is hard.
There's also a lot of stickiness which goes beyond pure functionality. Network effect and brand are also key parts (having a Bloomberg Terminal is a status symbol).
From what I recall there's a ticker somewhere which tracks lines of Fortran code which I vaguely recall being in the millions.
I have found that often "high profits" get in the way of customer service because they are a disincentive to "quality is free" thinking and make it possible to sustain the unsustainable for way too long.
To take an example, Cable companies are so profitable that they think nothing of the cost of replacing cable boxes that break, or of excessive truck rolls. These not only cost money but they anger consumers. My mother-in-law quit cable in disgust and switched to OTA TV after she had three cable boxes burn out in three months and would have to go stand in an long line to return it and pick up a new one.
Quality is free and screwing up is expensive -- even if you can afford to screw up it costs you customers and it costs you employees. But again, turnover is no problem if you are making enough money you can afford to spend 5x what things should really cost.
http://jobs.bloomberg.com/go/All-Open-Positions/515100/?q=&t...
BBG's killer features are support and _chat_ - chat with other _trusted_ counterparties who also paid the admission price.
Great place to work, too bad it's in NYC
Market share has grown since this was written in 2006.
Market share in 2007 26% and 2011 30%
Money: 315,000 Bloomberg Terminal subscribers (2 year subscriptions) worldwide at $20,000 per user = 6.3 BILLION dollars a year!
EDITED for spelling
As for Bloomberg there terminal and all of the services that they provide is top notch. I have a lot of friction with all of there data, and this is by far the best i have worked with.
Its very much like message boards. HN and Reddit have the hot hand and the connections overcome better technology of other boards. Once the connections become too toxic, people will migrate to the next thing. Then you might have a shot, but the group with the best connections will probably be the next thing.
Its not like you can throw together some trading system with an eventually consistent datastore in the back.
the entire finance industry is based on trust. Part of that is trust that bloomberg is correct.
Loosing a couple of trades without an audit trail crashes markets.
And support. Being able to pick up the phone any time at quickly get someone on the other end who really knows their shit is a big thing you're paying for.
That's not consistency, though, that's durability. Consistency in banking is basically the part where you don't allow people to spend money twice. That can be "eventual" and violating it can be OK under some circumstances, like when your bank issues you an overdraft (and applies a fee) or when your airline sells too many seats.
So the eventually consistent data store in the back is fine... as long as it's durable and as long as you're willing to invest man-years of effort in understanding the implications of your selected consistency model so that you don't screw it up and can perform operations which are compliant with some set of business rules that the business understands and approves of, naturally.
A trading platform must have a 1:1 mapping of buyer to assets when trades are agreed. You cannot have a situation where two people buy the same share, as it requires unwinding. A trade must be atomic.
traders want to see all the bids/asks of the whole market. Otherwise you'll effectively have a random assortment of tiny markets under one supposedly united trading platform.
Thats the other thing, trades need to be reversible. There is a good correlation between massive trade volumes and the need to reverse transaction. Having an eventually consistent model is just not going to work.
Being overdrawn is not eventual consistency. The act of an overdraft is not a failure of a bank to properly account for your balance, its a deliberate business move to generate cash.
So no, its consistency, one price, one bid, one ask.
But if you care about operating the EXCHANGE ITSELF then your allusion to the audit trail is a bit of a red herring, isn't it? In fact, the data store itself becomes pretty irrelevant for the live trade-matching - presumably you shard just the hell out of your data, keep the working set in RAM, keep most of your stuff colocated in the same place and just solve for availability by doing something crazy and hardware-intensive like having hot spares for everything.
$99/mo/user vs $21,000/year/terminal. Wow! I wonder what other kinds of entrenched, expensive products like the Bloomberg Terminal are in need of competition. I think this requires industry-specific experience to know, however.
A lot of scientific control system apps get built with LabVIEW or Agilent VEE. If it makes beyond the niche consultancy stage, such things are often (re)written in whatever an engineer happens to know: C, Python, Haskell, LISP, [favorite language religion here].
Edit: BTW, just came across a supposed successor to VMEbus [0] (a standard industrial data bus): VXIbus [1]
0: https://en.wikipedia.org/wiki/VMEbus#/media/File:VMEbus.jpg
[edit: I wonder what a modern version of RATFOR would look like]
Every time I see suggestions like this, I remember minions, rushing from one framework to another - "koonga la mala makuna, koonga!".
People skilled in these legacy languages and technology stacks are also amongst the highest paid and most in-demand in the country, at least here on the east coast in places like New York and DC.
Fortan programmer: 45,000 GBP (3 month avg), +23% salary change from last year
C++ programmers: between 47K GBP to 100K (for c++ quants).With 3-5% average salary change from year to year.
I don't know Fortran; but looks like the fame about its high costs comes from mainframe hardware (which also runs C++, btw).
How can you trivially conclude that its would be easy to provide the same speed as Blomberg terminal ?
If I am not mistaken even shaving off fractions in terms of time has a lot of value.
And maybe you need 25 million lines of FORTRAN code to achieve that efficiency ?
This sentence makes me dismiss the entire article:
"The trading model and real time issues are just technical problems. Google is smart enough to solve them."
And I have written mission critical Fortran software for big telcos' and tust me nothing contreates the mind when one on Vint's reports nudges you and says this had better be right or we are both looking for anew job.