Hacker News Reader

Comment by NickGerleman | Hacker News Reader

This is really cool but I'm curious why Bloomberg would need this. Ie, what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this. Was there nothing off the shelf that would have fit their needs?

I don't mean that in a derogatory way, I'm just curious what motivated making this.

View on HN

"Non-engineering company" -- ha!

But to answer your question: Mike Bloomberg's autobiography talks about this. When they started in the 80's there wasn't as much great off-the-shelf software as there is today.

Their customers were (and always have been) insanely demanding when it comes to reliability and speed. The last thing Mike wanted to do was be caught sheepishly explaining to the CIO of Merrill Lynch, "Well, gee, our Oracle database has this bug they can't fix for the next two weeks..." or even "...this optimization they can't make for the next six months"

This is a company that invented its own layer-2 network protocols 35 years ago just to squeeze every last drop of performance and reliability out of the hardware. Of course they wrote their own database (actually two -- there was a comdb 1 of of course, too)

Just for some scale, we have 4k+ engineering employees and if we were a public company, we'd be somewhere around the 5th largest software company in the world by revenue. (Strange stat, but our primary product is software and accounts for over 3/4 revenue)

Looking forward to seeing a fastsend / PRCCOM paper one day.

PRCCOM is no more! Don't need to keep bringing that one up :)

Oh, in that case I hope you guys see fit to write it up one day. It's a legitimately interesting and clever solution, even if obsolete.

I definitely wouldn't call Bloomberg a "non-engineering" company if you've ever worked with any of their trading / market data products. Pretty much true of any trading side finance company nowadays.

Bloomberg started as a technology company. Sure, they have expanded into media, but I'm pretty sure they are still first and foremost a provider of software (plus hardware) and data.

I would bet that the Terminals are responsible for something like 140% of their profits, and that the media empire is a huge loss leader / vanity project.

The whole point of Reuters news is to inform traders about things that will lead to more trades being done over the Reuters platform, so it's entirely likely Bloomberg operates the same model.

So. We did a few things better than the industry to this day. HA is a spectrum. Transparent masking of all failures in any state of arbitary SQL transaction is on the far right side of that spectrum.

To directly answer your question: go look at where 'replication' was on rdbms 14 years ago. PostgreSQL? MySQL?

14 years ago Sybase had strong replication. Sybase was wall street's darling for a long time starting in the late 80's.

>>what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this

They probably open sourced the database, and would do more -- to dispel the above notion, and therefore to attract more qualified recruits.

Financial services companies, and technology suppliers around them, have came up much of the 'modern day' tech, some times decade(s), before google's and facebook's of the world

  -- have had NoSQL with stored procs(eg Goldman S, early to mid 90s ),   
  -- in-house grown programming lang (APL+ based in Morgan Stanley) emphasizing vector-based operations
  -- Smart contracts 
    https://www.lexifi.com/product/technology/contract-description-language
  (where a contract is represented as algorithm specified in a domain specific language) 
    ).  This was way before etherium, started, I believe, in Credit Swiss (but not sure)

   -- one of the fastest time series database (kdb+)
   -- Data science and machine learning (modeling risk and valuations)

Today's hedge funds manage petabytes of data. So do many of the big investment banks...

https://bloomberg.github.io/comdb2/overview_home.html

>> We had several goals in mind. One was being wire-format compatible with an older internal product to allow developers to migrate applications easier. Another was to have a database that could be updated anywhere and stay in sync everywhere. The first goal could only be satisfied in-house. The second was either in its infancy for open source products, or available at high cost from commercial sources.

I've worked at midsize "non-engineering" companies that do boring stuff like logistics where they wrote their own RDBMS and even virtual machines for legacy hardware.

It's not that uncommon especially if you've been around a long time.

Bloomberg was doing cloud computing decades before the term was even invented.

Edit: As someone else points out, they were doing cloud computing before the internet existed.

People forget that Bloomberg had computer networks before the Internet.

I remember when we didn't even have comdb2.

Bloomberg is a massive tech company.

If even advertising companies are engineering shops, what makes you think fintech companies aren't?

Right, it's like everyone's forgot what Google's business model actually is!

The work started 12 years ago when there were probably no `NewSQL` [1] solutions out there.

https://en.wikipedia.org/wiki/NewSQL

Bloomberg, after all, is a high-tech news and information delivery network.

>Was there nothing off the shelf that would have fit their needs?

I can't think of any solid distributed RDBMS that would have been around in 2004. Does anyone with more knowledge have any idea (open or not)?

IBM DB2 and Oracle 9i were around for sure. Other solutions as well. DB landscape was well-established back then.

They were, but rediculously expensive at the time. $100k+ at the time to have a non-replicated version on Oracle running on a multicore server. Dealing with Oracle was like dealing with the mob.

Indeed, but that's a whole other issue.

DEC RDB might have done it, if it was still available.

Reply on news.ycombinator.com