MUMPS, the Archaic Health-Care Programming Language | Hacker News Reader

MUMPS, the Archaic Health-Care Programming Language

motherboard.vice.com

33 pointsvezzy-fnord49 comments

The hospital where my wife works just spent $1.2B (yes billion) on software written in MUMPS

https://www.bostonglobe.com/business/2015/05/31/partners-lau...

The company that provided the software is interesting:

http://www.epic.com/

Based in a suburb of Madison Wisconsin, founded in the mid 1970's by a grad student who wrote some software for a local hospital. They grew organically by word of mouth never spending on sales or marketing. They landed Johns Hopkins as a client years ago and now Epic chooses which hospitals are allowed to "invest" in their software. They hire young grads right out of college who have never worked anywhere else - who find themselves trapped with completely non-transferable skills. It is like this company evolved in complete isolation from any outside influence on software development.

Of course it is typical enterprise software developed by waterfall method that puts users last. My wife says the doctors are in open revolt. They have 1000's of open change requests - some critical to patient safety. For example the field to enter prescription instructions is only 64 characters. This is a hospital that administers chemotherapy so the instructions are more complicated than "take with food three times a day." The developer who got assigned that ticket had the nerve to suggest using abbreviations instead of making the change.

As a healthcare IT professional I find this to be a highly misleading portrayal of what Epic is and its place in the industry. Epic is admittedly typical enterprise software that doesn't meet the standards we've come to expect in the 21st century. But you have to take that in the context of healthcare which is at least 20 years behind in this respect.

Epic is far more sophisticated and good than 90% of the other options in use by real hospitals today. I'm only half joking when I tell my friends in tech that it's hard for me to talk about what I've seen. Not just bad software but horrible 80s era curses GUIs, truly garbage proprietary query languages, systems with 100 different pieces of shoddy vendor garbage bolted on here and there to cover the gaps in their functionality.

Epic is best in class--and that's sad. Maybe 20 years from now we'll have EMR and patient accounting systems that will please 2015 HN.

When I was young and dumb (now I guess I'm just dumb) and coming out of undergrad in the Midwest, I interviewed at Epic. Initially the experience was awesome - cool office spaces (the founder is an eccentric art collector), subsidized healthy lunch options, lots of young people around.

But after I left the interview I did some research to find awful Glassdoor reviews of the company - the worst I've seen. There were also articles which alluded to heavy lobbying (not necessarily a bad sign, just weird) to get their software and services legally or protectively into hospitals.

Overall an interesting company that I'm glad I avoided.

I do a bit of work with Epic as a 3rd party vendor, its interesting to say the least. The UX is pretty bad, but an internal human factors group seems to be gaining influence and decision making power in the organization. Its also odd because even as an older company, most of their workforce is in their mid 20's and early 30s, so change is coming eventually. That said, a bad Epic install will absolutely bankrupt a hospital.

And not to be a pedant, but Epic's big break was with Kaiser, Hopkins came a years or two later.

Most of their workforce has been in their mid 20's and early 30's for the last decade or so; demographic-inspired change isn't coming because they have such insanely high turnover.

What's entertaining is that MUMPS was developed at the Massachusetts General Hospital, which is now paying a third party over a billion dollars to use it in the form of Epic.

I never thought I'd suggest using a short URL on a prescription.

(fake) bitly . com /grandapas-chemo-meds

If you've used MyChart you've used EPIC. Both the hospital systems I've used here in the greater Seattle area use Epic for their EHR systems.

Mychart is a patient portal and hardly resembles their full blown emr. Its really just a snapshot of orders and results for a single patient. much of the complexity in an emr is the integration of services, query, and data input.

That's what you get when corporate heads pick the software vendor

Epic is the biggest EHR vendor.

I wrote MUMPS for a system called COSTAR (this was in 1989 or 1990). It was truly a terrible language. From what I've tried hard to forget: Everything is abbreviated, single letter functions everywhere. The database is a giant persistent hash table, accessed through programming language variables, with no schema, types or type checking (keep in mind this is used to store medical data). Everthing is global. There's virtually no structure in the language, I think they had subroutines but IIRC there were no named parameters. Existing programs were just unreadable spaghetti.

Edit: Not Co-Star, but COSTAR: https://en.wikipedia.org/wiki/Computer_Stored_Ambulatory_Rec...

Yes, you can use just the first letter of a function name. It might as well support Morse Code for all the good that does in terms of readability. I wish I knew what MassGen was thinking when they thought that was a good idea. I shutter when I think of having to use that language for mission critical systems (or any system for that matter.)

Keep in mind that MUMPS started around the same time as Unix (a little before, actually), so the same motivation for naming commands like cp, mv, and rm was likely at play in allowing abbreviations: interactive use on remote terminals.

They were thinking multi users on a few KB of RAM.

The whole development stack and runtime had to fit on a PDP-11/45. Contemporary assemblers and FORTRAN on comparable platforms were hardly better. COBOL required a much bigger machine. Pascal was still a slow toy from academia and C was in the future.

Try not to judge quite so much.

Sure .. on the other hand, it's not 1968 any longer, and we are talking about medical data, which if you get it wrong can have all sorts of life-changing/life-ending effects. It's probably time to think about removing such systems, if only from the point of view of patient safety.

MUMPS is a subject I've tried to study before but confess I didn't get very far. For my purposes back in the 90's, PostgresQL was the best fit and it has worked out well.

Meanwhile EHR systems have become a huge industry, but now having to catch up is hard. Learning about VistA is mind-boggling in its scope and complexity, and especially since it has been open-sourced for a long time. It's a point of curiosity that it's not been discussed more than a little here and there in the FOSS media.

So I appreciate the informative article, as it gives me some starting points to explore this vast field. Might even give another stab at MUMPS, though it's a hill to climb. However some of the open-source projects that grew out of (or on top of) VistA offer some interesting ways of approaching it.

One project with immediate appeal to me is EWD.js, which is a node.js javascript implementation of server and client via WebSockets, and a MUMPS DB backend. Should be interesting to try out. http://gradvs1.mgateway.com/download/EWDjs.pdf

I could be wrong but it seems someone just starting out could find opportunities in the field of health care computing. Given the political thrust to have EHRs everywhere seems logical the field will grow.

Above all, the need for security expertise in the EHR domain is enormous and will greatly expand in the next several years. The FOSS aspects might well create openings for innovators with creative products and ideas to reduce the cost burdens that high-priced EHRs have been contributing to health care systems.

I feel like this journalist was struggling to understand the nature of different kinds of databases.

> document-oriented databases (where different types of data are stored in one unified document instead of a bunch of tabular cells)

There isn't one document, and typically documents aren't "unified" in the sense of having an enforced schema.

> columnar databases (where data is laid out in key-valued arrays)

Er, no. Where data is laid out in columns, to speed up aggregate and analytical operations.

> (Confusingly, the NoSQL name corresponds to "not only SQL" rather than "no SQL.")

Backronyms ahoy!

> But the thing about abstraction in computer science, whether it's a high-level virtual machine-based programming language like Java or a bare-bones Unix shell, is that it always has a cost.

So does hard-baking a single storage and access pattern, which is that you get it wrong and have catastrophically slow access and aggregation times. A relational database is a generalist, query optimisers are generally good at guessing how best to traverse a normalised schema and the last time people used "abstractions are too costly!" as a serious argument it was in the days of rewriting desktop apps in assembler.

So our saas web app backend is written predominantly in Cache Objectscript, which is one of the few remaining implementations of MUMPS. We use it on a day to day basis, which I guess makes me a MUMPS developer.

As a programming language, it's a fairly standard procedural language. Cache introduces the ability for scoped java-like classes over the top of the standard procedural code and there's a SQL/ORM layer that at least allows a definition for the giant persistent hash table. That being said, a bunch of our code is in the legacy unscoped routines which is... interesting.

The two real downsides are the complete lack of libraries for it (imagine having to allocate time to port DEFLATE as part of your developement scheduling) and the fact that 20 years of schemaless, unvalidated, tightly coupled data access leads to a boatload of weird bugs.

Feel free to AMA.

How well does Cache interoperate with other languages?

Is it wrapped up in a library and consumed from another language that handles web requests/responses? Can you extend the Cache libraries by writing code in another language (even C) and importing it through a FFI?

There are bindings for most of the common languages to access Cache structures/functions (.Net, Java, Python, Perl, C/C++, Node.js) but not much the other way around. The only native to call out from Objectscript is via a low-level C interface.

At the moment we're solving the problem by calling out to a set of Golang microservices from Cache over http.

Cache itself actually has web support baked in, either via early-00's PHP style "echo directly down the pipe" or a newer MVC style framework they've built, but sadly stopped developing. Our product team made the choice to transition to their MVC style framework, another team built on ASP.Net so there are options.

Most clients interact with the database through a standard API like JDBC or REST, but there is an interface for calling functions written in C, and for getting into the database from C. It's kind of similar to Lua's C API. The Node.js binding is based on this interface.

Relevant WTF post: http://thedailywtf.com/articles/A_Case_of_the_MUMPS

Follow-ups:

http://thedailywtf.com/articles/MUMPS-Madness

http://thedailywtf.com/articles/Revenge-of-MUMPS-Madness!

"OPERATORS: No precedence, executed left to right, parenthesize as desired. 2+3*10 yields 50."

Wow. I mean... why?

Tiny little interpreter on early mini computers. You save RAM by not having to evaluate the whole expression.

And for a different but convergent take on the same approach to this problem space, here's the Pick Operating System, which evolved for business applications on minicomputers in the 1960s, along similar lines (global hash table database, built-in language):

https://en.wikipedia.org/wiki/Pick_operating_system

MUMPS can thus be seen as a member of a (loose) family of 1960s era systems.

My sister works for a place using Mumps. I read the manual she let me borrow and I decided I liked it better than the COBOL I was doing at the time.

Then I cried myself to sleep.

True story.

Funny this crops up. I've had to quickly get to grips with a somewhat related subject: a human readable healthcare interchange format called HL7 v2 (v3 has, somewhat rightfully, replaced the custom format with XML).

Here's an example:

MSH|^~\&|MegaReg|XYZHospC|SuperOE|XYZImgCtr|20060529090131-0500||ADT^A01^ADT_A01|01052901|P|2.5 EVN||200605290901||||200605290900 PID|||56782445^^^UAReg^PI||KLEINSAMPLE^BARRY^Q^JR||19620910|M||2028-9^^HL70005^RA99113^^XYZ|260 GOODWIN CREST DRIVE^^BIRMINGHAM^AL^35209^^M~NICKELL’S PICKLES^10000 W 100TH AVE^BIRMINGHAM^AL^35200^^O|||||||0105I30001^^^99DEF^AN PV1||I|W^389^1^UABH^^^^3||||12345^MORGAN^REX^J^^^MD^0010^UAMC^L||67890^GRAINGER^LUCY^X^^^MD^0010^UAMC^L|MED|||||A0||13579^ POTTER^SHERMAN^T^^^MD^0010^UAMC^L|||||||||||||||||||||||||||200605290900 OBX|1|NM|^Body Height||1.80|m^Meter^ISO+|||||F OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO+|||||F AL1|1||^ASPIRIN DG1|1||786.50^CHEST PAIN, UNSPECIFIED^I9|||A

Who said that it is suppose to be human readable?

HL7 v2 is actually fairly nice as a data structure. It's really easy to parse, compresses nicely, and is quite flexible in terms of schema. I'll grant you that the standard message schemas are terrible, but the data structure is elegant for what it needs to do.

>It's really easy to parse, compresses nicely, and is quite flexible in terms of schema

It fine in modern languages with good string parsing tools but if you have a v2 parser using simpler techniques in something that's difficult to debug with, it can be a nightmare when something goes slightly wrong. I'm not a fan of how we did it, but it has worked for 95% of cases and was coded in 1999, so I can't really complain all that much.

Part of the struggle is massaging message parses where a vendor is doing something slightly differently (or very terribly, like adding the lab's director on every fucking OBX segment as if the director will get fired mid-HL7 message and we gotta know about it).

That's what the bits and pieces of docs I read said it was. I'm fairly new to it as a format and "human readable" made me double take.

I've never seen a single place that uses v3 in the wild. Seems like everyone is using v2 still and talking about moving to FIHR.

Please tell me that's not real data.

No, I pulled it from Wikipedia. I wouldn't ever share real data, especially since it's the transmission of STI testing results.

Very telling about it's effectiveness though: you didn't spot "nickell's pickles" in there!

Oh, but I did :-) But given it's MUMPS, that could have been part of the file format...

>HL7 v2

Run, dude

Meditech runs on a variant of MUMPS: MAGIC (and its subsequent iterations). MAGIC was also developed by one of the researchers that made MUMPS. It's the only language I've come across that allows variable assignment during arithmetic.

Kind of wild this article appears now. I just yesterday was talking with a potential partner when they suddenly dropped MUMPS connectivity into the requirements. My first thought was, "Oh sh--!" I have some basic MUMPs knowledge but I certainly don't want to have my business depending on it!

Similar situation here, met with an associate who was trying to integrate with MUMPS.

And this issue with this is?

Earlier in life I ported MUMPS to the ATARI ST. I remember the syntax as being a little weird compared to all other languages I knew at the time. The persistence was a nice touch though, it allowed you work with data in a very intuitive manner.

MUMPS powers chunk of TD Ameritrade infrastructure. While sounds cool, this is just as scary as knowing that COBOL powers chunk of other banks infrastructures (Schwab, etc).

> In 2006 and 2012, Computerworld surveys found that over 60% of organizations used COBOL (more than C++ and Visual Basic .NET) and that for half of those, COBOL was used for the majority of their internal software. https://en.wikipedia.org/wiki/COBOL#Legacy

Also, COBOL got a spec update in 2014. It is not really a dead language.

> MUMPS concepts (directly) underly two of the largest contemporary NoSQL tools: GT.M and InterSystems Caché.

Huh?

And I still hear devs balk when I mention our (ehr) backend is written in perl. PERL!

How does this compare to languages with built in databases such as prolog?

Anyone who said that mumps was eradicated is kidding themselves.

Reply on news.ycombinator.com