Hacker News Reader

Comment by woah | Hacker News Reader

I implemented a credit check API connection to Equifax recently. The response data was encoded as a stream of offset-based text, with some offsets dynamically changing based on fields in the already-parsed data. Lots of work to write a parser for it.

We initially asked them if they had an updated version of this API using XML or JSON, and it turned into a call with several of their salespeople trying to upsell us on some complicated drag and drop rules engine that happened to return data as JSON. So we just stuck to the legacy API. They struck me as a pretty incompetent organization.

View on HN

That's not incompetent -- that's Enterprise!

A text file is not Enterprise. Real Enterprises would have a binary format embedded inside XML.

  <?xml version="1.0">
  <enterprise>[the octets of an access database, interpreted as latin-1 and then encoded into utf-8]</enterprise>

> interpreted as latin-1 and then encoded into utf-8

It troubles me how true this is.

You forgot the part where that XML is itself embedded inside another XML, and by "embedded" I mean "as a quoted text node". So:

   <enterprise>&lt;no-really-enterprise&gt; ...

That sounds like a very weird format.

Care to post a mini-example with fake data so we can better understand what you are describing?

It sounds like an X12-style EDI format. They'll frequently have fields (or parts of fields) that can enable alternative blocks that may be of a different size. I had to write and maintain EDI interfaces for four years at a major retailer: there's a good business in transforming those documents.

I guess something like:

    First name
    Last name
    Credit score
    Debtor bank
    Debts
    Bank balance

but:

    First name
    Middle name
    Last name
    Credit score
    Bank balance

So subsequent fields change depending on what fields came before. Something like that.

They are far from the only ones. Many times these horrible formats are covered by an NDA, so I can't even open source the parser if I wanted to :-(

It's not as weird as it sounds, you're parsing a text stream, not that much different in principle than parsing HTTP headers or network packets byte by byte.

Reply on news.ycombinator.com