The geniuses who led the project before me decided that the first step was to split the file into single payment chunks, then put each one into RabbitMQ, from where a different service would pick them up and parse them - then encode the parsed payments in JSON, and put them back in RabbitMQ, for a third process to pick up and handle. This was all for performance, of course.
I tried to explain that just parsing the whole file in one go would be much faster than splitting it up, writing it to a queue, and reading it back in. That it probably wouldn't be much slower than the splitting. But nobody was interested. Obviously distributing the work would be much faster.
It was a weird experience. My colleagues just had no ability to reason about the "physics" of what computers did. I think they mostly had Rails backgrounds, and I wonder if you just don't exercise that muscle much doing Rails.