The fact that ls can produce output that can't be reliably sorted using the sort utility shows that ls is actually doing three things instead of just one: Listing files, sorting the output and then displaying the output in a human readable format.

This is not to criticize ls though - I would imagine that no matter how small the operation (well aside of operations executed directly by the cpu), there's still the possibility to break it up into smaller parts.

At some point convenience just trumps modularity.

ls -lt is just more convenient than a (hypotetical) ls -t | sort | xargs fileinfo | display-fileinfo

On a related note: Windows PowerShell comes closer to this ideal by being able to send arbitrary objects between pipes. This still has the side-effect though that single commands become much more awkward to use as they don't produce output that's usable on its own.